Identity verification method and device, computer equipment and storage medium

Abstract

The invention belongs to the field of artificial intelligence, and relates to an identity verification method, which comprises the following steps: acquiring user voice and a face image of a user through intelligent glasses; performing voice enhancement processing on the user voice to obtain enhanced voice, and performing identity verification on the user according to the enhanced voice to obtain a first verification result; inputting the face image into an illumination invariant feature extraction model to obtain an illumination invariant feature of the face image, and performing identity verification on the user according to the illumination invariant feature to obtain a second verification result; extracting facial features of the face image, and performing identity verification on the user according to the facial features to obtain a third verification result; and generating an identity verification result of the user according to the first verification result, the second verification result and the third verification result. The invention further provides an identity verification device, computer equipment and a storage medium. According to the invention, the accuracy of identity verification based on the intelligent glasses is improved.

Description

Technical Field

[0001] The present application relates to the field of artificial intelligence technology, and in particular to an identity authentication method, apparatus, computer device and storage medium. Background Art

[0002] When withdrawing cash from a bank branch, smart glasses have significant advantages as a tool for identity verification. Smart glasses can collect real-time facial images and voices through built-in cameras and microphones, improving the convenience and security of the withdrawal process. Compared with traditional ATMs, smart glasses provide a more intuitive way of interaction, and users can complete operations without touching the screen.

[0003] However, in practical applications, smart glasses also face many challenges. First, the wearing stability of smart glasses affects the accuracy of facial recognition. If the glasses slip or shift, it may lead to inaccurate capture of facial features, which in turn affects identity authentication. The user's facial expressions, angles and obstructions may also affect the accuracy of facial recognition. Secondly, the environment of bank branches is noisy, and background noise will seriously affect the performance of voice recognition. In addition, the lighting conditions of bank branches are complex and changeable, which poses a challenge to the camera of smart glasses to capture clear facial images. All of these will affect the accuracy of identity authentication based on smart glasses. Summary of the invention

[0004] The purpose of the embodiments of the present application is to propose an identity authentication method, apparatus, computer device and storage medium to solve the accuracy of identity authentication based on smart glasses.

[0005] In order to solve the above technical problems, the present application embodiment provides an identity authentication method, which adopts the following technical solution:

[0006] Acquire the user's voice and face image through the smart glasses;

[0007] Performing voice enhancement processing on the user's voice to obtain enhanced voice, and performing identity authentication on the user according to the enhanced voice to obtain a first authentication result;

[0008] Inputting the face image into an illumination invariant feature extraction model to obtain illumination invariant features of the face image, and performing identity authentication on the user based on the illumination invariant features to obtain a second authentication result;

[0009] Extracting facial features of the face image, and performing identity verification on the user according to the facial features to obtain a third verification result;

[0010] An identity authentication result of the user is generated according to the first verification result, the second verification result and the third verification result.

[0011] In order to solve the above technical problems, the embodiment of the present application also provides an identity authentication device, which adopts the following technical solution:

[0012] A user acquisition module, used to acquire the user's voice and face image through the smart glasses;

[0013] A voice verification module, configured to perform voice enhancement processing on the user's voice to obtain enhanced voice, and perform identity verification on the user according to the enhanced voice to obtain a first verification result;

[0014] an illumination verification module, configured to input the face image into an illumination invariant feature extraction model to obtain illumination invariant features of the face image, and perform identity verification on the user based on the illumination invariant features to obtain a second verification result;

[0015] A facial verification module, used for extracting facial features of the face image, and performing identity verification on the user according to the facial features to obtain a third verification result;

[0016] A result generation module is used to generate an identity authentication result of the user according to the first verification result, the second verification result and the third verification result.

[0017] In order to solve the above technical problems, the embodiment of the present application further provides a computer device, which adopts the following technical solution:

[0018] Acquire the user's voice and face image through the smart glasses;

[0019] Performing voice enhancement processing on the user's voice to obtain enhanced voice, and performing identity authentication on the user according to the enhanced voice to obtain a first authentication result;

[0020] Inputting the face image into an illumination invariant feature extraction model to obtain illumination invariant features of the face image, and performing identity authentication on the user based on the illumination invariant features to obtain a second authentication result;

[0021] Extracting facial features of the face image, and performing identity verification on the user according to the facial features to obtain a third verification result;

[0022] An identity authentication result of the user is generated according to the first verification result, the second verification result and the third verification result.

[0023] In order to solve the above technical problems, the embodiment of the present application further provides a computer-readable storage medium, which adopts the following technical solution:

[0024] Acquire the user's voice and face image through the smart glasses;

[0025] Performing voice enhancement processing on the user's voice to obtain enhanced voice, and performing identity authentication on the user according to the enhanced voice to obtain a first authentication result;

[0026] Inputting the face image into an illumination invariant feature extraction model to obtain illumination invariant features of the face image, and performing identity authentication on the user based on the illumination invariant features to obtain a second authentication result;

[0027] Extracting facial features of the face image, and performing identity verification on the user according to the facial features to obtain a third verification result;

[0028] An identity authentication result of the user is generated according to the first verification result, the second verification result and the third verification result.

[0029] Compared with the prior art, the embodiments of the present application mainly have the following beneficial effects: obtaining the user's voice and facial image through smart glasses; performing voice enhancement processing on the user's voice to obtain enhanced voice, thereby improving the robustness of voiceprint recognition, accurately identifying the user's identity even in a noisy environment, and generating a first verification result; inputting the facial image into the illumination-invariant feature extraction model to obtain the illumination-invariant feature of the facial image, which overcomes the influence of ambient illumination changes on facial recognition, ensures the recognition stability under different illumination conditions, and ensures the accuracy of the second verification result; extracting the facial features of the facial image, improving the accuracy of identity recognition under complex facial postures and expression changes, and obtaining a third verification result; combining the first verification result, the second verification result, and the third verification result to generate the user's final identity authentication result, thereby reducing the risk of misidentification and deceptive attacks, and improving the accuracy of identity authentication based on smart glasses. BRIEF DESCRIPTION OF THE DRAWINGS

[0030] In order to more clearly illustrate the scheme in the present application, a brief introduction is given below to the drawings required for use in the description of the embodiments of the present application. Obviously, the drawings described below are some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0031] Figure 1 is an exemplary system architecture diagram to which the present application may be applied;

[0032] Figure 2 A flowchart of an embodiment of an identity authentication method according to the present application;

[0033] Figure 3is a schematic structural diagram of an embodiment of an identity verification device according to the present application;

[0034] Figure 4 It is a structural diagram of an embodiment of a computer device according to the present application. DETAILED DESCRIPTION

[0035] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as those commonly understood by technicians in the technical field of the present application; the terms used in the specification of the application herein are only for the purpose of describing specific embodiments and are not intended to limit the present application; the terms "including" and "having" and any variations thereof in the specification and claims of the present application and the above-mentioned drawings are intended to cover non-exclusive inclusions. The terms "first", "second", etc. in the specification and claims of the present application or the above-mentioned drawings are used to distinguish different objects, not to describe a specific order.

[0036] Reference to "embodiments" herein means that a particular feature, structure, or characteristic described in conjunction with the embodiments may be included in at least one embodiment of the present application. The appearance of the phrase in various locations in the specification does not necessarily refer to the same embodiment, nor is it an independent or alternative embodiment that is mutually exclusive with other embodiments. It is explicitly and implicitly understood by those skilled in the art that the embodiments described herein may be combined with other embodiments.

[0037] In order to enable those skilled in the art to better understand the solution of the present application, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the accompanying drawings.

[0038] like Figure 1 As shown, the system architecture 100 may include a terminal device 101, a network 102 and a server 103. The terminal device 101 may be a laptop 1011, a tablet computer 1012 or a mobile phone 1013. The network 102 is used to provide a medium for a communication link between the terminal device 101 and the server 103. The network 102 may include various connection types, such as wired, wireless communication links or optical fiber cables. In the present application, the terminal device 101 may also be smart glasses.

[0039] The user can use the terminal device 101 to interact with the server 103 through the network 102 to receive or send messages, etc. Various communication client applications can be installed on the terminal device 101, such as web browser applications, shopping applications, search applications, instant messaging tools, email clients, social platform software, etc.

[0040] The terminal device 101 can be various electronic devices with a display screen and supporting web browsing. In addition to the laptop computer 1011, the tablet computer 1012 or the mobile phone 1013, the terminal device 101 can also be an e-book reader, an MP3 player (Moving Picture Experts Group Audio Layer III), an MP4 (Moving Picture Experts Group Audio Layer IV) player, a laptop computer and a desktop computer, etc.

[0041] The server 103 may be a server that provides various services, such as a background server that provides support for a web page displayed on the terminal device 101 .

[0042] It should be noted that the identity authentication method provided in the embodiment of the present application is generally executed by a server, and accordingly, the identity authentication device is generally set in the server.

[0043] It should be understood that Figure 1 The number of terminal devices, networks and servers in the embodiment is only for illustration. Any number of terminal devices, networks and servers may be provided according to implementation requirements.

[0044] Continue to refer Figure 2 , shows a flow chart of an embodiment of an identity authentication method according to the present application. The identity authentication method comprises the following steps:

[0045] Step S201: Acquire the user's voice and face image through the smart glasses.

[0046] In this embodiment, the electronic device (eg, Figure 1 The server shown in the figure can communicate with the terminal device through a wired connection or a wireless connection. It should be noted that the above wireless connection method may include but is not limited to 3G / 4G / 5G connection, Wi-Fi connection, Bluetooth connection, Wi MAX connection, Zigbee connection, UWB (Ultra Wi-Fi Band) connection, and other wireless connection methods currently known or to be developed in the future.

[0047] Specifically, when a user performs operations such as withdrawing money at a bank branch, the user needs to be authenticated. The user can wear smart glasses, which capture the user's voice and face image.

[0048] The smart glasses can send the user's voice and face image to the backend server of the smart glasses, and the backend server sends the user's voice and face image to the bank's server. Alternatively, the smart glasses send the user's voice and face image to the terminal device of the bank branch, and the terminal device sends the user's voice and face image to the bank's server. The smart glasses can also directly send the user's voice and face image to the bank's server.

[0049] Step S202: Perform voice enhancement processing on the user's voice to obtain enhanced voice, and perform identity verification on the user based on the enhanced voice to obtain a first verification result.

[0050] Specifically, the user's voice is enhanced to eliminate background noise and other interference to ensure the clarity and integrity of the voice. The enhanced voice can be used for voiceprint recognition, extracting the voiceprint features in the voice and comparing them with the voiceprint feature library to obtain the first verification result.

[0051] Step S203, inputting the face image into the illumination invariant feature extraction model to obtain the illumination invariant features of the face image, and performing identity verification on the user based on the illumination invariant features to obtain a second verification result.

[0052] Specifically, the face image is input into the illumination-invariant feature extraction model. The model adjusts the illumination effect in the face image according to the detected illumination conditions, thereby extracting illumination-invariant features in the face. Illumination-invariant features are not affected by changes in ambient illumination, making identity authentication stable and accurate under various illumination conditions.

[0053] The illumination invariant feature is matched with the inventory invariant feature in the pre-established illumination invariant feature library, and the feature similarity is calculated. The identity authentication result of the illumination invariant feature dimension is determined based on the feature similarity. If there is a feature similarity greater than the set threshold, the identity authentication is successful, otherwise the identity authentication fails. By comparing the illumination invariant features, a second authentication result can be generated.

[0054] Step S204, extracting facial features of the face image, and performing identity verification on the user based on the facial features to obtain a third verification result.

[0055] Specifically, facial features of the face image are extracted. This process may involve processing steps such as standardization of two-dimensional images, three-dimensional reconstruction, posture estimation and posture compensation, and finally obtain the user's unique facial features. Based on the facial features, the third dimension of identity authentication is performed and a third verification result is generated. This step improves the accuracy of verification through multi-dimensional facial features such as three-dimensional geometric structure, texture and stereoscopic features.

[0056] Step S205: Generate an identity authentication result of the user according to the first verification result, the second verification result and the third verification result.

[0057] Specifically, a comprehensive analysis is performed based on the first verification result, the second verification result, and the third verification result to generate a final identity authentication result. Generally, if all three verification results are verified to be passed, an identity authentication result indicating that the user identity authentication is passed is generated.

[0058] This multimodal, multi-stage authentication method improves the accuracy and security of authentication and prevents misjudgments or attacks that may occur with a single authentication method.

[0059] In this embodiment, the user's voice and facial image are obtained through smart glasses; the user's voice is enhanced to obtain enhanced voice, which improves the robustness of voiceprint recognition, accurately identifies the user's identity even in a noisy environment, and generates a first verification result; the facial image is input into an illumination-invariant feature extraction model to obtain illumination-invariant features of the facial image, which overcome the influence of ambient lighting changes on facial recognition, ensure recognition stability under different lighting conditions, and ensure the accuracy of the second verification result; the facial features of the facial image are extracted to improve the accuracy of identity recognition under complex facial postures and expression changes, and obtain a third verification result; the first verification result, the second verification result, and the third verification result are combined to generate the user's final identity authentication result, which reduces the risk of misidentification and deceptive attacks and improves the accuracy of identity authentication based on smart glasses.

[0060] Furthermore, the above step S202 may include: preprocessing the user's voice to obtain a first voice; extracting frequency domain features of the first voice through a convolutional neural network; performing time domain modeling on the frequency domain features through a gated recurrent unit to obtain an enhanced voice; extracting voiceprint features of the enhanced voice through a voiceprint recognition model; comparing the voiceprint features with each stock voiceprint in a voiceprint feature library to authenticate the user and obtain a first verification result.

[0061] Specifically, the user's voice is first preprocessed, including removing noise, normalizing volume, and filtering unnecessary frequency bands to ensure the quality and consistency of the input voice. After the preprocessing, the first voice is obtained.

[0062] The first speech is input into a convolutional neural network (CNN) to extract the frequency domain features of the speech signal. CNN captures the potential and representative frequency patterns in the speech signal through layer-by-layer convolution and pooling operations. These frequency domain features are critical for speech enhancement and voiceprint recognition. They effectively compress the frequency information of the speech and retain the part that is recognizable to the user's identity.

[0063] The frequency domain features are input into the Gated Recurrent Unit (GRU) network for time domain modeling. GRU is a recurrent neural network (RNN) that can process sequence data and capture the temporal dependencies in speech signals. Through time domain modeling, the temporal dynamic information of the speech signal is combined with the frequency domain features to generate enhanced speech. The enhanced speech is optimized at the frequency level and is more stable and accurate in terms of temporal continuity.

[0064] The enhanced speech is input into the voiceprint recognition model to extract its voiceprint features. Voiceprint features are the unique attributes of the user's voice, including timbre, pronunciation habits, vocal tract shape, etc. These features are highly distinguishable in biometrics.

[0065] The voiceprint feature is compared with each stock voiceprint in the voiceprint feature library, for example, the similarity score between the voiceprint feature and each stock voiceprint is calculated by cosine similarity or Euclidean distance. If there is a similarity score higher than the set threshold, the verification is successful, confirming that the user's voice comes from the registered user; if the similarity score is lower than the threshold, the verification fails. After the comparison, a first verification result is generated, indicating the result of the voice identity verification.

[0066] In this embodiment, preprocessing eliminates environmental noise and inconsistencies in voice collection, ensuring the high quality of input voice; uses a convolutional neural network to extract frequency domain features of voice and captures the most distinctive frequency information in voice; uses a gated recurrent unit to perform time domain modeling, processes the temporal dynamic characteristics of voice, enhances the continuity and authenticity of voice, and improves the effect of voice enhancement; extracts voiceprint features and compares them with the stored voiceprints in the voiceprint feature library, so as to quickly and accurately identify the user's identity, and maintains efficient and accurate recognition capabilities in noisy or complex voice environments.

[0067] Furthermore, the above-mentioned step of inputting the facial image into the illumination invariant feature extraction model to obtain the illumination invariant feature of the facial image may include: performing illumination detection on the facial image to obtain the illumination condition of the facial image; inputting the facial image into the illumination invariant feature extraction model to perform feature transformation on the facial image according to the illumination condition through the illumination invariant feature extraction model to obtain a first facial image; and performing feature extraction on the first facial image through the illumination invariant feature extraction model to obtain the illumination invariant feature.

[0068] Specifically, facial images may be affected by lighting conditions, which may manifest as uneven lighting or shadow areas. Image processing technology is used to perform lighting detection on facial images to detect their lighting conditions, such as by calculating the brightness histogram of the image, detecting the light intensity and the position of the light source, etc. In one embodiment, the lighting conditions may include uniform lighting (the light distribution is relatively uniform, and no obvious shadows or bright spots will be produced on the face), strong light (direct illumination by the light source may cause overexposure of facial highlight areas, such as the forehead and the tip of the nose), shadows (improper position of the light source, resulting in obvious shadow areas, such as under the eye sockets), and backlight (the light source is located behind the shooting angle, resulting in insufficient lighting of facial features).

[0069] Different lighting conditions will cause changes in the visual features of face images, so different methods are needed to adjust these changes to ensure the stability and consistency of facial features.

[0070] The face image is input into the illumination invariant feature extraction model. The model transforms the image features by adjusting the brightness, contrast and color balance of the image. Depending on the illumination conditions, the model will adopt different transformation strategies. For example, under uniform illumination conditions, additional illumination compensation is usually not required, and the standard feature extraction algorithm (such as principal component analysis) is directly used to extract facial features; under strong light conditions, image enhancement techniques (such as histogram equalization and gamma correction) are used to adjust the overexposed area and reduce the impact of highlights to balance the brightness distribution; under shadow conditions, shadow compensation techniques (such as local contrast enhancement and shadow compensation filtering) are applied to reduce the impact of shadow areas on facial features; under backlight conditions, backlight compensation techniques (such as backlight correction and image compensation) are used to increase the brightness and visibility of the facial area, and adjust the brightness of the facial area in the image to a reasonable range for feature extraction. Through these feature transformations, a first face image with uniform illumination is generated, making the facial features more prominent and stable.

[0071] The illumination-invariant feature extraction model is used to further extract features of the first face image to obtain illumination-invariant features. The information reflected by the illumination-invariant features can remain consistent under various lighting conditions and can be used for identity authentication to ensure accurate identification of users under different lighting conditions.

[0072] In this embodiment, illumination detection is performed on the facial image to identify the influence of the current lighting environment, thereby providing a basis for subsequent image processing; the image is input into an illumination-invariant feature extraction model, and appropriate image transformation techniques are applied according to the lighting conditions to eliminate interference caused by illumination changes, thereby ensuring that the extracted facial features still have high stability in various subsequent lighting environments; feature extraction is performed on the first facial image to obtain illumination-invariant features, which are consistent under different lighting conditions, can improve the accuracy of identity authentication, and reduce recognition errors caused by illumination changes.

[0073] Furthermore, the above-mentioned step of extracting features from the first face image through the illumination invariant feature extraction model to obtain illumination invariant features may include: extracting facial key points and geometric features of the first face image through the illumination invariant feature extraction model; converting the facial key points and geometric features into feature vectors to obtain illumination invariant features.

[0074] Specifically, the facial key points are extracted from the first face image through the illumination-invariant feature extraction model, including the corners of the eyes, the tip of the nose, the corners of the mouth and other significant facial feature points. The positions of these key points are relatively fixed and will not shift significantly due to illumination changes. They can remain consistent under different illumination conditions.

[0075] Based on the facial key points, geometric features are further extracted, such as the spatial relationship and distance between facial key points, such as the distance between eyes, the distance from the nose tip to the corner of the mouth, etc. These geometric features are also not easily affected by changes in lighting and can stably reflect the geometric structure of the face.

[0076] The extracted facial key points and geometric features are converted into feature vectors, and the feature vectors are used as lighting-invariant features for identity authentication, ensuring that users can still be accurately identified under different lighting conditions.

[0077] In this embodiment, facial key points in the first face image are extracted through an illumination-invariant feature extraction model, and then facial geometric features are extracted. Facial key points and facial geometric features have strong anti-interference capabilities against illumination changes. They are converted into feature vectors and used as illumination-invariant features for identity authentication. Under different lighting conditions, users can be effectively identified, reducing recognition errors caused by illumination changes and improving the robustness and reliability of identity authentication.

[0078] Furthermore, the above-mentioned step of extracting facial features of a face image may include: normalizing the face image to obtain a two-dimensional face image; performing three-dimensional reconstruction on the two-dimensional face image to obtain a three-dimensional model of the face; calculating the three-dimensional model based on a posture estimation algorithm to obtain posture parameters of the face; performing posture compensation on the two-dimensional face image according to the posture parameters to obtain a two-dimensional corrected image, and performing posture compensation on the three-dimensional model according to the posture parameters to obtain a three-dimensional corrected model; and performing feature extraction on the two-dimensional corrected image and the three-dimensional corrected model to obtain facial features.

[0079] Specifically, the facial image is subjected to standardization processing such as denoising, contrast adjustment, and size adjustment to obtain a two-dimensional facial image. The two-dimensional facial image is reconstructed in three dimensions to generate a three-dimensional model that reflects the geometric structure of the user's face. The three-dimensional model retains the depth information of the face and also provides additional geometric details, which is conducive to more accurate analysis of the user's facial features.

[0080] Apply a posture estimation algorithm (such as a rotation matrix or a deep learning model) to the 3D model to calculate the posture parameters of the face, including pitch, yaw, and roll, etc. These posture parameters reflect the orientation and angle of the user's head when the image was taken.

[0081] According to these posture parameters, the original two-dimensional face image is posture compensated to obtain a two-dimensional corrected image; similarly, the three-dimensional model is posture compensated to generate a three-dimensional corrected model. Posture compensation is to correct facial images at different angles to similar angles so as to maintain consistency during feature extraction.

[0082] For two-dimensional facial images, rotation, scaling, perspective correction and other transformations are performed on the two-dimensional facial images according to the posture parameters (pitch angle, yaw angle, roll angle), so that the face in the image is "straightened" to a standard perspective close to the front, reducing the distortion of facial features caused by the shooting angle, and facilitating subsequent comparison with facial images under the standard frontal perspective.

[0083] For a three-dimensional model, the posture of the model is adjusted in the three-dimensional space to align it with the standard frontal view. In fact, the three-dimensional model is rotated so that the frontal orientation of the model is consistent with the view.

[0084] After posture compensation is completed, feature extraction is performed on the two-dimensional corrected image and the three-dimensional corrected model to obtain facial features, which can effectively reflect the uniqueness of the user's face.

[0085] In this embodiment, the standardization process ensures the consistency of the input image, and subsequent processing can be performed under a unified standard, thereby reducing recognition errors caused by differences in input image quality; three-dimensional reconstruction provides depth information and geometric details for face recognition, and can more comprehensively analyze the user's facial features; posture estimation and compensation solves the impact of changes in the user's head posture on recognition accuracy. Through posture compensation, the two-dimensional face image and the three-dimensional model are corrected to a standard posture, and face images at different angles can obtain consistent feature extraction results; feature extraction is performed on the corrected image and model, thereby improving the accuracy of feature extraction.

[0086] Furthermore, the above-mentioned step of extracting features from the two-dimensional corrected image and the three-dimensional corrected model to obtain facial features may include: extracting texture features and depth features from the two-dimensional corrected image; extracting three-dimensional geometric features and stereo features from the three-dimensional corrected model; mapping the texture features to the three-dimensional corrected model to obtain a composite model; extracting composite geometric features and composite texture features of the composite model; and determining the texture features, depth features, three-dimensional geometric features, stereo features, composite geometric features and composite texture features as facial features.

[0087] Specifically, texture features and depth features are extracted from the two-dimensional corrected image. Facial texture features are extracted from the compensated two-dimensional image through a convolutional neural network (CNN). These texture features include skin texture, local feature points (such as the shape and details of the eyes, nose, and mouth), etc. Depth features are higher-level feature vectors extracted through a deep learning model. Depth features can remain stable despite changes in facial posture and expression.

[0088] Extract 3D geometric features and stereo features from the 3D corrected model. 3D geometric features are a detailed description of the facial structure, including geometric information such as the curvature, edges, and contours of the face; stereo features include the relative positions and relationships of various parts of the face in 3D space. These features can reflect the inherent morphology of the user's face and are not affected by changes in expression or posture.

[0089] In order to better combine 2D and 3D information, the texture features extracted from the 2D corrected image are mapped to the 3D model to form a composite model. The composite model effectively integrates the surface details of the 2D image with the spatial structure of the 3D model, providing a more three-dimensional and realistic facial description.

[0090] Composite geometric features and composite texture features are extracted from the composite model. Composite geometric features combine the morphology of the 3D corrected model with the mapped texture features; composite texture features are surface detail information enhanced on the basis of the 3D structure. Composite features are more expressive than 2D or 3D features.

[0091] All the features extracted above, including texture features, depth features, three-dimensional geometric features, stereo features, composite geometric features and composite texture features, are determined as facial features.

[0092] In this embodiment, texture features and depth features are extracted from the two-dimensional corrected image to capture surface details and depth information; three-dimensional geometric features and stereoscopic features are extracted from the three-dimensional corrected model to enhance the understanding of facial structure, especially the spatial relationship and morphology of various facial parts; the texture features of the two-dimensional corrected image are mapped to the three-dimensional corrected model to form a composite model, which combines the advantages of two and three dimensions, so that the extracted composite geometric features and composite texture features have stronger description and distinction capabilities; the comprehensive application of these features constructs an accurate and stable facial feature vector, which greatly improves the reliability and robustness of identity authentication.

[0093] It should be emphasized that in order to further ensure the privacy and security of the above-mentioned user voice and facial images, the above-mentioned user voice and facial images can also be stored in a blockchain node.

[0094] The blockchain referred to in this application is a new application model of computer technologies such as distributed data storage, peer-to-peer transmission, consensus mechanism, encryption algorithm, etc. Blockchain is essentially a decentralized database, a string of data blocks generated by cryptographic methods. Each data block contains a batch of network transaction information, which is used to verify the validity of its information (anti-counterfeiting) and generate the next block. Blockchain can include the underlying blockchain platform, platform product service layer, and application service layer.

[0095] The embodiments of the present application can acquire and process relevant data based on artificial intelligence technology. Artificial Intelligence (AI) is the theory, method, technology and application system that uses digital computers or machines controlled by digital computers to simulate, extend and expand human intelligence, perceive the environment, acquire knowledge and use knowledge to obtain the best results.

[0096] AI basic technologies generally include sensors, dedicated AI chips, cloud computing, distributed storage, big data processing technology, operation / interaction systems, mechatronics, etc. AI software technologies mainly include computer vision technology, robotics technology, biometrics technology, speech processing technology, natural language processing technology, and machine learning / deep learning.

[0097] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiments can be implemented by instructing the relevant hardware through computer-readable instructions, and the computer-readable instructions can be stored in a computer-readable storage medium. When the program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, the aforementioned storage medium can be a non-volatile storage medium such as a disk, an optical disk, a read-only memory (ROM), or a random access memory (RAM).

[0098] It should be understood that, although the steps in the flowchart of the accompanying drawings are displayed in sequence as indicated by the arrows, these steps are not necessarily executed in sequence in the order indicated by the arrows. Unless otherwise specified herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least a part of the steps in the flowchart of the accompanying drawings may include multiple sub-steps or multiple stages, and these sub-steps or stages are not necessarily executed at the same time, but can be executed at different times, and their execution order is not necessarily sequential, but can be executed in turn or alternately with other steps or at least a part of the sub-steps or stages of other steps.

[0099] Further references Figure 3 , as a response to the above Figure 2 The present application provides an embodiment of an identity authentication device, and the device embodiment is Figure 2 Corresponding to the method embodiment shown, the device can be specifically applied to various electronic devices.

[0100] like Figure 3 As shown, the identity verification device 300 described in this embodiment includes: a user acquisition module 301, a voice verification module 302, a lighting verification module 303, a face verification module 304 and a result generation module 305, wherein:

[0101] The user acquisition module 301 is used to acquire the user's voice and face image through the smart glasses.

[0102] The voice verification module 302 is used to perform voice enhancement processing on the user's voice to obtain enhanced voice, and to perform identity verification on the user according to the enhanced voice to obtain a first verification result.

[0103] The illumination verification module 303 is used to input the face image into the illumination invariant feature extraction model to obtain the illumination invariant features of the face image, and perform identity verification on the user based on the illumination invariant features to obtain a second verification result.

[0104] The face verification module 304 is used to extract facial features of the face image and perform identity verification on the user based on the facial features to obtain a third verification result.

[0105] The result generation module 305 is used to generate a user identity authentication result according to the first verification result, the second verification result and the third verification result.

[0106] In this embodiment, the user's voice and facial image are obtained through smart glasses; the user's voice is enhanced to obtain enhanced voice, which improves the robustness of voiceprint recognition, accurately identifies the user's identity even in a noisy environment, and generates a first verification result; the facial image is input into an illumination-invariant feature extraction model to obtain illumination-invariant features of the facial image, which overcome the influence of ambient lighting changes on facial recognition, ensure recognition stability under different lighting conditions, and ensure the accuracy of the second verification result; the facial features of the facial image are extracted to improve the accuracy of identity recognition under complex facial postures and expression changes, and obtain a third verification result; the first verification result, the second verification result, and the third verification result are combined to generate the user's final identity authentication result, which reduces the risk of misidentification and deceptive attacks and improves the accuracy of identity authentication based on smart glasses.

[0107] In some optional implementations of this embodiment, the voice verification module 302 may include:

[0108] The preprocessing submodule is used to preprocess the user's voice to obtain the first voice.

[0109] The frequency domain extraction submodule is used to extract the frequency domain features of the first speech through a convolutional neural network.

[0110] The time domain modeling submodule is used to perform time domain modeling on the frequency domain features through a gated recurrent unit to obtain enhanced speech.

[0111] The voiceprint extraction submodule is used to extract the voiceprint features of the enhanced speech through the voiceprint recognition model.

[0112] The voiceprint comparison submodule is used to compare the voiceprint feature with each stored voiceprint in the voiceprint feature library to authenticate the user and obtain a first verification result.

[0113] In some optional implementations of this embodiment, the illumination verification module 303 may include:

[0114] The illumination detection submodule is used to perform illumination detection on the face image to obtain the illumination conditions of the face image.

[0115] The image transformation submodule is used to input the face image into the illumination invariant feature extraction model so as to perform feature transformation on the face image according to the illumination conditions through the illumination invariant feature extraction model to obtain the first face image.

[0116] The illumination extraction submodule is used to extract features from the first face image through an illumination invariant feature extraction model to obtain illumination invariant features.

[0117] In some optional implementations of this embodiment, the illumination extraction submodule may include:

[0118] The extraction unit is used to extract facial key points and geometric features of the first face image through an illumination invariant feature extraction model.

[0119] The conversion unit is used to convert facial key points and geometric features into feature vectors to obtain illumination invariant features.

[0120] In some optional implementations of this embodiment, the face verification module 304 may include:

[0121] The standardization processing submodule is used to perform standardization processing on the face image to obtain a two-dimensional face image.

[0122] The 3D reconstruction submodule is used to perform 3D reconstruction on a 2D face image to obtain a 3D model of the face.

[0123] The posture calculation submodule is used to calculate the three-dimensional model based on the posture estimation algorithm to obtain the posture parameters of the face.

[0124] The posture compensation submodule is used to perform posture compensation on the two-dimensional face image according to the posture parameters to obtain a two-dimensional corrected image, and to perform posture compensation on the three-dimensional model according to the posture parameters to obtain a three-dimensional corrected model.

[0125] The feature extraction submodule is used to extract features from the two-dimensional corrected image and the three-dimensional corrected model to obtain facial features.

[0126] In some optional implementations of this embodiment, the feature extraction submodule may include:

[0127] The two-dimensional extraction unit is used to extract texture features and depth features from the two-dimensional corrected image.

[0128] The three-dimensional extraction unit is used to extract three-dimensional geometric features and stereo features from the three-dimensional corrected model.

[0129] The texture mapping unit is used to map the texture features to the three-dimensional modified model to obtain a composite model.

[0130] The composite extraction unit is used to extract composite geometric features and composite texture features of the composite model.

[0131] The feature determination unit is used to determine texture features, depth features, three-dimensional geometric features, stereo features, composite geometric features and composite texture features as facial features.

[0132] To solve the above technical problems, the present application also provides a computer device. Figure 4 , Figure 4 This is a basic structural block diagram of the computer device in this embodiment.

[0133] The computer device 4 includes a memory 41, a processor 42, and a network interface 43 that are interconnected and communicated through a system bus. It should be noted that the figure only shows a computer device 4 having a memory 41, a processor 42, and a network interface 43, but it should be understood that it is not required to implement all the components shown, and more or fewer components can be implemented instead. Among them, those skilled in the art can understand that the computer device here is a device that can automatically perform numerical calculations and / or information processing according to pre-set or stored instructions, and its hardware includes but is not limited to a microprocessor, an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), a digital processor (DSP), an embedded device, etc.

[0134] The computer device may be a computing device such as a desktop computer, a notebook, a PDA, a cloud server, etc. The computer device may interact with a user through a keyboard, a mouse, a remote controller, a touch pad, or a voice control device.

[0135] The memory 41 includes at least one type of readable storage medium, and the readable storage medium includes flash memory, hard disk, multimedia card, card-type memory (for example, SD or DX memory, etc.), random access memory (RAM), static random access memory (SRAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), programmable read-only memory (PROM), magnetic memory, disk, optical disk, etc. In some embodiments, the memory 41 can be an internal storage unit of the computer device 4, such as a hard disk or memory of the computer device 4. In other embodiments, the memory 41 can also be an external storage device of the computer device 4, such as a plug-in hard disk equipped on the computer device 4, a smart memory card (Smart Media Card, SMC), a secure digital (Secure Digital, SD) card, a flash card (FlashCard), etc. Of course, the memory 41 can also include both the internal storage unit of the computer device 4 and its external storage device. In this embodiment, the memory 41 is generally used to store the operating system and various application software installed on the computer device 4, such as computer-readable instructions of the identity authentication method, etc. In addition, the memory 41 can also be used to temporarily store various types of data that have been output or are to be output.

[0136] The processor 42 may be a central processing unit (CPU), a controller, a microcontroller, a microprocessor, or other data processing chip in some embodiments. The processor 42 is generally used to control the overall operation of the computer device 4. In this embodiment, the processor 42 is used to run computer-readable instructions stored in the memory 41 or process data, such as computer-readable instructions for running the identity authentication method.

[0137] The network interface 43 may include a wireless network interface or a wired network interface. The network interface 43 is generally used to establish a communication connection between the computer device 4 and other electronic devices.

[0138] In this embodiment, the user's voice and facial image are obtained through smart glasses; the user's voice is enhanced to obtain enhanced voice, which improves the robustness of voiceprint recognition, accurately identifies the user's identity even in a noisy environment, and generates a first verification result; the facial image is input into an illumination-invariant feature extraction model to obtain illumination-invariant features of the facial image, which overcome the influence of ambient lighting changes on facial recognition, ensure recognition stability under different lighting conditions, and ensure the accuracy of the second verification result; the facial features of the facial image are extracted to improve the accuracy of identity recognition under complex facial postures and expression changes, and obtain a third verification result; the first verification result, the second verification result, and the third verification result are combined to generate the user's final identity authentication result, which reduces the risk of misidentification and deceptive attacks and improves the accuracy of identity authentication based on smart glasses.

[0139] The present application also provides another implementation, namely, providing a computer-readable storage medium, wherein the computer-readable storage medium stores computer-readable instructions, and the computer-readable instructions can be executed by at least one processor to enable the at least one processor to perform the steps of the identity authentication method as described above.

[0140] In this embodiment, the user's voice and facial image are obtained through smart glasses; the user's voice is enhanced to obtain enhanced voice, which improves the robustness of voiceprint recognition, accurately identifies the user's identity even in a noisy environment, and generates a first verification result; the facial image is input into an illumination-invariant feature extraction model to obtain illumination-invariant features of the facial image, which overcome the influence of ambient lighting changes on facial recognition, ensure recognition stability under different lighting conditions, and ensure the accuracy of the second verification result; the facial features of the facial image are extracted to improve the accuracy of identity recognition under complex facial postures and expression changes, and obtain a third verification result; the first verification result, the second verification result, and the third verification result are combined to generate the user's final identity authentication result, which reduces the risk of misidentification and deceptive attacks and improves the accuracy of identity authentication based on smart glasses.

[0141] Through the description of the above implementation methods, those skilled in the art can clearly understand that the above-mentioned embodiment methods can be implemented by means of software plus a necessary general hardware platform, and of course by hardware, but in many cases the former is a better implementation method. Based on such an understanding, the technical solution of the present application, or the part that contributes to the prior art, can be embodied in the form of a software product, which is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk), and includes a number of instructions for a terminal device (which can be a mobile phone, computer, server, air conditioner, or network device, etc.) to execute the methods described in each embodiment of the present application.

[0142] Obviously, the embodiments described above are only some embodiments of the present application, rather than all embodiments. The preferred embodiments of the present application are given in the accompanying drawings, but they do not limit the patent scope of the present application. The present application can be implemented in many different forms. On the contrary, the purpose of providing these embodiments is to make the understanding of the disclosure of the present application more thorough and comprehensive. Although the present application is described in detail with reference to the aforementioned embodiments, for those skilled in the art, it is still possible to modify the technical solutions recorded in the aforementioned specific implementation methods, or to perform equivalent replacement of some of the technical features therein. Any equivalent structure made using the contents of the specification and drawings of this application, directly or indirectly used in other related technical fields, is similarly within the scope of patent protection of this application.

Claims

1. An identity authentication method, characterized in that: The steps include: Acquire the user's voice and face image through the smart glasses; Performing voice enhancement processing on the user's voice to obtain enhanced voice, and performing identity authentication on the user according to the enhanced voice to obtain a first authentication result; Inputting the face image into an illumination invariant feature extraction model to obtain illumination invariant features of the face image, and performing identity authentication on the user based on the illumination invariant features to obtain a second authentication result; Extracting facial features of the face image, and performing identity verification on the user according to the facial features to obtain a third verification result; An identity authentication result of the user is generated according to the first verification result, the second verification result and the third verification result.

2. The identity authentication method according to claim 1, characterized in that: The step of performing voice enhancement processing on the user's voice to obtain enhanced voice, and performing identity authentication on the user according to the enhanced voice to obtain a first authentication result comprises: Preprocessing the user voice to obtain a first voice; Extracting frequency domain features of the first speech through a convolutional neural network; Performing time domain modeling on the frequency domain features through a gated recurrent unit to obtain enhanced speech; Extracting voiceprint features of the enhanced speech through a voiceprint recognition model; The voiceprint feature is compared with each stock voiceprint in a voiceprint feature library to authenticate the user and obtain a first authentication result.

3. The identity authentication method according to claim 1, characterized in that: The step of inputting the face image into the illumination invariant feature extraction model to obtain the illumination invariant features of the face image comprises: Performing light detection on the face image to obtain the light condition of the face image; Inputting the face image into an illumination invariant feature extraction model, so as to perform feature transformation on the face image according to the illumination condition through the illumination invariant feature extraction model to obtain a first face image; The illumination invariant feature extraction model is used to extract features from the first face image to obtain illumination invariant features.

4. The identity authentication method according to claim 3, characterized in that: The step of extracting features from the first face image using the illumination invariant feature extraction model to obtain illumination invariant features comprises: Extracting facial key points and geometric features of the first face image by using the illumination invariant feature extraction model; The facial key points and the geometric features are converted into feature vectors to obtain illumination invariant features.

5. The identity authentication method according to claim 1, characterized in that: The step of extracting facial features of the face image comprises: Performing standardization processing on the facial image to obtain a two-dimensional facial image; Performing three-dimensional reconstruction on the two-dimensional face image to obtain a three-dimensional model of the face; Calculating the three-dimensional model based on a posture estimation algorithm to obtain posture parameters of the face; Performing posture compensation on the two-dimensional face image according to the posture parameters to obtain a two-dimensional corrected image, and performing posture compensation on the three-dimensional model according to the posture parameters to obtain a three-dimensional corrected model; Feature extraction is performed on the two-dimensional corrected image and the three-dimensional corrected model to obtain facial features.

6. The identity authentication method according to claim 5, characterized in that: The step of extracting features from the two-dimensional corrected image and the three-dimensional corrected model to obtain facial features comprises: Extracting texture features and depth features from the two-dimensional corrected image; Extracting three-dimensional geometric features and stereoscopic features from the three-dimensional corrected model; Mapping the texture features to the three-dimensional corrected model to obtain a composite model; Extracting composite geometric features and composite texture features of the composite model; The texture feature, the depth feature, the three-dimensional geometric feature, the stereoscopic feature, the composite geometric feature and the composite texture feature are determined as facial features.

7. An identity verification device, characterized in that: include: A user acquisition module, used to acquire the user's voice and face image through the smart glasses; A voice verification module, configured to perform voice enhancement processing on the user's voice to obtain enhanced voice, and perform identity verification on the user according to the enhanced voice to obtain a first verification result; an illumination verification module, configured to input the face image into an illumination invariant feature extraction model to obtain illumination invariant features of the face image, and perform identity verification on the user based on the illumination invariant features to obtain a second verification result; A facial verification module, used for extracting facial features of the face image, and performing identity verification on the user according to the facial features to obtain a third verification result; A result generating module is used to generate the identity authentication result of the user according to the first verification result, the second verification result and the third verification result.

8. The identity verification device according to claim 7, characterized in that: The facial verification module comprises: A standardization processing submodule, used for performing standardization processing on the face image to obtain a two-dimensional face image; A three-dimensional reconstruction submodule, used to perform three-dimensional reconstruction on the two-dimensional face image to obtain a three-dimensional model of the face; A posture calculation submodule, used to calculate the three-dimensional model based on a posture estimation algorithm to obtain posture parameters of the face; a posture compensation submodule, configured to perform posture compensation on the two-dimensional face image according to the posture parameters to obtain a two-dimensional corrected image, and perform posture compensation on the three-dimensional model according to the posture parameters to obtain a three-dimensional corrected model; The feature extraction submodule is used to extract features from the two-dimensional corrected image and the three-dimensional corrected model to obtain facial features.

9. A computer device, characterized in that: It comprises a memory and a processor, wherein the memory stores computer-readable instructions, and when the processor executes the computer-readable instructions, the steps of the identity authentication method as described in any one of claims 1 to 6 are implemented.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer-readable instructions, which, when executed by a processor, implement the steps of the identity authentication method according to any one of claims 1 to 6.

Images