Identity verification method and device, electronic equipment and storage medium

Abstract

The embodiment of the invention provides an identity verification method and device, electronic equipment and a storage medium, and belongs to the technical field of financial science and technology. The method comprises the following steps: acquiring multi-modal biological recognition data of a user in a current environment; performing feature extraction on the face image through a convolutional neural network to obtain face features; performing feature extraction on the iris image and the fingerprint information through a support vector machine to generate iris features and fingerprint features; identifying the face posture of the user in the current environment and the environment type of the current environment; adjusting weight values corresponding to the face features, the iris features and the fingerprint features according to the face postures and the environment types; and performing weighted fusion on the face feature, the iris feature and the fingerprint feature according to the weight value, and generating an identity verification result of the corresponding user in the current environment. According to the embodiment of the invention, the security and accuracy of user identity verification can be improved, and identity verification can be intelligently and accurately carried out on users in different complex environments in a financial scene.

Description

Technical Field

[0001] The present application relates to the field of financial technology (Fi ntech) technology, and in particular to an identity authentication method and device, an electronic device and a storage medium. Background Art

[0002] Currently, when users authenticate their identity, it is done by identifying their biometrics. Biometric-based authentication refers to authentication based on unique features of each person's body, such as face, fingerprint, iris, etc. Existing authentication methods use single-modal biometrics or simply superimposed multi-modal biometrics technology to achieve authentication. However, single-modal biometrics technology is not secure enough, and simply superimposed multi-modal biometrics technology cannot intelligently adapt to the different complex environments in which users authenticate their identity, resulting in poor recognition accuracy and robustness.

[0003] In financial scenarios, such as bank account opening, loan approval, and large-value transfers, where high security is required, existing identity authentication methods have the above-mentioned technical defects, which poses the risk of identity theft and fraud. Based on this, in financial scenarios, how to improve the security and accuracy of user identity authentication has become a technical problem that needs to be solved urgently. Summary of the invention

[0004] The main purpose of the embodiments of the present application is to propose an identity authentication method and device, an electronic device and a storage medium, which can improve the security and accuracy of user identity authentication. Especially in financial scenarios, users in different complex environments can be authenticated intelligently and accurately, which helps to reduce the risk of identity theft and fraud, improve the security and accuracy of financial services, and improve customer satisfaction, thereby enhancing the competitiveness of financial enterprises.

[0005] To achieve the above purpose, a first aspect of an embodiment of the present application proposes an identity authentication method, the method comprising:

[0006] Acquire multimodal biometric data of the user in the current environment, wherein the multimodal biometric data includes a face image, an iris image, and fingerprint information;

[0007] Extracting features from the facial image using a convolutional neural network to obtain facial features;

[0008] Extracting features from the iris image and the fingerprint information using a support vector machine to generate iris features and fingerprint features;

[0009] Recognize the facial posture of the user in the current environment and the environment type of the current environment;

[0010] adjusting the weight values ​​corresponding to the facial features, the iris features, and the fingerprint features respectively according to the facial posture and the environment type;

[0011] The facial features, the iris features and the fingerprint features are weightedly fused according to the weight values ​​to generate an identity verification result corresponding to the user in the current environment.

[0012] In some embodiments, the identifying the facial posture of the user in the current environment and the environment type of the current environment includes:

[0013] Determining facial posture parameters of the user in a current environment according to the facial image, wherein the facial image is obtained by photographing the user with a camera;

[0014] Inputting the facial posture parameters into a preset facial posture identification module to identify the facial posture of the user in the current environment;

[0015] Acquiring environmental condition parameters of the current environment through an environmental sensor;

[0016] The environmental condition parameters are input into a preset environment identification module to identify the type of environment in which the user is currently located.

[0017] In some embodiments, adjusting the weight values ​​corresponding to the facial features, the iris features, and the fingerprint features according to the facial posture and the environment type includes:

[0018] determining a scene mode according to the facial gesture and the environment type;

[0019] Determining a weight value adjustment strategy output by a preset decision fusion layer according to the scene mode;

[0020] The first weight value corresponding to the facial feature, the second weight value corresponding to the iris feature, and the third weight value corresponding to the fingerprint feature are adjusted based on the weight value adjustment strategy.

[0021] In some embodiments, the weight value adjustment strategy includes:

[0022] When it is determined that the facial posture is facing the camera and the environment type is a high-light environment, increase the first weight value corresponding to the facial feature, and decrease the second weight value corresponding to the iris feature and the third weight value corresponding to the fingerprint feature;

[0023] When it is determined that the facial posture is facing the camera and the environment type is a low-light environment, reducing the first weight value corresponding to the facial feature, and increasing the second weight value corresponding to the iris feature and the third weight value corresponding to the fingerprint feature;

[0024] When it is determined that the facial posture is not facing the camera and the environment type is a high-light environment, reducing the first weight value corresponding to the facial feature and the second weight value corresponding to the iris feature, and increasing the third weight value corresponding to the fingerprint feature;

[0025] When it is determined that the facial posture is not facing the camera and the environment type is a low-light environment, the first weight value corresponding to the facial feature and the second weight value corresponding to the iris feature are reduced, and the third weight value corresponding to the fingerprint feature is increased.

[0026] In some embodiments, the training method of the decision fusion layer includes:

[0027] Acquiring the facial features, the iris features and the fingerprint features in different scene modes;

[0028] Standardizing the facial features, the iris features, and the fingerprint features to obtain standardized features;

[0029] The standardized features are input into the decision fusion layer for training. During the training process, the weights corresponding to the facial features, the iris features and the fingerprint features are tuned by a back propagation algorithm until a preset optimization target is reached, thereby obtaining a weight value adjustment strategy corresponding to different scene modes, wherein the decision fusion layer adopts a multi-layer perceptron model.

[0030] In some embodiments, the method further comprises:

[0031] In the case where the identity authentication result fails to be generated or the confidence level of the identity authentication result is lower than a preset confidence level, recording feedback data corresponding to the user, the feedback data including the multimodal biometric data of the user, the facial posture of the user in the current environment, and environmental condition parameters of the current environment;

[0032] The weight value adjustment strategy of the decision fusion layer is adjusted according to the feedback data.

[0033] In some embodiments, the method further comprises:

[0034] In the case where the facial gesture is blocked, optimizing the parameters of the facial gesture recognition module by using an occluded face recognition algorithm so that the facial gesture recognition module can recognize the facial gesture of the user;

[0035] When the illumination parameter in the environmental condition parameter exceeds the preset illumination threshold range, the environment discrimination module is optimized through the illumination compensation mechanism of the face recognition algorithm so that the environment discrimination module can recognize the current environment of the user.

[0036] To achieve the above-mentioned purpose, a second aspect of an embodiment of the present application provides an identity authentication device, the device comprising:

[0037] An acquisition module, used to acquire multimodal biometric data of the user in the current environment, wherein the multimodal biometric data includes a face image, an iris image, and fingerprint information;

[0038] A first extraction module, used for extracting features from the face image through a convolutional neural network to obtain face features;

[0039] A second extraction module, used for extracting features from the iris image and the fingerprint information by using a support vector machine to generate iris features and fingerprint features;

[0040] A recognition module, used to recognize the facial posture of the user in the current environment and the environment type of the current environment;

[0041] An adjustment module, used for adjusting the weight values ​​corresponding to the facial features, the iris features and the fingerprint features respectively according to the facial posture and the environment type;

[0042] A generation module is used to perform weighted fusion on the facial features, the iris features and the fingerprint features according to the weight values ​​to generate an identity authentication result corresponding to the user in the current environment.

[0043] To achieve the above objectives, a third aspect of an embodiment of the present application proposes an electronic device, which includes a memory and a processor, wherein the memory stores a computer program, and when the processor executes the computer program, the method described in the first aspect is implemented.

[0044] To achieve the above-mentioned purpose, the fourth aspect of an embodiment of the present application proposes a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the method described in the first aspect is implemented.

[0045] The identity authentication method and device, electronic device and storage medium proposed in the present application obtain multimodal biometric data of the user in the current environment, the multimodal biometric data includes face image, iris image and fingerprint information; extract features of face image through convolutional neural network to obtain face features; extract features of iris image and fingerprint information through support vector machine to generate iris features and fingerprint features; identify the face posture of the user in the current environment and the environment type of the current environment; adjust the weight values ​​corresponding to the face features, iris features and fingerprint features according to the face posture and environment type; perform weighted fusion of face features, iris features and fingerprint features according to the weight values ​​to generate the identity authentication result of the corresponding user in the current environment. By obtaining multimodal biometric data of the user in the current environment, and extracting features of face image, iris image and fingerprint information, face features, iris features and fingerprint features are obtained. And according to the facial posture of the user in the current environment and the environment type of the current environment, the weight values ​​corresponding to the facial features, iris features and fingerprint features are adjusted, and then the facial features, iris features and fingerprint features are weighted and fused according to the weight values ​​to generate the identity authentication result of the corresponding user in the current environment. The embodiment of the present application not only combines multimodal features to authenticate the user, which is highly secure, but also can dynamically and adaptively adjust the weight values ​​of different modes according to the facial posture and environment type of the user's environment to adapt to different user postures and environmental conditions. Since the weights of each mode are adjusted in real time according to different usage scenarios, the weight allocation is more intelligent and adaptable, thereby improving the accuracy and robustness of recognition in complex environments. Based on this, the embodiment of the present application can improve the security and accuracy of user identity authentication, especially in financial scenarios, it can intelligently and accurately authenticate users in different complex environments, help reduce the risk of identity theft and fraud, improve the security and accuracy of financial services, and improve customer satisfaction, thereby enhancing the competitiveness of financial enterprises. BRIEF DESCRIPTION OF THE DRAWINGS

[0046] Figure 1 is a flow chart of the identity authentication method provided in an embodiment of the present application;

[0047] Figure 2 yes Figure 1 Flowchart of step S104 in FIG.

[0048] Figure 3 yes Figure 1 Flow chart of step S105 in FIG.

[0049] Figure 4 is a flowchart of a training method for a decision fusion layer provided in an embodiment of the present application;

[0050] Figure 5 is a flowchart from step S501 to step S502;

[0051] Figure 6 It is a schematic diagram of the structure of the identity authentication device provided in the embodiment of the present application;

[0052] Figure 7 It is a schematic diagram of the hardware structure of the electronic device provided in the embodiment of the present application. DETAILED DESCRIPTION

[0053] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0054] It should be noted that, although the functional modules are divided in the device schematic diagram and the logical order is shown in the flowchart, in some cases, the steps shown or described may be performed in a different order than the module division in the device or the order in the flowchart. The terms "first", "second", etc. in the specification, claims and the above drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence.

[0055] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as those commonly understood by those skilled in the art to which this application belongs. The terms used herein are only for the purpose of describing the embodiments of this application and are not intended to limit this application.

[0056] First, some nouns involved in this application are analyzed:

[0057] Artificial intelligence (AI) is a new technical science that studies and develops theories, methods, technologies and application systems for simulating, extending and expanding human intelligence. AI is a branch of computer science that attempts to understand the essence of intelligence and produce a new type of intelligent machine that can respond in a similar way to human intelligence. Research in this field includes robots, language recognition, image recognition, natural language processing and expert systems. AI can simulate the information process of human consciousness and thinking. AI is also a theory, method, technology and application system that uses digital computers or machines controlled by digital computers to simulate, extend and expand human intelligence, perceive the environment, acquire knowledge and use knowledge to obtain the best results.

[0058] Biometric recognition technology: refers to the technology that uses the human body's inherent physiological characteristics (fingerprints, irises, facial features, DNA, etc.) or behavioral characteristics (gait, typing habits, etc.) to identify individuals through computers.

[0059] Multimodal biometrics: refers to the integration or fusion of two or more biometric technologies, using the unique advantages of multiple biometric technologies and combining data fusion technology to make the authentication and identification process more accurate and secure. The main difference from the traditional single biometric method is that multimodal biometrics can collect different biometric features (such as fingerprints, finger veins, faces, iris images, etc.) through independent or multiple collection methods, and identify and authenticate by analyzing and judging the feature values ​​of multiple biometric methods.

[0060] Convolutional Neural Network (CNN): is a deep learning algorithm, and its name comes from the convolutional layer in its structure. CNN is usually used for image recognition and processing, but can also be applied to other types of data, such as text and sound. The main features of CNN are local perception, weight sharing, and downsampling (pooling). Local perception means that neurons are only connected to a small part of the input data, weight sharing reduces the number of parameters that need to be learned, and downsampling helps reduce the complexity of the data and reduce overfitting.

[0061] Support Vector Machine (SVM): is a supervised learning model mainly used for classification and regression analysis. It is based on statistical learning theory and aims to find an optimal hyperplane to separate different categories of data in a way that maximizes the classification interval. The core idea of ​​SVM is to deal with nonlinear problems through the kernel function method and find an optimal hyperplane in high-dimensional space to maximize the interval between the hyperplane and the nearest data point. In this way, support vector machines achieve data classification and perform well in dealing with high-dimensional data and nonlinear problems.

[0062] Deep learning (DL) is a new research direction in the field of machine learning (ML). It is introduced into machine learning to make it closer to its original goal - artificial intelligence (AI). Deep learning is to learn the inherent laws and representation levels of sample data. The information obtained in the learning process is of great help in the interpretation of data such as text, images and sounds. Its ultimate goal is to enable machines to have analytical learning capabilities like humans and to recognize data such as text, images and sounds. Deep learning is a complex machine learning algorithm, and its results in speech and image recognition far exceed those of previous related technologies. Deep learning has achieved many results in search technology, data mining, machine learning, machine translation, natural language processing, multimedia learning, speech, recommendation and personalization technology, and other related fields. Deep learning enables machines to imitate human activities such as seeing, hearing and thinking, solves many complex pattern recognition problems, and makes great progress in artificial intelligence related technologies.

[0063] Currently, when users authenticate their identity, it is done by identifying their biometrics. Biometric-based authentication refers to authentication based on unique features of each person's body, such as face, fingerprint, iris, etc. Existing authentication methods use single-modal biometrics or simply superimposed multi-modal biometrics technology to achieve authentication. However, single-modal biometrics technology is not secure enough, and simply superimposed multi-modal biometrics technology cannot intelligently adapt to the different complex environments in which users authenticate their identity, resulting in poor recognition accuracy and robustness.

[0064] In financial scenarios, such as bank account opening, loan approval, and large-value transfers, where high security is required, existing identity authentication methods have the above-mentioned technical defects, which poses the risk of identity theft and fraud. Based on this, in financial scenarios, how to improve the security and accuracy of user identity authentication has become a technical problem that needs to be solved urgently.

[0065] Based on this, the embodiment of the present application provides an identity authentication method and device, an electronic device and a storage medium, which obtains multimodal biometric data of a user in a current environment, and the multimodal biometric data includes a face image, an iris image and fingerprint information; extracts features of the face image through a convolutional neural network to obtain face features; extracts features of the iris image and fingerprint information through a support vector machine to generate iris features and fingerprint features; identifies the face posture of the user in the current environment and the environment type of the current environment; adjusts the weight values ​​corresponding to the face features, iris features and fingerprint features according to the face posture and environment type; performs weighted fusion of the face features, iris features and fingerprint features according to the weight values ​​to generate the identity authentication result of the corresponding user in the current environment. By obtaining the multimodal biometric data of the user in the current environment, and extracting features of the face image, iris image and fingerprint information, the face features, iris features and fingerprint features are obtained. And according to the facial posture of the user in the current environment and the environment type of the current environment, the weight values ​​corresponding to the facial features, iris features and fingerprint features are adjusted, and then the facial features, iris features and fingerprint features are weighted and fused according to the weight values ​​to generate the identity authentication result of the corresponding user in the current environment. The embodiment of the present application not only combines multimodal features to authenticate the user, which is highly secure, but also can dynamically and adaptively adjust the weight values ​​of different modes according to the facial posture and environment type of the user's environment to adapt to different user postures and environmental conditions. Since the weights of each mode are adjusted in real time according to different usage scenarios, the weight allocation is more intelligent and adaptable, thereby improving the accuracy and robustness of recognition in complex environments. Based on this, the embodiment of the present application can improve the security and accuracy of user identity authentication, especially in financial scenarios, it can intelligently and accurately authenticate users in different complex environments, help reduce the risk of identity theft and fraud, improve the security and accuracy of financial services, and improve customer satisfaction, thereby enhancing the competitiveness of financial enterprises.

[0066] The identity authentication method and device, electronic device and storage medium provided in the embodiments of the present application are specifically illustrated through the following embodiments. First, the identity authentication method in the embodiments of the present application is described.

[0067] The embodiments of the present application can acquire and process relevant data based on artificial intelligence technology. Artificial Intelligence (AI) is the theory, method, technology and application system that uses digital computers or machines controlled by digital computers to simulate, extend and expand human intelligence, perceive the environment, acquire knowledge and use knowledge to obtain the best results.

[0068] AI basic technologies generally include sensors, dedicated AI chips, cloud computing, distributed storage, big data processing technology, operation / interaction systems, mechatronics, etc. AI software technologies mainly include computer vision technology, robotics technology, biometrics technology, speech processing technology, natural language processing technology, and machine learning / deep learning.

[0069] The identity authentication method provided in the embodiment of the present application relates to the field of artificial intelligence technology. The identity authentication method provided in the embodiment of the present application can be applied to a terminal, can be applied to a server side, or can be software running in a terminal or a server side. In some embodiments, the terminal can be a smart phone, a tablet computer, a laptop computer, a desktop computer, etc.; the server side can be configured as an independent physical server, or a server cluster or distributed system composed of multiple physical servers, or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, CDN, and big data and artificial intelligence platforms; the software can be an application that implements the identity authentication method, etc., but is not limited to the above forms.

[0070] The present application can be used in many general or special computer system environments or configurations. For example: personal computers, server computers, handheld or portable devices, tablet devices, multiprocessor systems, microprocessor-based systems, set-top boxes, programmable consumer electronics, network PCs, minicomputers, mainframe computers, distributed computing environments including any of the above systems or devices, etc. The present application can be described in the general context of computer-executable instructions executed by a computer, such as program modules. Generally, program modules include routines, programs, objects, components, data structures, etc. that perform specific tasks or implement specific abstract data types. The present application can also be practiced in distributed computing environments, in which tasks are performed by remote processing devices connected through a communication network. In a distributed computing environment, program modules can be located in local and remote computer storage media including storage devices.

[0071] It should be noted that in each specific implementation of the present application, when it comes to the need to perform relevant processing based on data related to user identity or characteristics such as user information, user behavior data, user historical data, and user location information, the user's permission or consent will be obtained first, and the collection, use, and processing of these data will comply with relevant laws, regulations, and standards. In addition, when the embodiment of the present application needs to obtain the user's sensitive personal information, the user's separate permission or consent will be obtained through a pop-up window or by jumping to a confirmation page. After clearly obtaining the user's separate permission or consent, the necessary user-related data for the normal operation of the embodiment of the present application will be obtained.

[0072] Figure 1 is an optional flow chart of the identity authentication method provided in the embodiment of the present application. Figure 1 The method may include but is not limited to steps S101 to S106.

[0073] Step S101, obtaining multimodal biometric data of the user in the current environment, the multimodal biometric data including face image, iris image and fingerprint information;

[0074] Step S102, extracting features from the face image using a convolutional neural network to obtain face features;

[0075] Step S103, extracting features from the iris image and fingerprint information using a support vector machine to generate iris features and fingerprint features;

[0076] Step S104, identifying the facial posture of the user in the current environment and the environment type of the current environment;

[0077] Step S105, adjusting the weight values ​​corresponding to the facial features, iris features and fingerprint features according to the facial posture and environment type;

[0078] Step S106, weighted fusion of facial features, iris features and fingerprint features is performed according to the weight values ​​to generate an identity authentication result of the corresponding user in the current environment.

[0079] In step S101 of some embodiments, multimodal biometric data of the user in the current environment is obtained, and the multimodal biometric data includes facial images, iris images and fingerprint information. Specifically, multimodal biometric data including face, fingerprint and iris are collected, and the multimodal biometric data covers the user's facial posture and environmental conditions, wherein the facial posture includes but is not limited to looking straight ahead, looking sideways, looking up and looking down, and the environmental conditions include but are not limited to light, temperature and humidity. These multimodal biometric data can be synchronously acquired using a multimodal sensor, and these multimodal biometric data can also be preprocessed, and the preprocessing can include normalization, denoising, etc. In a financial scenario, the customer can be photographed in real time through the multimodal biometric system of a financial institution to obtain the customer's multimodal biometric data.

[0080] In step S102 of some embodiments, feature extraction is performed on the face image through a convolutional neural network to obtain facial features. Convolutional neural networks (CNNs) have shown extremely high performance in image processing, and are particularly good at extracting complex, hierarchical features from two-dimensional images. Compared with traditional manual feature extraction methods, CNN can automatically learn the spatial relationship of facial features, such as edges, corners, textures, etc., through multiple convolutional layers, thereby extracting high-dimensional features that are more robust and abstract than traditional methods. When processing unstructured image data (such as faces), CNN has a highly automated learning ability and can adapt to a variety of complex scene changes (such as different lighting, expressions, occlusions, etc.), which can significantly improve the accuracy and reliability of face recognition in actual use.

[0081] In step S103 of some embodiments, feature extraction is performed on the iris image and fingerprint information by a support vector machine to generate iris features and fingerprint features. Support vector machine (SVM) is a powerful algorithm for classification and regression, and is particularly suitable for data classification in high-dimensional feature space. The features of fingerprint information and iris images are usually high-dimensional texture and structural features, and SVM can effectively distinguish the fingerprint information or iris features of different individuals by finding the optimal classification boundary in high-dimensional space. SVM performs well in processing small samples of high-dimensional data, especially for data types such as fingerprints and irises, which have relatively stable and unique features. SVM can provide fast and accurate classification results. Compared with other algorithms, SVM has better generalization ability and is particularly suitable for accurate classification in the case of small amounts of data, so that in actual use, the accuracy and reliability of iris recognition and fingerprint recognition can be significantly improved.

[0082] In step S104 of some embodiments, the facial posture of the user in the current environment and the environment type of the current environment are identified. Specifically, the facial posture of the user can be determined by obtaining a facial image. The environmental condition parameters of the current environment can be identified by an environmental sensor, and the type of environment in which the user is located can be determined based on the environmental condition parameters. Among them, the facial posture includes but is not limited to looking straight ahead, looking sideways, looking up and looking down, and the environment type includes but is not limited to a high-light environment and a low-light environment.

[0083] In step S105 of some embodiments, the weight values ​​corresponding to the facial features, iris features, and fingerprint features are adjusted according to the facial posture and the environment type. For example, when the user's facial posture is facing the camera and the environment is a high-light environment, the first weight value corresponding to the facial features can be increased, and the second weight value corresponding to the iris features and the third weight value corresponding to the fingerprint features can be reduced; when the user's facial posture is facing the camera and the environment is a low-light environment, the first weight value corresponding to the facial features can be reduced, and the second weight value corresponding to the iris features and the third weight value corresponding to the fingerprint features can be increased; when the user's facial posture is not facing the camera, the first weight value corresponding to the facial features and the second weight value corresponding to the iris features can be reduced, and the third weight value corresponding to the fingerprint features can be increased. Based on this, the embodiment of the present application can dynamically and adaptively adjust the weight values ​​of different modalities according to the facial posture and environment type of the user's environment to adapt to different user postures and environmental conditions.

[0084] In step S106 of some embodiments, the face features, iris features and fingerprint features are weighted and fused according to the weight values ​​to generate the identity authentication result of the corresponding user in the current environment. Since the weight values ​​of each modality are adjusted in real time according to different usage scenarios, the weight value allocation is more intelligent and adaptable. Therefore, the face features, iris features and fingerprint features are weighted and fused according to the weight value allocation, and the identity authentication result generated is more suitable for the complex environment in which the user is located, thereby improving the accuracy and robustness of user identity recognition in complex environments.

[0085] In financial scenarios, for example, in scenarios that require high security such as bank account opening, loan approval, and large-amount transfers, the identity authentication method of the embodiment of the present application can significantly reduce the risk of identity theft and fraud, and improve the security and accuracy of financial services. For another example, in a banking system, when a user authenticates their identity, by simultaneously collecting multimodal data including the user's face, fingerprint, and iris, and comprehensively verifying their identity, the security and accuracy of banking system services can be improved. For another example, in an ATM self-service scenario, even at night or in poor light, the identity authentication method of the embodiment of the present application can still accurately authenticate the user, thereby greatly improving the security and accuracy of ATM self-service.

[0086] Steps S101 to S106 shown in the embodiment of the present application are to obtain multimodal biometric data of the user in the current environment, the multimodal biometric data including face image, iris image and fingerprint information; extract features of the face image through a convolutional neural network to obtain face features; extract features of the iris image and fingerprint information through a support vector machine to generate iris features and fingerprint features; identify the face posture of the user in the current environment and the environment type of the current environment; adjust the weight values ​​corresponding to the face features, iris features and fingerprint features according to the face posture and environment type; perform weighted fusion of the face features, iris features and fingerprint features according to the weight values ​​to generate the identity authentication result of the corresponding user in the current environment. By obtaining multimodal biometric data of the user in the current environment and extracting features of the face image, iris image and fingerprint information, face features, iris features and fingerprint features are obtained. And according to the facial posture of the user in the current environment and the environment type of the current environment, the weight values ​​corresponding to the facial features, iris features and fingerprint features are adjusted, and then the facial features, iris features and fingerprint features are weighted and fused according to the weight values ​​to generate the identity authentication result of the corresponding user in the current environment. The embodiment of the present application not only combines multimodal features to authenticate the user, which is highly secure, but also can dynamically and adaptively adjust the weight values ​​of different modes according to the facial posture and environment type of the user's environment to adapt to different user postures and environmental conditions. Since the weights of each mode are adjusted in real time according to different usage scenarios, the weight allocation is more intelligent and adaptable, thereby improving the accuracy and robustness of recognition in complex environments. Based on this, the embodiment of the present application can improve the security and accuracy of user identity authentication, especially in financial scenarios, it can intelligently and accurately authenticate users in different complex environments, help reduce the risk of identity theft and fraud, improve the security and accuracy of financial services, and improve customer satisfaction, thereby enhancing the competitiveness of financial enterprises.

[0087] See also Figure 2 In some embodiments, step S104 may include but is not limited to steps S201 to S204:

[0088] Step S201, determining facial posture parameters of a user in a current environment according to a facial image, where the facial image is obtained by photographing the user with a camera;

[0089] Step S202, inputting the facial posture parameters into a preset facial posture identification module to identify the facial posture of the user in the current environment;

[0090] Step S203, obtaining environmental condition parameters of the current environment through an environmental sensor;

[0091] Step S204: input the environmental condition parameters into a preset environment identification module to identify the environment type of the user's current environment.

[0092] The user's face image can be obtained by photographing the user through a camera, and the facial posture parameters of the user in the current environment can be obtained based on the facial image, and then the facial posture parameters are input into a preset facial posture recognition module to identify the facial posture of the user in the current environment. Facial postures include but are not limited to looking straight at the camera and not looking straight at the camera, wherein not looking straight at the camera includes postures such as sideways looking, looking up, and looking down.

[0093] The environmental condition parameters of the current environment can be obtained through environmental sensors, wherein the environmental sensors include but are not limited to light sensors, temperature sensors, and humidity sensors. The light sensor is used to detect the light intensity parameters of the current environment, the temperature sensor is used to detect the temperature parameters of the current environment, and the humidity sensor is used to detect the humidity parameters of the current environment. The environmental condition parameters include light intensity parameters, temperature parameters, and humidity parameters. The environmental condition parameters are input into a preset environmental discrimination module, and the environmental discrimination module determines the environmental type of the current environment through a machine learning algorithm (such as a decision tree or a support vector machine), thereby identifying the environmental type of the user in the current environment. Taking the lighting environment condition as an example, the environmental type can include a high-light environment and a low-light environment.

[0094] See also Figure 3 In some embodiments, step S105 may include but is not limited to steps S301 to S303:

[0095] Step S301, determining a scene mode according to facial posture and environment type;

[0096] Step S302, determining a weight value adjustment strategy output by a preset decision fusion layer according to the scene mode;

[0097] Step S303: adjusting the first weight value corresponding to the facial feature, the second weight value corresponding to the iris feature, and the third weight value corresponding to the fingerprint feature based on the weight value adjustment strategy.

[0098] In some embodiments, the embodiments of the present application introduce a decision fusion layer based on deep learning. When the facial posture of the user in the current environment and the environment type of the user in the current environment are identified, the scene mode that the decision fusion layer can recognize is determined based on the facial posture and the environment type, so that the decision fusion layer outputs the corresponding weight value adjustment strategy according to the recognized scene mode. Then, based on the weight value adjustment strategy, the first weight value corresponding to the facial feature, the second weight value corresponding to the iris feature, and the third weight value corresponding to the fingerprint feature are adjusted.

[0099] In some embodiments, when it is determined that the user's facial posture is facing the camera and the environment type is a high-light environment, the first weight value corresponding to the facial feature is increased, and the second weight value corresponding to the iris feature and the third weight value corresponding to the fingerprint feature are decreased;

[0100] In some embodiments, when it is determined that the user's facial posture is facing the camera and the environment type is a low-light environment, the first weight value corresponding to the facial feature is reduced, and the second weight value corresponding to the iris feature and the third weight value corresponding to the fingerprint feature are increased;

[0101] In some embodiments, when it is determined that the user's facial posture is not facing the camera and the environment type is a high-light environment, the first weight value corresponding to the facial feature and the second weight value corresponding to the iris feature are reduced, and the third weight value corresponding to the fingerprint feature is increased;

[0102] In some embodiments, when it is determined that the user's facial posture is not facing the camera and the environment type is a low-light environment, the first weight value corresponding to the facial feature and the second weight value corresponding to the iris feature are reduced, and the third weight value corresponding to the fingerprint feature is increased.

[0103] It should be noted that the recognition of facial images needs to be carried out under high-light conditions; the recognition of iris images relies on near-infrared light and has no special requirements for external lighting conditions.

[0104] See also Figure 4 In some embodiments, the training method of the decision fusion layer may include but is not limited to steps S401 to S403:

[0105] Step S401, acquiring facial features, iris features and fingerprint features in different scene modes;

[0106] Step S402, standardizing the facial features, iris features and fingerprint features to obtain standardized features;

[0107] Step S403, input the standardized features into the decision fusion layer for training. During the training process, the weights corresponding to the facial features, iris features and fingerprint features are tuned by the back propagation algorithm until the preset optimization target is reached, and the weight value adjustment strategy corresponding to different scene modes is obtained, wherein the decision fusion layer adopts a multi-layer perceptron model.

[0108] In some embodiments, the decision fusion layer is a multi-layer perceptron MLP model based on deep learning. The decision fusion layer receives the high-dimensional feature vectors output by each modal feature extractor as input, fuses them through multiple fully connected layers, and outputs the final recognition result. During the training process, the decision fusion layer will first standardize the features extracted by each modality, and then input them into the MLP model. The weights corresponding to each modal feature are tuned through the back propagation algorithm until the preset optimization goal is reached, and the weight value adjustment strategy corresponding to different scene modes is obtained to achieve adaptive adjustment to different environments. Based on this, the decision fusion layer learns how to best combine multimodal features including facial features, iris features, and fingerprint features in different scenarios during training to ensure that the weight value adjustment strategy of the comprehensive output can accurately respond to different scene modes.

[0109] See also Figure 5 In some embodiments, the identity authentication method of the present application may also include but is not limited to steps S501 to S502:

[0110] Step S501, when the generation of the identity authentication result fails or the confidence level of the identity authentication result is lower than a preset confidence level, recording the corresponding user's feedback data, the feedback data including the user's multimodal biometric data, the user's facial posture in the current environment, and the environmental condition parameters of the current environment;

[0111] Step S502: adjusting the weight value adjustment strategy of the decision fusion layer according to the feedback data.

[0112] The identity authentication method of the embodiment of the present application also has an abnormal situation feedback and optimization mechanism. During the recognition process, special situations encountered (such as extreme lighting, occlusion, etc.) will be recorded in real time, and the relevant feedback data will be fed back to the decision fusion layer for model optimization to continuously improve the recognition performance. Among them, the feedback data includes the user's multimodal biometric data, the user's facial posture in the current environment, and the environmental condition parameters of the current environment.

[0113] In some embodiments, each time a recognition fails or a low confidence result occurs, the feedback data at that time is recorded. Subsequently, the feedback data is fed back to the decision fusion layer for further training to further optimize the model parameters. The decision fusion layer readjusts the weight value adjustment strategy based on the feedback data, for example, adjusting the weight calculation rules and weight allocation rules of the decision fusion layer.

[0114] In some embodiments, when the facial gesture is blocked, the facial gesture recognition module is optimized by using the blocked face recognition algorithm so that the facial gesture recognition module can recognize the user's facial gesture;

[0115] In some embodiments, when the illumination parameter in the environmental condition parameter exceeds a preset illumination threshold range, the environment discrimination module is optimized through the illumination compensation mechanism of the face recognition algorithm so that the environment discrimination module can recognize the current environment of the user.

[0116] Based on this, the identity authentication method of the embodiment of the present application enables the model to continuously learn and improve through real-time feedback and self-optimization mechanisms, ensuring that the recognition performance will not decline during long-term use. Instead, the recognition accuracy of the model will gradually improve as more feedback data is accumulated.

[0117] Based on this, the identity authentication method of the embodiment of the present application combines environmental perception with deep learning, and the decision fusion layer can adjust the weight value adjustment strategy in real time to ensure high-accuracy recognition even in complex environments.

[0118] In financial scenarios, for example, in scenarios that require high security such as bank account opening, loan approval, and large-amount transfers, the identity authentication method of the embodiment of the present application can significantly reduce the risk of identity theft and fraud, and improve the security and accuracy of financial services. For another example, in a banking system, when a user authenticates their identity, by simultaneously collecting multimodal data including the user's face, fingerprint, and iris, and comprehensively verifying their identity, the security and accuracy of banking system services can be improved. For another example, in an ATM self-service scenario, even at night or in poor light, the identity authentication method of the embodiment of the present application can still accurately authenticate the user, thereby greatly improving the security and accuracy of ATM self-service.

[0119] See also Figure 6 The embodiment of the present application also provides an identity authentication device, which can implement the above identity authentication method, and the device includes:

[0120] An acquisition module 610 is used to acquire multimodal biometric data of the user in the current environment, where the multimodal biometric data includes a face image, an iris image, and fingerprint information;

[0121] A first extraction module 620, configured to extract features from a face image using a convolutional neural network to obtain face features;

[0122] The second extraction module 630 is used to extract features from the iris image and fingerprint information by using a support vector machine to generate iris features and fingerprint features;

[0123] Recognition module 640, used to recognize the facial posture of the user in the current environment and the environment type of the current environment;

[0124] An adjustment module 650, used to adjust the weight values ​​corresponding to the facial features, iris features and fingerprint features according to the facial posture and environment type;

[0125] The generation module 660 is used to perform weighted fusion of facial features, iris features and fingerprint features according to weight values ​​to generate an identity verification result of the corresponding user in the current environment.

[0126] In some embodiments of the present application, the acquisition module 610 acquires multimodal biometric data of the user in the current environment, and the multimodal biometric data includes facial images, iris images and fingerprint information; the first extraction module 620 extracts features of the facial image through a convolutional neural network to obtain facial features; the second extraction module 630 extracts features of the iris image and fingerprint information through a support vector machine to generate iris features and fingerprint features; the recognition module 640 recognizes the facial posture of the user in the current environment and the environment type of the current environment; the adjustment module 650 adjusts the weight values ​​corresponding to the facial features, iris features and fingerprint features according to the facial posture and the environment type; the generation module 660 performs weighted fusion of the facial features, iris features and fingerprint features according to the weight values ​​to generate the identity authentication result of the corresponding user in the current environment.

[0127] In some embodiments of the present application, multimodal biometric data of the user in the current environment is obtained, and the multimodal biometric data includes facial images, iris images, and fingerprint information. Specifically, multimodal biometric data including face, fingerprint, and iris are collected, and the multimodal biometric data covers the user's facial posture and environmental conditions, wherein the facial posture includes but is not limited to looking straight ahead, looking sideways, looking up, and looking down, and the environmental conditions include but are not limited to light, temperature, and humidity. These multimodal biometric data can be synchronously acquired using a multimodal sensor, and these multimodal biometric data can also be preprocessed, and the preprocessing may include normalization, denoising, etc. In a financial scenario, the customer can be photographed in real time through the multimodal biometric system of a financial institution to obtain the customer's multimodal biometric data.

[0128] In some embodiments of the present application, a convolutional neural network is used to extract features from a facial image to obtain facial features. Convolutional neural networks (CNNs) have shown extremely high performance in image processing, and are particularly good at extracting complex, hierarchical features from two-dimensional images. Compared with traditional manual feature extraction methods, CNNs can automatically learn the spatial relationships of facial features, such as edges, corners, textures, etc., through multiple convolutional layers, thereby extracting high-dimensional features that are more robust and abstract than traditional methods. When processing unstructured image data (such as faces), CNNs have highly automated learning capabilities and can adapt to a variety of complex scene changes (such as different lighting, expressions, occlusions, etc.), which can significantly improve the accuracy and reliability of face recognition in actual use.

[0129] In some embodiments of the present application, feature extraction of iris images and fingerprint information is performed by support vector machines to generate iris features and fingerprint features. Support vector machine (SVM) is a powerful algorithm for classification and regression, and is particularly suitable for data classification in high-dimensional feature space. The features of fingerprint information and iris images are usually high-dimensional texture and structural features, and SVM can effectively distinguish the fingerprint information or iris features of different individuals by finding the optimal classification boundary in high-dimensional space. SVM performs well in processing high-dimensional data with small samples, especially for data types such as fingerprints and irises with relatively stable and unique features, SVM can provide fast and accurate classification results. Compared with other algorithms, SVM has better generalization ability and is particularly suitable for accurate classification in the case of small amounts of data, so that in actual use, the accuracy and reliability of iris recognition and fingerprint recognition can be significantly improved.

[0130] In some embodiments of the present application, the facial posture of the user in the current environment and the environment type of the current environment are identified. Specifically, the facial posture of the user can be determined by obtaining a facial image. The environmental condition parameters of the current environment can be identified by an environmental sensor, and the type of environment the user is in can be determined based on the environmental condition parameters. Among them, the facial posture includes but is not limited to looking straight ahead, looking sideways, looking up and looking down, and the environment type includes but is not limited to a high-light environment and a low-light environment.

[0131] In some embodiments of the present application, the weight values ​​corresponding to the facial features, iris features, and fingerprint features are adjusted according to the facial posture and the environment type. For example, when the user's facial posture is facing the camera and the environment is a high-light environment, the first weight value corresponding to the facial features can be increased, and the second weight value corresponding to the iris features and the third weight value corresponding to the fingerprint features can be reduced; when the user's facial posture is facing the camera and the environment is a low-light environment, the first weight value corresponding to the facial features can be reduced, and the second weight value corresponding to the iris features and the third weight value corresponding to the fingerprint features can be increased; when the user's facial posture is not facing the camera, the first weight value corresponding to the facial features and the second weight value corresponding to the iris features can be reduced, and the third weight value corresponding to the fingerprint features can be increased. Based on this, the embodiments of the present application can dynamically and adaptively adjust the weight values ​​of different modes according to the facial posture and environment type of the user's environment to adapt to different user postures and environmental conditions.

[0132] In some embodiments of the present application, face features, iris features, and fingerprint features are weighted and fused according to weight values ​​to generate identity authentication results for the corresponding user in the current environment. Since the weight values ​​of each modality are adjusted in real time according to different usage scenarios, the weight value allocation is more intelligent and adaptable. Therefore, face features, iris features, and fingerprint features are weighted and fused according to the weight value allocation, and the generated identity authentication results are more compatible with the complex environment in which the user is located, thereby improving the accuracy and robustness of user identity recognition in complex environments.

[0133] Based on this, in the identity authentication device of the embodiment of the present application, the acquisition module 610 acquires the multimodal biometric data of the user in the current environment, and the multimodal biometric data includes facial images, iris images and fingerprint information; the first extraction module 620 extracts features of the facial image through a convolutional neural network to obtain facial features; the second extraction module 630 extracts features of the iris image and fingerprint information through a support vector machine to generate iris features and fingerprint features; the recognition module 640 recognizes the facial posture of the user in the current environment and the environment type of the current environment; the adjustment module 650 adjusts the weight values ​​corresponding to the facial features, iris features and fingerprint features according to the facial posture and the environment type; the generation module 660 performs weighted fusion of the facial features, iris features and fingerprint features according to the weight values ​​to generate the identity authentication result of the corresponding user in the current environment. The present application obtains multimodal biometric data of the user in the current environment, and the multimodal biometric data includes face image, iris image and fingerprint information; extracts features of the face image through a convolutional neural network to obtain face features; extracts features of the iris image and fingerprint information through a support vector machine to generate iris features and fingerprint features; identifies the face posture of the user in the current environment and the environment type of the current environment; adjusts the weight values ​​corresponding to the face features, iris features and fingerprint features according to the face posture and environment type; performs weighted fusion of the face features, iris features and fingerprint features according to the weight values ​​to generate the identity authentication result of the corresponding user in the current environment. By obtaining multimodal biometric data of the user in the current environment, and extracting features of the face image, iris image and fingerprint information, the face features, iris features and fingerprint features are obtained. And according to the face posture of the user in the current environment and the environment type of the current environment, the weight values ​​corresponding to the face features, iris features and fingerprint features are adjusted, and then the face features, iris features and fingerprint features are weighted fused according to the weight values ​​to generate the identity authentication result of the corresponding user in the current environment. The embodiment of the present application not only combines multimodal features to authenticate users, which is highly secure, but also can dynamically and adaptively adjust the weight values ​​of different modalities according to the facial posture and environment type of the user's environment to adapt to different user postures and environmental conditions. Since the weights of each modality are adjusted in real time according to different usage scenarios, the weight allocation is more intelligent and adaptive, thereby improving the accuracy and robustness of recognition in complex environments. Based on this, the embodiment of the present application can improve the security and accuracy of user identity authentication, especially in financial scenarios, and can intelligently and accurately authenticate users in different complex environments, which helps to reduce the risk of identity theft and fraud, improve the security and accuracy of financial services, and improve customer satisfaction, thereby enhancing the competitiveness of financial enterprises.

[0134] The specific implementation of the identity authentication device is basically the same as the specific implementation of the above-mentioned identity authentication method, and will not be repeated here.

[0135] The embodiment of the present application also provides an electronic device, the electronic device includes a memory and a processor, the memory stores a computer program, and the processor implements the above-mentioned identity authentication method when executing the computer program. The electronic device can be any intelligent terminal including a tablet computer, a car computer, etc.

[0136] See also Figure 7 , Figure 7 The hardware structure of an electronic device of another embodiment is illustrated, and the electronic device includes:

[0137] The processor 701 can be implemented by a general-purpose CPU (Central Processing Unit), a microprocessor, an application-specific integrated circuit (ASIC), or one or more integrated circuits, and is used to execute relevant programs to implement the technical solutions provided in the embodiments of the present application.

[0138] The memory 702 can be implemented in the form of a read-only memory (ROM), a static storage device, a dynamic storage device or a random access memory (RAM). The memory 702 can store an operating system and other applications. When the technical solution provided in the embodiment of this specification is implemented by software or firmware, the relevant program code is stored in the memory 702, and the processor 701 is called to execute the identity authentication method of the embodiment of this application, that is, by obtaining multimodal biometric data of the user in the current environment, the multimodal biometric data includes a face image, an iris image and fingerprint information; extracting features of the face image through a convolutional neural network to obtain face features; extracting features of the iris image and fingerprint information through a support vector machine to generate iris features and fingerprint features; identifying the face posture of the user in the current environment and the environment type of the current environment; adjusting the weight values ​​corresponding to the face features, iris features and fingerprint features according to the face posture and the environment type; weighted fusion of the face features, iris features and fingerprint features according to the weight values ​​to generate the identity authentication result of the corresponding user in the current environment. By acquiring the multimodal biometric data of the user in the current environment, and extracting features from the face image, iris image and fingerprint information, the face features, iris features and fingerprint features are obtained. The weight values ​​corresponding to the face features, iris features and fingerprint features are adjusted according to the face posture of the user in the current environment and the environment type of the current environment, and then the face features, iris features and fingerprint features are weighted and fused according to the weight values ​​to generate the identity authentication result of the corresponding user in the current environment. The embodiment of the present application not only combines multimodal features to authenticate the user, which has high security, but also can dynamically and adaptively adjust the weight values ​​of different modes according to the face posture and environment type of the user's environment to adapt to different user postures and environmental conditions. Since the weights of each mode are adjusted in real time according to different usage scenarios, the weight distribution is more intelligent and adaptable, thereby improving the accuracy and robustness of recognition in complex environments. Based on this, the embodiments of the present application can improve the security and accuracy of user identity authentication, especially in financial scenarios, and can intelligently and accurately authenticate users in different complex environments, helping to reduce the risk of identity theft and fraud, improve the security and accuracy of financial services, and improve customer satisfaction, thereby enhancing the competitiveness of financial enterprises.

[0139] The input / output interface 703 is used to implement information input and output.

[0140] The communication interface 704 is used to realize the communication interaction between this device and other devices. Communication can be realized through wired methods (such as USB, network cable, etc.) or wireless methods (such as mobile network, WI FI, Bluetooth, etc.).

[0141] The bus transmits information between various components of the device (eg, the processor 701 , the memory 702 , the input / output interface 703 , and the communication interface 704 ).

[0142] The processor 701 , the memory 702 , the input / output interface 703 and the communication interface 704 are connected to each other in communication within the device via a bus.

[0143] An embodiment of the present application also provides a computer-readable storage medium, which stores a computer program, and when the computer program is executed by a processor, the above-mentioned identity authentication method is implemented.

[0144] The memory, as a non-transient computer-readable storage medium, can be used to store non-transient software programs and non-transient computer executable programs. In addition, the memory may include a high-speed random access memory, and may also include a non-transient memory, such as at least one disk storage device, a flash memory device, or other non-transient solid-state storage device. In some embodiments, the memory may optionally include a memory remotely disposed relative to the processor, and these remote memories may be connected to the processor via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0145] The identity authentication method, identity authentication device, electronic device and storage medium provided in the embodiment of the present application obtain multimodal biometric data of the user in the current environment, the multimodal biometric data includes face image, iris image and fingerprint information; extract features of the face image through a convolutional neural network to obtain face features; extract features of the iris image and fingerprint information through a support vector machine to generate iris features and fingerprint features; identify the face posture of the user in the current environment and the environment type of the current environment; adjust the weight values ​​corresponding to the face features, iris features and fingerprint features according to the face posture and environment type; perform weighted fusion of the face features, iris features and fingerprint features according to the weight values ​​to generate the identity authentication result of the corresponding user in the current environment. The multimodal biometric data of the user in the current environment is obtained, and the face image, iris image and fingerprint information are extracted to obtain face features, iris features and fingerprint features. And according to the facial posture of the user in the current environment and the environment type of the current environment, the weight values ​​corresponding to the facial features, iris features and fingerprint features are adjusted, and then the facial features, iris features and fingerprint features are weighted and fused according to the weight values ​​to generate the identity authentication result of the corresponding user in the current environment. The embodiment of the present application not only combines multimodal features to authenticate the user, which is highly secure, but also can dynamically and adaptively adjust the weight values ​​of different modes according to the facial posture and environment type of the user's environment to adapt to different user postures and environmental conditions. Since the weights of each mode are adjusted in real time according to different usage scenarios, the weight allocation is more intelligent and adaptable, thereby improving the accuracy and robustness of recognition in complex environments. Based on this, the embodiment of the present application can improve the security and accuracy of user identity authentication, especially in financial scenarios, it can intelligently and accurately authenticate users in different complex environments, help reduce the risk of identity theft and fraud, improve the security and accuracy of financial services, and improve customer satisfaction, thereby enhancing the competitiveness of financial enterprises.

[0146] It will be appreciated by those skilled in the art that all or some of the steps and systems in the disclosed method above may be implemented as software, firmware, hardware and appropriate combinations thereof. Some physical components or all physical components may be implemented as software executed by a processor, such as a central processing unit, a digital signal processor or a microprocessor, or may be implemented as hardware, or may be implemented as an integrated circuit, such as an application specific integrated circuit. Such software may be distributed on a computer-readable medium, which may include a computer storage medium (or a non-transitory medium) and a communication medium (or a temporary medium). As known to those skilled in the art, the term computer storage medium includes volatile and non-volatile, removable and non-removable media implemented in any method or technology for storing information (such as a computer-readable program, a data structure, a program module or other data). Computer storage media include, but are not limited to, RAM, ROM, EEPROM, flash memory or other memory technology, CD-ROM, digital versatile disk (DVD) or other optical disk storage, magnetic cassette, magnetic tape, disk storage or other magnetic storage device, or any other medium that may be used to store desired information and may be accessed by a computer. Furthermore, it is well known to those skilled in the art that communication media generally contain computer-readable programs, data structures, program modules, or other data in a modulated data signal such as a carrier wave or other transport mechanism, and may include any information delivery media.

[0147] The embodiments described in the embodiments of the present application are intended to more clearly illustrate the technical solutions of the embodiments of the present application and do not constitute a limitation on the technical solutions provided in the embodiments of the present application. Those skilled in the art will appreciate that with the evolution of technology and the emergence of new application scenarios, the technical solutions provided in the embodiments of the present application are also applicable to similar technical problems.

[0148] Those skilled in the art will appreciate that the technical solutions shown in the figures do not constitute a limitation on the embodiments of the present application, and may include more or fewer steps than shown in the figures, or a combination of certain steps, or different steps.

[0149] The device embodiments described above are merely illustrative, and the units described as separate components may or may not be physically separated, that is, they may be located in one place or distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0150] Those skilled in the art will appreciate that all or some of the steps in the methods disclosed above, and the functional modules / units in the systems and devices may be implemented as software, firmware, hardware, or a suitable combination thereof.

[0151] The terms "first", "second", "third", "fourth", etc. (if any) in the specification of the present application and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable where appropriate, so that the embodiments of the present application described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any of their variations are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device comprising a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0152] It should be understood that in the present application, "at least one (item)" means one or more, and "plurality" means two or more. "And / or" is used to describe the association relationship of associated objects, indicating that three relationships may exist. For example, "A and / or B" can mean: only A exists, only B exists, and A and B exist at the same time, where A and B can be singular or plural. The character " / " generally indicates that the objects associated before and after are in an "or" relationship. "At least one of the following" or similar expressions refers to any combination of these items, including any combination of single or plural items. For example, at least one of a, b or c can mean: a, b, c, "a and b", "a and c", "b and c", or "a and b and c", where a, b, c can be single or multiple.

[0153] In the several embodiments provided in the present application, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of the above units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0154] The units described above as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0155] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0156] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application is essentially or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including multiple instructions to enable a computer device (which can be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods of various embodiments of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (Read-Only Memory, referred to as ROM), random access memory (Random Access Memory, referred to as RAM), disk or optical disk and other media that can store programs.

[0157] The preferred embodiments of the present invention are described above with reference to the accompanying drawings, but the scope of the rights of the present invention is not limited thereto. Any modification, equivalent substitution and improvement made by a person skilled in the art without departing from the scope and essence of the present invention should be within the scope of the rights of the present invention.

Claims

1. An identity authentication method, characterized in that: The method comprises: Acquire multimodal biometric data of the user in the current environment, wherein the multimodal biometric data includes a face image, an iris image, and fingerprint information; Extracting features from the facial image using a convolutional neural network to obtain facial features; Extracting features from the iris image and the fingerprint information using a support vector machine to generate iris features and fingerprint features; Recognize the facial posture of the user in the current environment and the environment type of the current environment; adjusting the weight values ​​corresponding to the facial features, the iris features, and the fingerprint features respectively according to the facial posture and the environment type; The facial features, the iris features and the fingerprint features are weightedly fused according to the weight values ​​to generate an identity verification result corresponding to the user in the current environment.

2. The method according to claim 1, characterized in that The identifying the facial posture of the user in the current environment and the environment type of the current environment includes: Determining facial posture parameters of the user in a current environment according to the facial image, wherein the facial image is obtained by photographing the user with a camera; Inputting the facial posture parameters into a preset facial posture identification module to identify the facial posture of the user in the current environment; Acquiring environmental condition parameters of the current environment through an environmental sensor; The environmental condition parameters are input into a preset environment identification module to identify the type of environment in which the user is currently located.

3. The method according to claim 2, characterized in that The adjusting the weight values ​​corresponding to the facial features, the iris features, and the fingerprint features respectively according to the facial posture and the environment type includes: determining a scene mode according to the facial gesture and the environment type; Determining a weight value adjustment strategy output by a preset decision fusion layer according to the scene mode; The first weight value corresponding to the facial feature, the second weight value corresponding to the iris feature, and the third weight value corresponding to the fingerprint feature are adjusted based on the weight value adjustment strategy.

4. The method according to claim 3, characterized in that The weight value adjustment strategy includes: When it is determined that the facial posture is facing the camera and the environment type is a high-light environment, increase the first weight value corresponding to the facial feature, and decrease the second weight value corresponding to the iris feature and the third weight value corresponding to the fingerprint feature; When it is determined that the facial posture is facing the camera and the environment type is a low-light environment, reducing the first weight value corresponding to the facial feature, and increasing the second weight value corresponding to the iris feature and the third weight value corresponding to the fingerprint feature; When it is determined that the facial posture is not facing the camera and the environment type is a high-light environment, reducing the first weight value corresponding to the facial feature and the second weight value corresponding to the iris feature, and increasing the third weight value corresponding to the fingerprint feature; When it is determined that the facial posture is not facing the camera and the environment type is a low-light environment, the first weight value corresponding to the facial feature and the second weight value corresponding to the iris feature are reduced, and the third weight value corresponding to the fingerprint feature is increased.

5. The method according to claim 3, characterized in that: The training method of the decision fusion layer includes: Acquiring the facial features, the iris features and the fingerprint features in different scene modes; Standardizing the facial features, the iris features, and the fingerprint features to obtain standardized features; The standardized features are input into the decision fusion layer for training. During the training process, the weights corresponding to the facial features, the iris features and the fingerprint features are tuned by a back propagation algorithm until a preset optimization target is reached, thereby obtaining a weight value adjustment strategy corresponding to different scene modes, wherein the decision fusion layer adopts a multi-layer perceptron model.

6. The method according to claim 3, characterized in that The method further comprises: In the case where the identity authentication result fails to be generated or the confidence level of the identity authentication result is lower than a preset confidence level, recording feedback data corresponding to the user, the feedback data including the multimodal biometric data of the user, the facial posture of the user in the current environment, and environmental condition parameters of the current environment; The weight value adjustment strategy of the decision fusion layer is adjusted according to the feedback data.

7. The method according to claim 6, characterized in that The method further comprises: In the case where the facial gesture is blocked, optimizing the parameters of the facial gesture recognition module by using an occluded face recognition algorithm so that the facial gesture recognition module can recognize the facial gesture of the user; When the illumination parameter in the environmental condition parameter exceeds the preset illumination threshold range, the environment discrimination module is optimized through the illumination compensation mechanism of the face recognition algorithm so that the environment discrimination module can recognize the current environment of the user.

8. An identity verification device, characterized in that: The device comprises: An acquisition module, used to acquire multimodal biometric data of the user in the current environment, wherein the multimodal biometric data includes a face image, an iris image, and fingerprint information; A first extraction module, used for extracting features from the face image through a convolutional neural network to obtain face features; A second extraction module, used for extracting features from the iris image and the fingerprint information by using a support vector machine to generate iris features and fingerprint features; A recognition module, used to recognize the facial posture of the user in the current environment and the environment type of the current environment; An adjustment module, used for adjusting the weight values ​​corresponding to the facial features, the iris features and the fingerprint features respectively according to the facial posture and the environment type; A generation module is used to perform weighted fusion on the facial features, the iris features and the fingerprint features according to the weight values ​​to generate an identity authentication result corresponding to the user in the current environment.

9. An electronic device, characterized in that: The electronic device comprises a memory and a processor, the memory stores a computer program, and the processor implements the identity authentication method according to any one of claims 1 to 7 when executing the computer program.

10. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the identity authentication method according to any one of claims 1 to 7 is implemented.

Images