Biological characteristic authentication method and device

Abstract

The invention discloses a biological characteristic authentication method and device, and belongs to the technical field of electronic equipment, and the method comprises the steps: obtaining a multi-mode biological characteristic and an environment characteristic; wherein the multi-mode biological characteristics are biological characteristic information combined with two or more different modes, and each mode represents an independent biological characteristic type; based on the environment features, determining a fusion weight of a biological feature of each modal in the multi-modal biological features; and based on the fusion weight of the biological feature of each modal in the multi-modal biological features, authenticating the multi-modal biological features to obtain an authentication result.

Description

Technical Field

[0001] The present application belongs to the technical field of electronic equipment, and specifically relates to a biometric authentication method and device. Background Art

[0002] Biometric authentication technology uses an individual's unique physiological characteristics to authenticate their identity. Compared to traditional passwords or tokens, biometric authentication offers uniqueness, irreproducibility, and convenience, making it a key solution for secure authentication. With the continuous development of biometric authentication technology and the advancement of electronic device hardware, its application in electronic devices has become a mainstream trend in security authentication. However, current biometric authentication technology suffers from poor environmental adaptability, resulting in low authentication accuracy. Summary of the Invention

[0003] The purpose of the embodiments of the present application is to provide a biometric authentication method and device, which can improve the adaptability of biometric authentication technology to the environment, thereby improving the accuracy of biometric authentication in different environments.

[0004] In a first aspect, an embodiment of the present application provides a biometric authentication method, applied to an electronic device, the method comprising:

[0005] Acquiring multimodal biometric and environmental characteristics; wherein the multimodal biometric is a combination of biometric information from two or more different modalities, each modality representing an independent biometric type;

[0006] Determining, based on the environmental features, a fusion weight of the biometric features of each modality in the multimodal biometric features;

[0007] Based on the fusion weight of the biometric feature of each modality in the multimodal biometric feature, the multimodal biometric feature is authenticated to obtain an authentication result.

[0008] In a second aspect, an embodiment of the present application provides a biometric authentication device, applied to an electronic device, the device comprising:

[0009] A first acquisition module is configured to acquire multimodal biometric features and environmental features; wherein the multimodal biometric features are biometric information combining two or more different modalities, each modality representing an independent biometric type;

[0010] a determination module, configured to determine a fusion weight of a biometric feature of each modality in the multimodal biometric feature based on the environmental feature;

[0011] An authentication module, configured to authenticate the multimodal biometric features based on the fusion weights of the biometric features of each modality in the multimodal biometric features, and obtain an authentication result.

[0012] In a third aspect, an embodiment of the present application provides an electronic device, which includes a processor and a memory. The memory stores a program or instruction that can run on the processor. When the program or instruction is executed by the processor, the steps of the biometric authentication method described in the first aspect are implemented.

[0013] In a fourth aspect, an embodiment of the present application provides a readable storage medium, on which a program or instruction is stored. When the program or instruction is executed by a processor, the steps of the biometric authentication method described in the first aspect are implemented.

[0014] In a fifth aspect, an embodiment of the present application provides a chip, which includes a processor and a communication interface. The communication interface is coupled to the processor, and the processor is configured to run a program or instruction to implement the biometric authentication method described in the first aspect.

[0015] In a sixth aspect, an embodiment of the present application provides a computer program product, which is stored in a storage medium and is executed by at least one processor to implement the biometric authentication method described in the first aspect.

[0016] In the embodiment of the present application, multimodal biometric features and environmental features are acquired. Among them, the multimodal biometric features are a combination of biometric feature information of two or more different modalities, and each modality represents an independent biometric feature type. Based on the environmental features, the fusion weights of the biometric features of each modality in the multimodal biometric features are determined. Since the fusion weights of the biometric features of each modality are determined based on the environmental features and can be adaptively adjusted as the authentication environment changes, authenticating the multimodal biometric features based on the fusion weights of the biometric features of each modality in the multimodal biometric features can improve the adaptability of the biometric authentication technology to the environment, thereby improving the accuracy of biometric authentication in different environments and reducing the false alarm rate of authentication. Description of the Drawings

[0017] Figure 1 is one of the flowcharts of a biometric authentication method provided by some embodiments of the present application;

[0018] Figure 2 is the second flowchart of a biometric authentication method provided by some embodiments of the present application;

[0019] Figure 3 is the structural block diagram of a biometric authentication device provided by an embodiment of the present application;

[0020] Figure 4 It is a schematic structural diagram of an electronic device provided by an embodiment of the present application;

[0021] Figure 5 It is a schematic hardware structure diagram of an electronic device for implementing various embodiments of the present application. Specific embodiments

[0022] Next, the technical solutions in the embodiments of the present application will be clearly described in conjunction with the accompanying drawings in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all the embodiments obtained by those of ordinary skill in the art belong to the scope of protection of the present application.

[0023] The terms "first", "second", etc. in the specification and claims of the present application are used to distinguish similar objects, rather than to describe a specific order or sequence. It should be understood that such terms can be interchanged under appropriate circumstances so that the embodiments of the present application can be implemented in an order other than those illustrated or described herein, and the objects distinguished by "first", "second", etc. are usually of the same category, and the number of objects is not limited. For example, the first object can be one or multiple. In addition, "and / or" in the specification and claims means at least one of the connected objects, and the character " / " generally indicates an "or" relationship between the associated objects before and after.

[0024] For the sake of easy understanding, some related concepts and application scenarios involved in the embodiments of the present application will be introduced first below.

[0025] I. Related concepts

[0026] A large model is a deep learning model with a large number of parameters (usually in the order of billions to trillions). Through pre-training on a large amount of data (labeled or unlabeled data such as Internet text, images, code, etc.), it has powerful language understanding, generation, reasoning, and cross-modal processing capabilities.

[0027] The Trusted Execution Environment (TEE) is an isolated area on the device's main processor that is separated from the system's main operating system (OS). Through a combination of hardware and software, it provides security for sensitive data and computations. Among them, the TEE provides an independent encrypted memory area (such as Enclave or SecureWorld), and external programs cannot access or tamper with the data therein; the data is processed in plaintext inside the TEE, but only the encrypted form can be obtained during external access to ensure confidentiality; the code running inside the TEE is independent of the main operating system. Even if the main operating system is attacked, the TEE can still ensure data security; it supports Remote Attestation, allowing external users to verify whether the TEE is running trusted code; the TEE ensures that the code and data are not tampered with during execution and adopts the SealedStorage mechanism to ensure the security of sensitive data even after the device is powered off.

[0028] The expert module is an intelligent network that combines expert knowledge with machine learning models. Aimed at enhancing the evaluation and analysis capabilities of complex features through the synergistic effect of the experience of domain experts and model algorithms, this network can more accurately evaluate the importance, effectiveness of features and their contributions to target tasks (such as classification, prediction or decision-making) by integrating the qualitative judgments of experts on features and the quantitative analysis of the model.

[0029] The Attention Mechanism is a technology that mimics the way human attention is allocated. When processing sequence data (such as natural language, speech, images, etc.), it dynamically focuses on different parts of the input sequence to capture key information and improve the performance of the model. The attention mechanism can include the following two main forms: Self-Attention and Cross-Attention. Among them, Self-Attention allows the model to capture the internal dependencies within the sequence when processing the sequence, and Cross-Attention allows the model to refer to the information of another sequence when processing one sequence, which is used to establish the alignment relationship between different sequences.

[0030] II. Application Scenarios

[0031] In traditional biometric authentication methods, most are single-modal solutions such as fingerprint and face recognition. However, single-modal solutions are vulnerable to forgery attacks (such as 3D masks, high-definition pictures, etc.). In existing biometric authentication methods, most adopt multi-modal static fusion solutions, that is, fixed weights are used to fuse multi-modal biometric features (such as fixed-weight face + voiceprint). However, the static fusion solution cannot dynamically adapt to environmental changes (such as a decrease in face recognition rate in insufficient light). At the same time, existing biometric authentication methods can, to a certain extent, combat forgery attacks by relying on a rule base or a lightweight model, but it is difficult to cope with new deep forgery attacks in the current era of large models. It can be seen that the biometric authentication methods in related technologies have the following deficiencies and defects:

[0032] 1) Poor environmental adaptability: The environmental adaptability of single-modal biometric authentication methods and multi-modal biometric authentication methods with fixed-weight fusion is poor. When environmental changes such as light and noise cause fluctuations in the reliability of a single biometric feature, the fixed-weight fusion strategy is prone to misjudgment (for example, when face recognition fails under strong light, the voiceprint weight is not increased in time), and the authentication accuracy cannot be dynamically optimized.

[0033] 2) Lagging defense means: Forgery attacks rely on a known attack feature library, have weak detection capabilities for zero-day attacks (such as forged biometric features generated by large models), and the defense model update cycle is long.

[0034] 3) Insufficient cross-layer security isolation: The processing of biometric features such as extraction, fusion, and authentication is performed in an insecure environment, and an end-to-end TEE protection chain cannot be formed, resulting in the risk of side-channel attacks.

[0035] Next, in conjunction with the accompanying drawings, the biometric authentication method provided by the embodiments of the present application will be described in detail.

[0036] It should be noted that the biometric authentication method provided by the embodiments of the present application is applicable to electronic devices. In practical applications, the electronic device may include: mobile terminals such as smartphones, tablets, smart watches, and personal digital assistants, or may also include: computer devices such as laptop computers, desktop computers, and desktops. The embodiments of the present application do not make any limitations in this regard.

[0037] Figure 1 is one of the flowcharts of a biometric authentication method provided by some embodiments of the present application, which is applied to an electronic device, such as Figure 1 shown. The method may include the following steps: Step 101, Step 102, and Step 103.

[0038] In Step 101, multi-modal biometric features and environmental features are obtained, where the multi-modal biometric features are the combination of biometric feature information of two or more different modalities, and each modality represents an independent biometric feature type.

[0039] In the embodiments of the present application, the multi-modal biometric features may include at least two of the following: fingerprint feature, palmprint feature, voiceprint feature, face feature, iris feature, retina feature, and vein feature.

[0040] In the embodiments of the present application, different multi-modal biometric features may be indicated for the user to input according to different authentication scenarios. For example, for a financial scenario of large-amount transactions, the user may be instructed to input fingerprint feature, voiceprint feature, and face feature; for a daily unlocking scenario, the user may be instructed to input face feature and fingerprint feature.

[0041] It can be seen that in the embodiments of the present application, authentication can be performed for different combinations of biometric features. Since different combinations of biometric features usually correspond to different authentication scenarios, the applicable authentication scenarios are relatively wide and the flexibility is relatively high.

[0042] In the embodiments of the present application, the environmental features may include at least one of the following: environmental illumination feature, environmental noise feature, environmental humidity feature, environmental temperature feature, and environmental location feature.

[0043] In the embodiments of the present application, different environmental features may be obtained according to different biometric features required for authentication. For example, for face feature, the environmental illumination feature may be obtained; for voiceprint feature, the environmental noise feature may be obtained; for fingerprint feature, the environmental humidity feature may be obtained, etc.

[0044] It can be seen that in the embodiments of the present application, authentication can be performed for different combinations of biometric features. Since different combinations of biometric features are affected by different environmental factors to different degrees and different combinations of biometric features usually correspond to different authentication scenarios, the applicable authentication scenarios are relatively wide and the flexibility is relatively high.

[0045] In some embodiments, step 101 may include the following steps: step 1011 and step 1012.

[0046] In step 1011, environmental data of the environment where the electronic device is located is obtained through an environmental sensor, and environmental features are obtained according to the environmental data.

[0047] In the embodiments of the present application, the environmental sensor includes but is not limited to: illumination sensor, temperature sensor, humidity sensor, noise sensor, and position sensor.

[0048] Exemplarily, the environmental illumination intensity of the environment where the electronic device is located is obtained through an illumination sensor, and the obtained illumination intensity is converted into an environmental illumination feature through the following formula (1):

[0049]

[0050] Among them, L actual is the obtained light intensity, L max 10000 lux, L min 0lux, L normalized It is the ambient lighting feature.

[0051] For example, the ambient noise of the electronic device is obtained through a noise sensor, the noise decibel of the ambient noise is calculated using A-weighted sound level, and interference signals with a frequency less than 100 Hz in the noise decibel are filtered out to obtain the ambient noise characteristics.

[0052] For example, a location sensor such as GPS is used to obtain the current location of the electronic device's environment, and the current location is compared with historical common locations, and the comparison result is used as the environmental location feature.

[0053] Exemplarily, the temperature of the environment in which the electronic device is located is acquired through a temperature sensor, and the acquired temperature is normalized to obtain an environmental temperature characteristic.

[0054] For example, the humidity of the environment in which the electronic device is located is obtained through a humidity sensor, and the obtained humidity is normalized to obtain an environmental humidity feature.

[0055] In step 1012, multimodal biometric data input by the user to the electronic device is obtained through a biometric sensor, and multimodal biometric features are obtained based on the multimodal biometric data.

[0056] In the embodiment of the present application, the biometric sensor includes but is not limited to: a fingerprint sensor, a microphone, and a camera.

[0057] Exemplarily, a fingerprint sensor is used to collect a user's fingerprint image, and fingerprint features are obtained from the fingerprint image.

[0058] For example, the user's voice is collected through a microphone, and the voiceprint feature is obtained from the voice.

[0059] Exemplarily, a facial image of the user is captured by a camera, and facial features are obtained from the facial image.

[0060] Exemplarily, a camera is used to capture an image of the user's eyes, and iris features and retinal features are obtained from the image of the eyes.

[0061] For example, a camera is used to capture an image of the user's palm, and palm print features are obtained from the palm image.

[0062] Exemplarily, a user's vein image is captured by a camera, and vein features are obtained from the vein image.

[0063] In the embodiments of the present application, considering the security of data, the data obtained by the above sensors can be transmitted out through the TEE secure channel to the TEE environment, and the environmental features and multimodal biometric features can be obtained within the TEE environment.

[0064] In step 102, based on the environmental features, the fusion weights of the biometric features of each modality in the multimodal biometric features are determined.

[0065] Considering that in the multi-biometric authentication scheme with fixed weight fusion, the fusion weights of different biometric features (such as the fusion weight of face features is 60% and the fusion weight of voiceprint features is 40%) are fixed during system design and cannot be dynamically adjusted according to the real-time environment. When the reliability of a certain biometric feature decreases, the system still distributes results according to the original weight, which may lead to authentication errors. For example, face recognition fails under strong light: the system does not timely increase the voiceprint weight and still relies on the face recognition result, resulting in misjudgment. Another example is that voice recognition fails under noise interference: the system does not reduce the voiceprint weight and still relies on voice data, resulting in authentication failure. In view of the above problems, in the embodiments of the present application, the environmental changes (such as light intensity, noise level) will be monitored in real time, and the fusion weights of the biometric features of each modality will be dynamically adjusted according to the environmental changes. For example, in a strong light environment, the face recognition weight is reduced, and the voiceprint or iris recognition weight is increased; in a low-noise environment, voiceprint recognition is preferentially used, and the dependence on fingerprint or iris recognition is reduced to improve the robustness of authentication.

[0066] In the embodiments of the present application, the environmental features can be analyzed in real time by a large model to dynamically adjust the fusion weights of the biometric features of each modality. Since the large model has powerful reasoning and cross-modal processing capabilities, it can accurately determine the fusion weights adapted to the current environment of the electronic device.

[0067] In step 103, based on the fusion weights of the biometric features of each modality in the multimodal biometric features, the multimodal biometric features are authenticated to obtain an authentication result.

[0068] In the embodiments of the present application, the authentication result may include: authentication passed, authentication not passed.

[0069] In the embodiments of the present application, the multimodal biometric features can be fused according to the fusion weights of the biometric features of each modality in the multimodal biometric features, and the fused biometric features are authenticated to obtain an authentication result.

[0070] In the embodiments of the present application, adversarial samples (such as face images with added noise) can be generated using a large model. The defense model is pre-trained with the adversarial samples generated by the large model. The fused biometric features are input into the defense model, and the defense model authenticates the fused biometric features. Since the adversarial samples generated by the large model cover new types of deepfake attacks in the large model era, the defense model has strong generalization ability and can identify new types of deepfake attacks, and the finally output authentication result is relatively accurate. <> <>

[0071] In the embodiments of the present application, steps 101 to 103 above can all be executed by a large model; among them, the large model is a pre-trained deep learning model. Since the large model has powerful language understanding, generation, reasoning, and cross-modal processing capabilities, executing the above steps by the large model can improve the accuracy of the finally output authentication result. <> <>

[0072] In the embodiments of the present application, steps 101 to 103 above can all be executed in the TEE environment of the electronic device to form an end-to-end TEE protection chain. Since TEE provides an independent encrypted memory area and external programs cannot access or tamper with the data therein, the entire authentication process can achieve hardware-level isolation of physical attacks (such as JTAG debugging) and software attacks (such as memory injection, malicious code interference at the operating system layer), realizing DMA attack protection and memory isolation, thereby ensuring the security of the entire authentication process. <> <>

[0073] As can be seen from the above embodiments, in this embodiment, multi-modal biometric features and environmental features are obtained; among them, the multi-modal biometric features are the combination of biometric feature information of two or more different modalities, and each modality represents an independent biometric feature type; based on the environmental features, the fusion weights of the biometric features of each modality in the multi-modal biometric features are determined; since the fusion weights of the biometric features of each modality are determined based on the environmental features and can be adaptively adjusted with the change of the authentication environment, authenticating the multi-modal biometric features based on the fusion weights of the biometric features of each modality in the multi-modal biometric features can improve the adaptability of the biometric authentication technology to the environment, thereby improving the accuracy of biometric authentication in different environments and reducing the false alarm rate of authentication. <> <>

[0074] <> Figure 2 is the second flowchart of a biometric authentication method provided by some embodiments of the present application, applied to an electronic device, such as <> Figure 2 shown, the method may include the following steps: step 201, step 202, and step 203. <> <>

[0075] In step 201, multimodal biometric features, environmental features, and behavioral features are obtained. Among them, multimodal biometric features are the combination of biometric feature information of two or more different modalities, and each modality represents an independent biometric feature type.

[0076] In the embodiments of the present application, the multimodal biometric features may include at least two of the following: fingerprint feature, palmprint feature, voiceprint feature, face feature, iris feature, retina feature, vein feature.

[0077] In the embodiments of the present application, the environmental features may include at least one of the following: environmental light feature, environmental noise feature, environmental humidity feature, environmental temperature feature, environmental location feature.

[0078] In the embodiments of the present application, the behavioral features may include at least one of the following: holding feature, touch feature; among them, the holding feature includes at least one of the following: the posture of holding the electronic device, the force of holding the electronic device.

[0079] In the embodiments of the present application, since the attack behaviors of black production are usually different from the behaviors of real human operations. For example, when black production forges an attack, it usually fixes the electronic device on a device bracket, and the pose of the electronic device is usually fixed and unchanged. While during real human operation, the electronic device is usually held by hand, and the pose of the electronic device may change, and the holding force may also change. Another example is that when black production forges a touch attack, the behavior of touching the touch screen of the electronic device is also different from the behavior of a real person touching the touch screen of the electronic device. Therefore, in order to further ensure the accuracy rate of the authentication result, when determining the fusion weights of biometric features of each modality, in addition to referring to environmental features, behavioral features can also be referred to.

[0080] In some embodiments, the above step 201 may include: step 2011, step 2012, and step 2013.

[0081] In step 2011, environmental data of the environment where the electronic device is located is obtained through an environmental sensor, and environmental features are obtained based on the environmental data.

[0082] In step 2012, multimodal biometric data input by the user for the electronic device is obtained through a biometric sensor, and multimodal biometric features are obtained based on the multimodal biometric data.

[0083] In step 2013, behavioral data of the user is obtained based on a motion sensor and / or a pressure sensor, and behavioral features are obtained based on the behavioral data.

[0084] In the embodiments of the present application, the motion sensor includes but is not limited to: a three-axis gyroscope, an accelerometer.

[0085] Exemplarily, the pitch angle, yaw angle, and roll angle of the electronic device are obtained through a three-axis gyroscope and an accelerometer, and the posture of the user holding the electronic device is determined according to the above pitch angle, yaw angle, and roll angle.

[0086] Exemplarily, the force of the user holding the electronic device is obtained through a pressure sensor arranged on the edge of the outer frame of the electronic device.

[0087] Exemplarily, through a pressure sensor arranged under the touch screen of the electronic device, the change of the pressure signal when the user touches the screen is obtained, the change of the pressure signal is analyzed, and a touch feature is obtained, where the touch feature includes: touch behavior and its timing.

[0088] In the embodiments of the present application, considering the security of data, the data obtained by the above sensors can be transmitted out through the TEE secure channel to the TEE environment, and the environmental features, behavioral features, and multimodal biometric features are obtained in the TEE environment.

[0089] In step 202, based on the environmental features and behavioral features, the fusion weights of the biometric features of each modality in the multimodal biometric features are determined.

[0090] In the embodiments of the present application, the environmental features and behavioral features can be analyzed in real time through a large model, and the fusion weights of the biometric features of each modality can be dynamically adjusted. Since the large model has powerful reasoning and cross-modal processing capabilities, it can accurately determine the fusion weights adapted to the current environment of the electronic device.

[0091] In some embodiments, the above step 202 may include the following steps: step 2021 and step 2022.

[0092] In step 2021, at least one of the behavioral features and environmental features is verified for spatio-temporal consistency.

[0093] Spatio-temporal consistency verification is a process of ensuring the consistency and accuracy of data or information in the time and space dimensions. In the embodiments of the present application, by performing behavioral pattern analysis in the time dimension and feature matching degree verification in the space dimension on at least one of the behavioral features and environmental features, a multi-dimensional three-dimensional protection system is constructed to prevent biometric forgery.

[0094] Exemplarily, for the time consistency verification of touch behaviors, a sliding window statistical modeling and an adaptive window length adjustment algorithm can be adopted to ensure that fast gestures can still capture effective features. At the same time, a Hidden Markov Model (HMM) is used for verification. Five touch behaviors (rest, tap, slide, press, multi-finger interaction) are defined, and the state sequence is calculated through the Viterbi algorithm. A score is given to each state sequence. If the score is greater than a pre-set threshold (e.g., 60 points), the verification is considered passed; otherwise, it is considered failed. For example, when the state sequence "rest → press → slide" appears, the score is 40 points. Since 40 points is lower than 60 points, the verification is considered failed.

[0095] Exemplarily, for the spatial consistency verification, the projection relationship from the touch coordinates to the face position is established through the camera calibration matrix, and the coincidence degree between the actual face position and the touch projection area is verified. If the coincidence degree is greater than a pre-set threshold (e.g., 60%), the verification is considered passed; otherwise, it is considered failed.

[0096] In step 2022, when the spatio-temporal consistency verification is passed, the fusion weights of the biometric features of each modality in the multi-modal biometric features are determined according to the environmental features and behavioral features.

[0097] In the embodiment of the present application, the passing of the spatio-temporal consistency verification indicates a lower possibility of biometric forgery. Further, according to the environmental features and behavioral features, the fusion weights of the biometric features of each modality in the multi-modal biometric features are determined.

[0098] It can be seen that in the embodiment of the present application, the spatio-temporal consistency verification can be used as a link in biometric anti-counterfeiting, and a dynamic defense system is constructed through spatio-temporal multi-dimensional cross-verification, so as to achieve active defense against deep fake attacks and improve the dynamic liveness detection ability.

[0099] In some embodiments, when determining the fusion weights of the biometric features of each modality in the multi-modal biometric features according to the environmental features and behavioral features, the heterogeneous feature alignment of the behavioral features and the multi-modal biometric features can be performed first; then, through the self-attention network, each aligned feature is enhanced to obtain the enhanced features; then, through the cross-attention network, the cross-attention calculation is performed on every two enhanced features to obtain the attention weights of the biometric features of each modality in the multi-modal biometric features; then, through the environmental adaptation network, the environmental features are processed to obtain the environmental weights; finally, according to the environmental weights, the attention weights of the biometric features of each modality in the multi-modal biometric features are corrected to obtain the fusion weights of the biometric features of each modality in the multi-modal biometric features.

[0100] In the embodiments of the present application, the self-attention network is used to enhance the features within each modality through self-attention. For example, the temporal data of touch features captures long-range dependencies through self-attention.

[0101] In the embodiments of the present application, the cross-attention network is used for the interaction between different modalities. For example, how touch features perform cross-attention calculations with voiceprint or face features.

[0102] In the embodiments of the present application, the environment adaptation network is used to assign corresponding environment weights to each environmental feature.

[0103] In the embodiments of the present application, the self-attention network, the cross-attention network, and the environment adaptation network can all be networks driven by large models.

[0104] It can be seen that in the embodiments of the present application, through the attention mechanism and environment adaptation gating, environmental features, behavior features, and multi-modal biometric features can be processed to obtain the fusion weights of the biometric features of each modality in the multi-modal biometric features. Since the attention mechanism can enhance and fuse various features, and the environment adaptation gating can accurately adjust the fusion ratio of each biometric feature according to environmental changes, accurate fusion weights can be obtained.

[0105] In step 203, based on the fusion weights of the biometric features of each modality in the multi-modal biometric features, the multi-modal biometric features are authenticated to obtain an authentication result.

[0106] In the embodiments of the present application, the authentication result may include: authentication passed, authentication failed.

[0107] In the embodiments of the present application, the multi-modal biometric features can be fused according to the fusion weights of the biometric features of each modality in the multi-modal biometric features, and the fused biometric features are authenticated to obtain an authentication result.

[0108] In some embodiments, the above step 203 may include the following steps: step 2031.

[0109] In step 2031, the biometric features of each modality in the multi-modal biometric features and the corresponding fusion weights are input into the defense model for processing, and an authentication result is output; wherein, the defense model includes: at least two different biometric expert modules and a decision module. Different biometric expert modules are used to make decisions on different combinations of biometric features based on different combinations of biometric features and the corresponding fusion weights, and the decision module is used to generate an authentication result based on the decision results of all expert modules of the defense model.

[0110] In the embodiments of the present application, the defense model can adopt a multi-expert voting mechanism to construct a heterogeneous decision-making system, balance security and user experience, and achieve dynamic scenario adaptive authentication.

[0111] In the embodiments of the present application, the defense model may further include: a behavior feature expert module, where the behavior feature expert module is used to make decisions on behavior features.

[0112] Exemplarily, the expert module and its decision-making strategy are shown in Table 1 below.

[0113]

[0114] Table 1

[0115] In the embodiments of the present application, the decision result of each expert module is usually a numerical value. If the numerical value is greater than the threshold of the decision-making strategy of the expert module, it is considered that the expert module holds an agreement opinion; otherwise, it is considered that the expert module holds an opposition opinion.

[0116] In the embodiments of the present application, if all expert modules hold an agreement opinion, the authentication result output by the decision module is authentication passed. If the opinions of the expert modules conflict, the decision module performs a weighted sum of the decision results of each expert module. If the sum result is greater than a set threshold (such as 0.5), the authentication result output by the decision module is authentication passed; otherwise, the authentication result output by the decision module is not authenticated passed. If the expert module continuously opposes, hardware-level verification is triggered.

[0117] In some embodiments, step 2031 may include the following steps: step 20311.

[0118] In step 20311, according to the second security mechanism, the biometric feature of each modality in the multi-modal biometric feature and the corresponding fusion weight are input into the defense model for processing, and an authentication result is output; where the second security mechanism includes at least one of the following: a gradient obfuscation protection mechanism and a model dynamic fingerprint mechanism.

[0119] In the embodiments of the present application, the gradient obfuscation protection mechanism is used to inject random noise in the forward propagation of the defense model. By injecting random noise in the forward propagation process, it interferes with the attacker's analysis of the model gradient (such as adversarial sample generation). Since it is difficult for the attacker to accurately estimate the true gradient direction, the difficulty for the attacker to construct effective adversarial samples is increased, thereby enhancing the security of the defense model.

[0120] Exemplarily, when the input data x is propagated forward, random noise ∈ is dynamically injected, and the calculation formula is as follows formula (2):

[0121]

[0122] Where is the gradient of the loss function J with respect to the input x, sign() represents the sign function, and ∈~N(0,0.1) to ensure that the noise amplitude ∈ does not significantly affect the accuracy of the normal inference of the model.

[0123] In the embodiments of the present application, the model dynamic fingerprint mechanism is used to generate a unique model variant for each electronic device to enhance the security and privacy of the model. The model dynamic fingerprint mechanism makes the models on each device have a certain uniqueness, so that even if an attacker can obtain the model parameters of a certain device, it is difficult to generalize them to other devices.

[0124] Exemplarily, for each device, the weights W′ of its fully connected layer are randomly perturbed through the following formula (3):

[0125] W′ = W + ΔW (||ΔW|| F <0.01) (3)

[0126] where ||ΔW|| F represents the Frobenius norm of ΔW, which is used to limit the amplitude of the perturbation, and ||ΔW|| F <0.01.

[0127] In the embodiments of the present application, the parameters of the defense model can also be updated regularly (such as every 72 hours) to enable the model to adapt to new data distributions and attack methods, while reducing the risk of the model being attacked for a long time.

[0128] It can be seen that in the embodiments of the present application, by combining gradient obfuscation protection and model dynamic fingerprint technology, a more perfect defense response mechanism can be constructed. Gradient obfuscation protection can interfere with the attacker's gradient analysis during the forward propagation process, while the model dynamic fingerprint can increase the security and privacy of the model by generating unique model variants and regularly updating model parameters. This comprehensive application method can effectively improve the model's ability to resist various attacks and protect the security and stability of the model.

[0129] In some embodiments, the defense model can be trained based on synthetic attack samples and real attack samples generated by a large model for image generation.

[0130] In the embodiments of the present application, adversarial samples (such as face images and fingerprint images with added noise) can be generated using a large model to train the generalization ability of the defense model and resist new deepfake attacks.

[0131] Exemplarily, a conditional Wasserstein GAN architecture is adopted, which includes: a fingerprint generator and a physical property simulator. First, random noise and a fingerprint material label are input into the fingerprint generator. The fingerprint generator outputs a fingerprint texture map according to the random noise and the fingerprint material label. Then, the fingerprint texture map is input into the physical property simulator to simulate the fingerprint acquisition characteristics of a capacitive / optical sensor, and a forged fingerprint image is output by the physical property simulator.

[0132] Exemplarily, a three-dimensional differentiable rendering architecture is adopted, and the NeRF framework is used to generate face attack samples with continuously changing viewpoints. Sampling in the lighting parameter space: L ∈ R 9 (including ambient light, directional light intensity and angle), and a forged face image is generated.

[0133] In the embodiments of the present application, adversarial samples (such as noise-added face images, synthetic fingerprint images, etc.) can be generated by a large model. Through the adversarial samples generated by the large model, a defense model is pre-trained. The fused biometric features are input into the defense model, and the defense model authenticates the fused biometric features. Since the adversarial samples generated by the large model cover new deepfake attacks in the large model era, the generalization ability of the defense model is strong, and it can identify new deepfake attacks, and the final authentication result output is relatively accurate.

[0134] In some embodiments, the training process of the above defense model may include the following steps: extracting the spatial features and frequency domain features of synthetic attack samples and real attack samples; fusing the spatial features and frequency domain features to obtain fused features; training the network to be trained based on the fused features to obtain the defense model.

[0135] Exemplarily, first, an adaptive sample mixing strategy is adopted, as shown in the following formula (4), to dynamically adjust the ratio ρ of real attack samples to synthetic attack samples t :

[0136] P t = 0.3 + 0.2·sin(2πt / T) (4)

[0137] where T = 1000. Then, a 3D convolution kernel is used to extract the spatial features of synthetic attack samples and real attack samples. Through a complex convolution layer, the frequency domain features of synthetic attack samples and real attack samples are extracted. The spatial features and frequency domain features are fused through dynamic weighting by an attention mechanism to obtain fused features. Finally, the network to be trained is iteratively trained multiple times based on the fused features, the loss function is calculated, and the model parameters are updated through the loss function until the training conditions are met to obtain the defense model.

[0138] It can be seen that in the embodiments of the present application, when training the defense model, features in two different dimensions of the spatial domain and the frequency domain can be combined for training. Since spatial features can reflect the form of an object in the real space and frequency domain features can reflect the characteristics of an object in the frequency space, both can capture local and global information simultaneously, avoiding the limitations of a single feature domain, improving the feature expression ability, generalization ability, and higher adversarial robustness.

[0139] In some embodiments, in the case where the authentication result fails the authentication, the multimodal biometric features are added to the sample library for fine-tuning of the defense model.

[0140] In the embodiments of the present application, the update protocol of the defense model can adopt secure differential update.

[0141] Exemplarily, the patch is signed by Elliptic Curve Cryptography (ECDSA-384), the model patch is generated in the cloud using formula (5), and the patch is applied after verifying the signature within the TEE using formula (6).

[0142] Δθ = θ new -θ 0ld (5)

[0143]

[0144] In the embodiments of the present application, the sample library can adopt a sample deduplication mechanism such as locality-sensitive hashing to avoid the accumulation of redundant samples.

[0145] It can be seen that in the embodiments of the present application, after a new attack sample is detected by the defense model, the attack sample can be added to the sample library for fine-tuning of the defense model to improve the authentication ability of the defense model for biometric features.

[0146] In the embodiments of the present application, the above steps 201 to 203 can all be executed by a large model; where the large model is a pre-trained deep learning model. Since the large model has powerful language understanding, generation, reasoning, and cross-modal processing capabilities, executing the above steps by the large model can improve the accuracy of the final authentication result.

[0147] In the embodiments of the present application, the above steps 201 to 203 can all be executed in the TEE environment of the electronic device to form an end-to-end TEE protection chain. Since the TEE provides an independent encrypted memory area and external programs cannot access or tamper with the data therein, the entire authentication process can achieve hardware-level isolation of physical attacks (such as JTAG debugging) and software attacks (such as memory injection, malicious code interference at the operating system layer), achieve DMA attack protection and memory isolation, and ensure the security of the entire process.

[0148] As can be seen from the above embodiments, in this embodiment, multimodal biometric features, environmental features, and behavioral features are obtained; among them, the multimodal biometric features are the combination of biometric feature information of two or more different modalities, and each modality represents an independent biometric feature type; based on the environmental features and behavioral features, the fusion weights of the biometric features of each modality in the multimodal biometric features are determined; since the fusion weights of the biometric features of each modality are determined based on the environmental features and can be adaptively adjusted with the change of the authentication environment and the user's behavior, therefore, based on the fusion weights of the biometric features of each modality in the multimodal biometric features, the multimodal biometric features are authenticated, which can improve the adaptability of biometric authentication to the environment, thereby improving the accuracy of biometric authentication in different environments and reducing the false positive rate of authentication.

[0149] In some other embodiments provided by the present application, for the biometric authentication method provided, after the above step 103 or step 203, the following steps may also be added:

[0150] Based on the first security mechanism, an authentication result is output; where the first security mechanism includes at least one of the following: a multi-signature mechanism and a context binding mechanism.

[0151] In the embodiments of the present application, the multi-signature mechanism may be: performing triple signature within the TEE, such as hardware key signature, biometric binding signature, and environmental context signature.

[0152] In the embodiments of the present application, the context binding mechanism may be: at runtime, generating a hash fingerprint according to the current process PID, CR3 register value (such as the memory page table base address), EFLAGS register status (such as interrupt flag, arithmetic flag bit), TSC timestamp counter low 32-bit entropy value, and the hardware root key generated based on PUF, and generating a new fingerprint every millisecond to effectively prevent replay attacks.

[0153] It can be seen that in the embodiments of the present application, after obtaining the authentication result, the credibility of the decision result can be guaranteed through the multi-signature and context binding mechanisms.

[0154] In some other embodiments provided by the present application, for the biometric authentication method provided, on the basis of Figure 2 the embodiments shown, the following steps may also be added:

[0155] At least one of the multimodal biometric features, environmental feature data, behavioral features, the fusion weights of the biometric features of each modality, and the authentication result is cached hierarchically; where different data adopts different levels of caching methods.

[0156] Exemplarily, for some scenarios with high requirements for latency and accuracy, the data involved in the authentication process can be cached in different levels, and the mapping relationship between the cache levels and the cache methods is shown in Table 2 below.

[0157] Cache level Cache method L1 High-frequency decision mode hash L2 Scenario-based decision template L3 Complete decision tree

[0158] Table 2

[0159] In the embodiments of the present application, by caching the data in each biometric authentication process, it is convenient to query whether there is reference data in the cached data when there is a new biometric authentication later. If there is reference data, duplicate data processing can be avoided, thus saving computing resources.

[0160] In summary, in the embodiments of the present application, by dynamically adjusting the weights of multi-modal biometric features based on the large model, environmental adaptability is achieved, the accuracy of authentication can be improved, and the false positive rate of authentication can be reduced; by training the defense model with adversarial samples generated by the large model, the generalization ability of the defense model is improved, and it can resist new deep forgery attacks, enhancing the forward-looking nature of the defense; the entire process from data collection to result output is protected by the TEE. By blocking attack means such as physical probe injection and memory scanning through the TEE, the security of the authentication process is improved.

[0161] The biometric authentication method provided by the embodiments of the present application may be executed by a biometric authentication device. In the embodiments of the present application, taking the biometric authentication device executing the biometric authentication method as an example, the biometric authentication device provided by the embodiments of the present application is described.

[0162] Figure 3 is a structural block diagram of a biometric authentication device provided by some embodiments of the present application, as Figure 3 shown. The biometric authentication device 300 may include: a first acquisition module 301, a determination module 302, and an authentication module 303.

[0163] The first acquisition module 301 is configured to acquire multi-modal biometric features and environmental features; wherein, the multi-modal biometric features are combined with biometric feature information of two or more different modalities, and each modality represents an independent biometric feature type;

[0164] The determination module 302 is configured to determine the fusion weight of the biometric features of each modality in the multi-modal biometric features based on the environmental features;

[0165] The authentication module 303 is configured to authenticate the multi-modal biometric features based on the fusion weight of the biometric features of each modality in the multi-modal biometric features to obtain an authentication result.

[0166] As can be seen from the above embodiments, in this embodiment, multi-modal biometric features and environmental features are obtained; wherein, the multi-modal biometric features are the combination of biometric feature information of two or more different modalities, and each modality represents an independent biometric feature type; based on the environmental features, the fusion weights of the biometric features of each modality in the multi-modal biometric features are determined; since the fusion weights of the biometric features of each modality are determined based on the environmental features and can be adaptively adjusted as the authentication environment changes, therefore, based on the fusion weights of the biometric features of each modality in the multi-modal biometric features, authenticating the multi-modal biometric features can improve the adaptability of biometric authentication to the environment, thereby improving the accuracy rate of biometric authentication in different environments and reducing the false positive rate of authentication.

[0167] Optionally, as an embodiment, the biometric authentication device 300 may further include:

[0168] A second acquisition module, configured to acquire behavioral features;

[0169] The determination module 302 may include:

[0170] A determination sub-module, configured to determine the fusion weights of the biometric features of each modality in the multi-modal biometric features according to the environmental features and the behavioral features.

[0171] Optionally, as an embodiment, the determination sub-module may include:

[0172] A verification unit, configured to perform spatio-temporal consistency verification on at least one of the behavioral features and the environmental features;

[0173] A determination unit, configured to determine the fusion weights of the biometric features of each modality in the multi-modal biometric features according to the environmental features and the behavioral features when the spatio-temporal consistency verification is passed.

[0174] Optionally, as an embodiment, the determination sub-module may include:

[0175] An alignment processing unit, configured to perform heterogeneous feature alignment processing on the behavioral features and the multi-modal biometric features;

[0176] An enhancement processing unit, configured to enhance each aligned feature through a self-attention network to obtain enhanced features;

[0177] A cross-processing unit, configured to perform cross-attention calculation on every two enhanced features through a cross-attention network to obtain the attention weights of the biometric features of each modality in the multi-modal biometric features;

[0178] An adaptation processing unit, configured to process the environmental features through an environment adaptation network to obtain an environmental weight;

[0179] A correction unit, configured to correct the attention weight of the biometric feature of each modality in the multi-modal biometric feature according to the environmental weight to obtain the fusion weight of the biometric feature of each modality in the multi-modal biometric feature.

[0180] Optionally, as an embodiment, the authentication module 303 may include:

[0181] A processing sub-module, configured to input the biometric feature of each modality in the multi-modal biometric feature and the corresponding fusion weight into a defense model for processing, and output an authentication result;

[0182] Wherein, the defense model includes: at least two different biometric expert modules and a decision module. The different biometric expert modules are configured to make decisions on different combinations of biometric features based on different combinations of biometric features and the corresponding fusion weights, and the decision module is configured to generate the authentication result based on the decision results of all expert modules of the defense model.

[0183] Optionally, as an embodiment, the biometric authentication device 300 may further include:

[0184] An output module, configured to output the authentication result based on a first security mechanism; wherein, the first security mechanism includes at least one of the following: a multi-signature mechanism and a context binding mechanism.

[0185] The biometric authentication device in the embodiments of the present application may be an electronic device or a component in an electronic device, such as an integrated circuit or a chip. The electronic device may be a terminal or a device other than a terminal. Exemplarily, the electronic device may be a mobile phone, a tablet computer, a notebook computer, a handheld computer, a vehicle-mounted electronic device, a Mobile Internet Device (MID), an Augmented Reality (AR) / Virtual Reality (VR) device, a robot, a wearable device, an Ultra-Mobile Personal Computer (UMPC), a netbook, or a Personal Digital Assistant (PDA), etc., and may also be a server, a Network Attached Storage (NAS), a Personal Computer (PC), a Television (TV), a teller machine, or a self-service machine, etc. The embodiments of the present application do not make specific limitations.

[0186] The biometric authentication device in the embodiments of the present application can be a device with an operating system. The operating system can be an Android operating system, an iOS operating system, or other possible operating systems, which are not specifically limited in the embodiments of the present application.

[0187] The biometric authentication device provided by the embodiments of the present application can implement each process implemented by the above method embodiments. To avoid repetition, it will not be elaborated here.

[0188] Optionally, as Figure 4 shown, the embodiments of the present application further provide an electronic device 400, including a processor 401 and a memory 402. A program or instruction that can run on the processor 401 is stored on the memory 402. When the program or instruction is executed by the processor 401, it implements each step of the above biometric authentication method embodiments and can achieve the same technical effects. To avoid repetition, it will not be elaborated here.

[0189] It should be noted that the electronic devices in the embodiments of the present application include the above-mentioned mobile electronic devices and non-mobile electronic devices.

[0190] Figure 5 is a schematic hardware structure diagram of an electronic device for implementing various embodiments of the present application.

[0191] The electronic device 500 includes but is not limited to: a radio frequency unit 501, a network module 502, an audio output unit 503, an input unit 504, a sensor 505, a display unit 506, a user input unit 507, an interface unit 508, a memory 509, and a processor 510, etc.

[0192] Those skilled in the art can understand that the electronic device 500 may further include a power supply (such as a battery) for supplying power to each component. The power supply can be logically connected to the processor 510 through a power management system, so as to implement functions such as management of charging, discharging, and power consumption management through the power management system. Figure 5 The structure of the electronic device shown in [reference] does not constitute a limitation on the electronic device. The electronic device may include more or fewer components than shown, or combine certain components, or have different component arrangements, which will not be elaborated here.

[0193] Among them, the processor 510 is used to obtain multimodal biometric features and environmental features. Among them, the multimodal biometric features are the combination of biometric feature information of two or more different modalities, and each modality represents an independent biometric feature type. Based on the environmental features, determine the fusion weights of the biometric features of each modality in the multimodal biometric features. Based on the fusion weights of the biometric features of each modality in the multimodal biometric features, authenticate the multimodal biometric features to obtain an authentication result.

[0194] It can be seen that in the embodiments of the present application, since the fusion weights of the biometric features of each modality are determined based on the environmental features and can be adaptively adjusted as the authentication environment changes, therefore, based on the fusion weights of the biometric features of each modality in the multimodal biometric features, authenticating the multimodal biometric features can improve the adaptability of biometric authentication to the environment, thereby improving the accuracy of biometric authentication in different environments and reducing the false alarm rate of authentication.

[0195] Optionally, as an embodiment, the processor 510 is specifically configured to obtain behavioral features, and determine the fusion weights of the biometric features of each modality in the multimodal biometric features according to the environmental features and the behavioral features.

[0196] Optionally, as an embodiment, the processor 510 is specifically configured to perform spatio-temporal consistency verification on at least one of the behavioral features and the environmental features. When the spatio-temporal consistency verification passes, determine the fusion weights of the biometric features of each modality in the multimodal biometric features according to the environmental features and the behavioral features.

[0197] Optionally, as an embodiment, the processor 510 is specifically configured to perform heterogeneous feature alignment on the behavioral features and the multimodal biometric features, enhance each aligned feature through a self-attention network to obtain enhanced features, perform cross-attention calculation on every two enhanced features through a cross-attention network to obtain the attention weights of the biometric features of each modality in the multimodal biometric features, process the environmental features through an environment adaptation network to obtain environmental weights, and correct the attention weights of the biometric features of each modality in the multimodal biometric features according to the environmental weights to obtain the fusion weights of the biometric features of each modality in the multimodal biometric features.

[0198] Optionally, as an embodiment, the processor 510 is specifically configured to input the biometric features of each modality in the multimodal biometric features and the corresponding fusion weights into a defense model for processing, and output an authentication result; wherein, the defense model includes: at least two different biometric expert modules and a decision module, and the different biometric expert modules are used to make decisions on different combinations of biometric features based on different combinations of biometric features and the corresponding fusion weights, and the decision module is used to generate the authentication result based on the decision results of all expert modules of the defense model.

[0199] Optionally, as an embodiment, the processor 510 is further configured to output the authentication result based on a first security mechanism; wherein, the first security mechanism includes at least one of the following: a multi-signature mechanism and a context binding mechanism.

[0200] It should be understood that in the embodiments of the present application, the input unit 504 may include a Graphics Processing Unit (GPU) 5041 and a microphone 5042, and the graphics processor 5041 processes the image data of static pictures or videos obtained by an image capturing device (such as a camera) in a video capture mode or an image capture mode. The display unit 506 may include a display panel 5061, and the display panel 5061 may be configured in the form of a liquid crystal display, an organic light emitting diode, etc. The user input unit 507 includes at least one of a touch panel 5071 and an input device 5072. The touch panel 5071 is also called a touch screen. The touch panel 5071 may include two parts: a touch detection device and a touch controller. The input device 5072 may include, but is not limited to, a physical keyboard, function keys (such as volume control keys, switch keys, etc.), a trackball, a mouse, a joystick, which will not be elaborated here.

[0201] The memory 509 can be used to store software programs and various data. The memory 509 may mainly include a first storage area for storing programs or instructions and a second storage area for storing data. Among them, the first storage area may store an operating system, application programs or instructions required for at least one function (such as a sound playback function, an image playback function, etc.). In addition, the memory 509 may include a volatile memory or a non-volatile memory, or the memory 509 may include both a volatile memory and a non-volatile memory. Among them, the non-volatile memory may be a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), or a flash memory. The volatile memory may be a random access memory (RAM), a static random access memory (SRAM), a dynamic random access memory (DRAM), a synchronous dynamic random access memory (SDRAM), a double data rate synchronous dynamic random access memory (DDR SDRAM), an enhanced synchronous dynamic random access memory (ESDRAM), a synchlink dynamic random access memory (SLDRAM), and a direct rambus random access memory (DRRAM). The memory 509 in the embodiments of the present application includes but is not limited to these and any other suitable types of memories.

[0202] The processor 510 may include one or more processing units; optionally, the processor 510 integrates an application processor and a modem processor. Among them, the application processor mainly processes operations related to the operating system, user interface, and application programs, etc., and the modem processor mainly processes wireless communication signals, such as a baseband processor. It can be understood that the above modem processor may not be integrated into the processor 510 either.

[0203] The embodiments of the present application also provide a readable storage medium, on which a program or instruction is stored. When the program or instruction is executed by a processor, it implements each process of the above-mentioned embodiments of the biometric authentication method and can achieve the same technical effect. To avoid repetition, it will not be elaborated here.

[0204] Among them, the processor is the processor in the electronic device described in the above embodiments. The readable storage medium includes computer-readable storage media such as computer read-only memory ROM, random access memory RAM, magnetic disks, or optical discs, etc.

[0205] An embodiment of the present application further provides a chip, which includes a processor and a communication interface. The communication interface is coupled to the processor. The processor is used to run programs or instructions to implement each process of the above embodiment of the biometric authentication method and can achieve the same technical effects. To avoid repetition, it will not be elaborated here.

[0206] It should be understood that the chip mentioned in the embodiments of the present application may also be referred to as a system-on-chip, system chip, chip system, or system-on-chip, etc.

[0207] An embodiment of the present application further provides a computer program product. The program product is stored in a storage medium and is executed by at least one processor to implement each process of the above embodiment of the biometric authentication method and can achieve the same technical effects. To avoid repetition, it will not be elaborated here.

[0208] It should be noted that in this article, the term "comprising", "including" or any of its variants is intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements not only includes those elements, but also includes elements not explicitly listed, or also includes elements inherent to such process, method, article or device. Without further limitations, an element defined by the statement "including one..." does not exclude the existence of other identical elements in the process, method, article or device including the element. In addition, it should be pointed out that the scope of the methods and devices in the embodiments of the present application is not limited to performing functions in the order shown or discussed, and may also include performing functions in a substantially simultaneous manner or in a reverse order according to the functions involved. For example, the described methods may be performed in an order different from that described, and various steps may be added, omitted, or combined. In addition, the features described with reference to certain examples may be combined in the examples.

[0209] Through the description of the above embodiments, those skilled in the art can clearly understand that the above-described embodiment methods can be implemented by means of software plus a necessary general hardware platform. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method. Based on such an understanding, the technical solution of the present application, in essence or the part that contributes to the prior art, can be embodied in the form of a computer software product. The computer software product is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk), and includes several instructions for causing a terminal (such as a mobile phone, computer, server, or network device, etc.) to execute the methods described in various embodiments of the present application.

[0210] The embodiments of the present application have been described above in conjunction with the accompanying drawings. However, the present application is not limited to the above specific embodiments. The above specific embodiments are merely illustrative and not restrictive. Under the inspiration of the present application, those of ordinary skill in the art can also make many forms without departing from the purpose of the present application and the scope protected by the claims, and all of them belong to the protection scope of the present application.

Claims

1. A biometric authentication method, applied to an electronic device, characterized in that, The method includes: Obtaining multimodal biometric features and environmental features; wherein, the multimodal biometric features are the combination of biometric feature information of two or more different modalities, and each modality represents an independent biometric feature type; Based on the environmental features, determining the fusion weights of the biometric features of each modality in the multimodal biometric features; Based on the fusion weights of the biometric features of each modality in the multimodal biometric features, authenticating the multimodal biometric features to obtain an authentication result.

2. The method according to claim 1, wherein Before the step of determining the fusion weights of the biometric features of each modality in the multimodal biometric features based on the environmental features, the method further includes: Obtaining behavioral features; The step of determining the fusion weights of the biometric features of each modality in the multimodal biometric features based on the environmental features includes: Determining the fusion weights of the biometric features of each modality in the multimodal biometric features according to the environmental features and the behavioral features.

3. The method according to claim 2, wherein The step of determining the fusion weights of the biometric features of each modality in the multimodal biometric features according to the environmental features and the behavioral features includes: Performing spatio-temporal consistency verification on at least one of the behavioral features and the environmental features; In the case where the spatio-temporal consistency verification passes, determining the fusion weights of the biometric features of each modality in the multimodal biometric features according to the environmental features and the behavioral features.

4. The method according to claim 2, wherein The step of determining the fusion weights of the biometric features of each modality in the multimodal biometric features according to the environmental features and the behavioral features includes: Performing heterogeneous feature alignment on the behavioral features and the multimodal biometric features; Through a self-attention network, enhancing each aligned feature to obtain enhanced features; Through a cross-attention network, performing cross-attention calculation on every two enhanced features to obtain the attention weights of the biometric features of each modality in the multimodal biometric features; Through an environment adaptation network, processing the environmental features to obtain environmental weights; According to the environmental weights, correcting the attention weights of the biometric features of each modality in the multimodal biometric features to obtain the fusion weights of the biometric features of each modality in the multimodal biometric features.

5. The method according to any one of claims 1-4, characterized in that, The step of authenticating the multimodal biometric features based on the fusion weights of the biometric features of each modality in the multimodal biometric features to obtain an authentication result includes: Inputting the biometric features of each modality in the multimodal biometric features and the corresponding fusion weights into a defense model for processing, and outputting an authentication result; Wherein, the defense model includes: at least two different biometric expert modules and a decision module. The different biometric expert modules are used to make decisions on different combinations of biometric features based on different combinations of biometric features and the corresponding fusion weights, and the decision module is used to generate the authentication result based on the decision results of all expert modules of the defense model.

6. The method according to claim 1, characterized in that After the step of authenticating the multimodal biometric features based on the fusion weights of the biometric features of each modality in the multimodal biometric features to obtain an authentication result, the method further includes: Output the authentication result based on a first security mechanism, where the first security mechanism includes at least one of the following: a multi-signature mechanism and a context binding mechanism.

7. A biometric authentication device, applied to an electronic device, characterized in that, The device includes: A first acquisition module, configured to acquire multimodal biometric features and environmental features, where the multimodal biometric features are a combination of biometric feature information of two or more different modalities, and each modality represents an independent biometric feature type; A determination module, configured to determine the fusion weight of the biometric feature of each modality in the multimodal biometric features based on the environmental features; An authentication module, configured to authenticate the multimodal biometric features based on the fusion weight of the biometric feature of each modality in the multimodal biometric features to obtain an authentication result.

8. The device according to claim 7, characterized in that, The device further includes: A second acquisition module, configured to acquire behavioral features; The determination module includes: A determination sub-module, configured to determine the fusion weight of the biometric feature of each modality in the multimodal biometric features according to the environmental features and the behavioral features.

9. The device according to claim 8, characterized in that, The determination sub-module includes: A verification unit, configured to perform spatio-temporal consistency verification on at least one of the behavioral features and the environmental features; A determination unit, configured to determine the fusion weight of the biometric feature of each modality in the multimodal biometric features according to the environmental features and the behavioral features when the spatio-temporal consistency verification passes.

10. The device according to claim 8, characterized in that, The determination sub-module includes: An alignment processing unit, configured to perform heterogeneous feature alignment processing on the behavioral features and the multimodal biometric features; An enhancement processing unit, configured to enhance each aligned feature through a self-attention network to obtain enhanced features; A cross-processing unit, configured to perform cross-attention calculation on every two enhanced features through a cross-attention network to obtain the attention weight of the biometric feature of each modality in the multimodal biometric features; An environment adaptation processing unit, configured to process the environmental features through an environment adaptation network to obtain an environment weight; A correction unit, configured to correct the attention weight of the biometric feature of each modality in the multimodal biometric features according to the environment weight to obtain the fusion weight of the biometric feature of each modality in the multimodal biometric features.

11. The device according to any one of claims 7 to 10, characterized in that, The authentication module includes: A processing sub-module, configured to input the biometric feature of each modality in the multimodal biometric features and the corresponding fusion weight into a defense model for processing and output an authentication result; where the defense model includes: at least two different biometric expert modules and a decision-making module. The different biometric expert modules are used to make decisions on different combinations of biometric features based on different combinations of biometric features and the corresponding fusion weights, and the decision-making module is used to generate the authentication result based on the decision results of all expert modules of the defense model.

12. The device according to claim 7, characterized in that, The device further includes: An output module, configured to output the authentication result based on a first security mechanism, where the first security mechanism includes at least one of the following: a multi-signature mechanism and a context binding mechanism.

Images