A dynamic portrait identity verification method and device, electronic equipment and storage medium

Abstract

This application provides a dynamic facial recognition method, apparatus, electronic device, and storage medium, belonging to the fields of image recognition and biometric recognition, and applicable to financial and medical applications. The method includes: responding to the client's identity verification result; if the identity verification result indicates the user is a live entity, receiving multimodal data and historical identity information sent by the client; integrating the multimodal data and historical identity information to obtain a dynamic context vector; inputting the dynamic context vector into a pre-trained risk level classification model to obtain a risk score output by the risk level classification model; determining the risk level based on a comparison between the risk score and a preset risk threshold; and executing a corresponding identity verification strategy based on the risk level to obtain the user's identity verification result. This application implements risk level-adaptive differentiated identity verification, reducing computational overhead.

Description

Technical Field

[0001] This application relates to the fields of image recognition and biometric recognition, and in particular to a dynamic human image identity verification method, device, electronic device and storage medium. Background Technology

[0002] Currently, with the deep penetration of internet technology into the financial and insurance sectors, remote identity verification has become a crucial link in core business scenarios such as insurance signing, loan approval, and claims application. The rapid development of large-scale language models and computer vision technology has provided intelligent technical support for facial recognition, liveness detection, and anti-fraud analysis, significantly improving business processing efficiency and verification accuracy, and meeting the industry's demand for efficient and automated risk control services.

[0003] Current identity verification methods primarily rely on a combination of static document information extraction and single-frame facial feature comparison. However, their defenses are passive and lagging, making them vulnerable to attacks such as photo manipulation, screen playback, 3D masks, and deepfake videos. Static liveness detection suffers from a high rate of bypassing fraud in online channels, and its adaptability to real-world scenarios with complex lighting, large-angle poses, and cross-age groups is poor. Furthermore, its accuracy drops significantly in uncontrolled environments. Therefore, how to achieve dynamic facial recognition without additional hardware investment, while improving security, efficiency, and generalization, has become a pressing technical problem in this field. Summary of the Invention

[0004] The main objective of this application is to propose a dynamic facial recognition method, device, electronic device, and storage medium, aiming to solve the problems of low accuracy and inefficiency in existing facial recognition technology due to the vulnerability of facial recognition to non-liveness attacks such as photos and video playback.

[0005] To achieve the above objectives, a first aspect of this application proposes a dynamic facial recognition method, the method comprising: In response to the client's identity verification result, which is obtained by performing liveness detection on the user through a lightweight liveness detection model pre-deployed on the client; If the identity verification result indicates that the user is a live entity, the system receives multimodal data and historical identity information sent by the client. The multimodal data includes keyframe video streams collected during the liveness detection process of the user, and the historical identity information is used to describe the user's historical verification information. The multimodal data and historical identity information are integrated to obtain a dynamic context vector; The dynamic context vector is input into a pre-trained risk level classification model to obtain the risk score output by the risk level classification model. The risk level is determined based on the comparison between the risk score and the preset risk threshold. The corresponding identity verification strategy is executed according to the risk level to obtain the user's identity verification result.

[0006] To achieve the above objectives, a second aspect of this application provides a dynamic facial recognition device, the device comprising: The response module is used to respond to the identity verification result of the client, which is obtained by performing liveness detection on the user through a lightweight liveness detection model pre-deployed on the client. The acquisition module is used to receive multimodal data and historical identity information sent by the client when the identity verification result indicates that the user is a live entity. The multimodal data includes keyframe video streams collected during the liveness detection process of the user, and the historical identity information is used to describe the user's historical verification information. The extraction module is used to integrate the multimodal data and historical identity information to obtain a dynamic context vector; The classification module is used to input the dynamic context vector into a pre-trained risk level classification model to obtain the risk score output by the risk level classification model. The comparison module is used to determine the risk level based on the comparison result between the risk score and the preset risk threshold; The execution module is used to execute the corresponding identity verification strategy according to the risk level and obtain the user's identity verification result.

[0007] To achieve the above objectives, a third aspect of this application provides an electronic device, which includes a memory and a processor. The memory stores a computer program, and the processor executes the computer program to implement the method described in the first aspect.

[0008] To achieve the above objectives, a fourth aspect of the present application provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the method described in the first aspect.

[0009] The dynamic facial recognition method, device, electronic device, and storage medium proposed in this application respond to the client's facial recognition result; when the facial recognition result indicates that the user is a live person, it receives multimodal data and historical identity information sent by the client; it integrates the multimodal data and historical identity information to obtain a dynamic context vector; it inputs the dynamic context vector into a pre-trained risk level classification model to obtain a risk score output by the risk level classification model; it determines the risk level based on the comparison result between the risk score and a preset risk threshold; and it executes the corresponding facial recognition strategy according to the risk level to obtain the user's facial recognition result. This achieves dynamic facial recognition without additional hardware investment, thereby solving the technical problems of existing static document information extraction and single-frame facial feature comparison schemes, such as passive and lagging defenses, susceptibility to attacks by photo re-photographing, screen playback, 3D masks, and deepfake videos, and poor adaptability to real-world scenarios such as complex lighting, large-angle poses, and cross-age groups. This improves the security, efficiency, and generalization of facial recognition. Attached Figure Description

[0010] Figure 1 This is a schematic diagram of an application environment for the dynamic facial recognition method provided in this application embodiment; Figure 2 This is a flowchart of the dynamic facial recognition method provided in the embodiments of this application; Figure 3 yes Figure 2 The flowchart of step S102 in the document; Figure 4 yes Figure 2 The flowchart of step S103 in the process; Figure 5 yes Figure 2 A flowchart of step S106 in the process; Figure 6 yes Figure 5 The flowchart of step S401 in the process; Figure 7 yes Figure 5 The flowchart of step S402 in the document; Figure 8 yes Figure 2 Another flowchart of step S106 in the process; Figure 9 This is a schematic diagram of the structure of the dynamic facial recognition device provided in the embodiments of this application; Figure 10 This is a schematic diagram of the hardware structure of the electronic device provided in the embodiments of this application. Detailed Implementation

[0011] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0012] It should be noted that although functional modules are divided in the device schematic diagram and a logical order is shown in the flowchart, in some cases, the steps shown or described may be performed in a different order than the module division in the device or the order in the flowchart. The terms "first," "second," etc., in the specification, claims, and the aforementioned drawings are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence.

[0013] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application belongs. The terminology used herein is for the purpose of describing embodiments of this application only and is not intended to limit this application.

[0014] In the current insurance industry, identity verification and fraud prevention are core risk control aspects. Traditional methods mainly rely on static document recognition (such as OCR reading ID card information) and simple facial comparison technology, typically using single-frame image capture for verification. These solutions have several drawbacks: First, they are vulnerable to non-liveness attacks such as photo attacks and video playback attacks, resulting in low security. Second, traditional models are poorly adaptable to real-world scenarios such as changes in lighting, pose shifts, and large age ranges, leading to decreased recognition accuracy. Third, existing systems lack context awareness and cannot comprehensively judge based on multi-dimensional data such as user behavior sequences, device fingerprints, and geographical location, making it difficult to identify fraudulent activities. Furthermore, most systems use fixed model structures and lack online learning and dynamic update mechanisms, making it difficult to adapt to the evolution of new fraud methods. Some advanced systems introduce 3D liveness detection or infrared imaging technology, but the high hardware costs limit their widespread adoption.

[0015] Based on this, embodiments of this application provide a dynamic facial recognition method, apparatus, electronic device, and storage medium, aiming to solve the problems of low accuracy and inefficiency in existing facial recognition technology due to susceptibility to interference.

[0016] The dynamic facial recognition method, device, electronic device, and storage medium provided in this application are specifically described through the following embodiments. First, the dynamic facial recognition method in this application is described.

[0017] The dynamic facial recognition method provided in this application can be applied to, for example, Figure 1In the application environment shown, the client (such as a smartphone, tablet, or other mobile terminal) communicates with the server (such as an insurance business system server) via a network. The client responds to the user's identity verification result by performing liveness detection based on multimodal data (including video streams, device motion data, and environmental audio clips) and sending the multimodal data and historical identity information to the server based on the detection results. The server, as the core executor of the verification decision, receives the data uploaded by the client and executes the corresponding identity verification strategy according to the risk level. It has a pre-trained risk level classification model and verification models such as a first-precision model and a second-precision model deployed within it. Simultaneously, a Model Context Protocol (MCP) context engine is deployed on the server to integrate multimodal data and historical identity information to generate dynamic context vectors, thereby controlling the inference path selection and attention weight allocation of subsequent AI models. The client can be, but is not limited to, various personal computers, laptops, smartphones, tablets, and portable wearable devices; the server can be implemented using a standalone server or a server cluster consisting of multiple servers. The following detailed description of specific embodiments further illustrates this application.

[0018] The embodiments of this application can acquire and process relevant data based on artificial intelligence technology. Artificial intelligence (AI) refers to the theories, methods, technologies, and application systems that use digital computers or machines controlled by digital computers to simulate, extend, and expand human intelligence, perceive the environment, acquire knowledge, and use that knowledge to obtain optimal results.

[0019] Foundational technologies for artificial intelligence generally include sensors, dedicated AI chips, cloud computing, distributed storage, big data processing, operating / interactive systems, and mechatronics. AI software technologies mainly encompass computer vision, robotics, biometrics, speech processing, natural language processing, and machine learning / deep learning.

[0020] The dynamic facial recognition method provided in this application relates to the fields of image recognition and biometric recognition technology, and can be applied in financial, medical, and other application scenarios. The dynamic facial recognition method provided in this application can be applied to a terminal, a server, or software running on either a terminal or a server. In some embodiments, the terminal can be a smartphone, tablet, laptop, desktop computer, etc.; the server can be configured as an independent physical server, a server cluster or distributed system composed of multiple physical servers, or a cloud server providing basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, CDN, and big data and artificial intelligence platforms; the software can be an application implementing the dynamic facial recognition method, but is not limited to the above forms.

[0021] This application can be used in a wide variety of general-purpose or special-purpose computer system environments or configurations. Examples include: personal computers, server computers, handheld or portable devices, tablet devices, multiprocessor systems, microprocessor-based systems, set-top boxes, programmable consumer electronics, network PCs, minicomputers, mainframe computers, and distributed computing environments including any of the above systems or devices. This application can be described in the general context of computer-executable instructions executed by a computer, such as program modules. Generally, program modules include routines, programs, objects, components, data structures, etc., that perform specific tasks or implement specific abstract data types. This application can also be practiced in distributed computing environments where tasks are performed by remote processing devices connected via a communication network. In distributed computing environments, program modules can reside in local and remote computer storage media, including storage devices.

[0022] It should be noted that in all specific embodiments of this application, when processing data related to user identity or characteristics, such as user information, user behavior data, user historical data, and user location information, user permission or consent is obtained first. Furthermore, the collection, use, and processing of this data comply with relevant laws, regulations, and standards. In addition, when embodiments of this application require access to sensitive personal information of users, separate permission or consent from the user is obtained through pop-ups or redirection to confirmation pages. Only after obtaining the user's separate permission or consent is the necessary user-related data required for the proper functioning of these embodiments acquired.

[0023] Figure 2 This is an optional flowchart of the dynamic facial recognition method provided in the embodiments of this application. Figure 2The method may include, but is not limited to, steps S101 to S106.

[0024] Step S101: Respond to the client's identity verification result, which is obtained by performing liveness detection on the user through a lightweight liveness detection model pre-deployed on the client.

[0025] In this context, the client refers to the terminal device used by users to initiate identity verification when handling financial transactions such as insurance signing, loan approval, and claims applications, or medical transactions such as registration for medical appointments, medical insurance settlement, electronic health card applications, and remote consultations. The client can be an application owned by the insurance company or hospital itself, or a web application or mini-program embedded in a third-party platform. For example, when a user opens an insurance company's mobile client on their smartphone and clicks the identity verification button on the insurance signing page, the client immediately activates a lightweight liveness detection model to perform a liveness check on the user. Similarly, when a patient selects medical insurance registration at a hospital's self-service registration terminal, the terminal immediately activates a lightweight liveness detection model to perform a liveness check on the patient.

[0026] The identity verification result refers to the detection result obtained by the client after performing a liveness detection on the user using a pre-deployed lightweight liveness detection model. The identity verification result indicates whether the user is alive and is a preliminary judgment sent by the client to the server after completing local liveness detection. For example, the identity verification result can be a Boolean value, with a true value indicating the user is alive and a false value indicating the user is not alive; or it can be a structured data object containing two fields: a liveness confidence score and a judgment label.

[0027] Lightweight liveness detection models refer to machine learning models deployed on the client side for real-time liveness detection of users. These models employ a network structure with a small number of parameters, enabling real-time operation with limited client computing resources. They are used for initial filtering of common attack methods such as photo manipulation and screen playback. For example, a lightweight liveness detection model could be based on MobileNetV3 combined with a time-shifting module, with fewer than 2 megabytes of parameters, capable of real-time inference at 25 frames per second on a smartphone.

[0028] Liveness detection refers to the technology used to determine whether a captured object is a real, living person. Liveness detection distinguishes real faces from non-liveness attacks such as photographs, screen playback, and 3D masks by analyzing the dynamic features of faces in a video stream. For example, liveness detection can determine whether a user is alive by analyzing the micro-movements, texture features, and temporal consistency of faces in consecutive video frames.

[0029] In this step, the server responds to the identity verification result sent by the client and proceeds to the next processing step. When the identity verification result indicates that the user is alive, the server continues to execute step S102; when the identity verification result indicates that the user is not alive, the server directly terminates the verification process and returns a verification failure result. This design, which places liveness detection at the client level, allows obvious non-liveness attacks to be quickly intercepted on the client side without uploading them to the server and consuming bandwidth and computing resources, thereby improving the overall processing efficiency of the system.

[0030] Step S102: If the identity verification result indicates that the user is alive, receive multimodal data and historical identity information sent by the client. The multimodal data includes key frame video streams collected during the user's liveness detection process, and the historical identity information is used to describe the user's historical verification information.

[0031] Multimodal data refers to fused data containing multiple modalities of information collected during identity verification. Multimodal data includes at least keyframe video streams and may also include device motion data. The fusion of multiple modal data can depict the user's true state from multiple dimensions such as vision and motion perception, thus providing richer evidence for subsequent information integration and risk assessment. For example, multimodal data may include keyframe video streams captured by a front-facing camera and device motion data collected by accelerometers and gyroscopes.

[0032] In liveness detection, a keyframe video stream refers to a sequence of key video frames containing the user's image extracted from the original detection video stream. Compared to the original detection video stream, the keyframe video stream has a smaller data volume, reducing data transmission between the client and server while retaining crucial information for subsequent facial comparison and risk assessment. For example, a keyframe video stream could be a sequence of 25 keyframes extracted from a 5-second original detection video stream at a sampling rate of 5 frames per second, with each frame having a resolution of 640 x 480.

[0033] Historical identity information refers to data used to describe a user's historical verification information. This information includes the spatiotemporal context of client-user interactions, the user's historical verification records, and the user's risk tags, used to characterize the user's risk status from non-biometric dimensions such as historical behavior and environmental context. For example, historical identity information may include the timestamp of the current verification, the client's internet protocol address, device identifier, the user's past verification records, and matching results from third-party risk databases.

[0034] In this step, after confirming that the client's liveness detection has passed, the server receives the multimodal data and historical identity information sent by the client. When sending the above data, the client can encapsulate the multimodal data and historical identity information into a verification data upload message and send it to the server via the Hypertext Transfer Protocol (HTTP). This design, which completes the liveness detection on the client side first and then uploads the multimodal data and historical identity information to the server, avoids unnecessary data transmission in non-liveness attack scenarios and provides rich, multi-dimensional data support for the server's risk assessment.

[0035] Step S103: Integrate multimodal data and historical identity information to obtain dynamic context vector.

[0036] Information integration refers to the process of unifying the representation of various modal features and non-biological features contained in multimodal data and historical identity information. Information integration involves extracting features from data from different sources and modalities, then concatenating or fusing the extracted features to obtain a unified feature vector. For example, information integration may include extracting video features from keyframe video streams, extracting motion features from device motion data, extracting spatiotemporal features from spatiotemporal context, extracting behavioral features from historical verification records, and extracting risk label features from risk labels, then concatenating these five features into a high-dimensional feature vector.

[0037] The dynamic context vector (VCV) is a feature vector representing the current verification session context state of a user, obtained by integrating multimodal data and historical identity information. The VVC combines the user's biometric and non-biometric information, comprehensively depicting the user's risk status in the current verification session, and serves as input for subsequent risk level classification models. For example, the VVC can be a 1024-dimensional floating-point vector, where the first 512 dimensions come from video features, and the last 512 dimensions are a concatenation of motion features, spatiotemporal features, behavioral features, and risk label features.

[0038] In this step, the server integrates the received multimodal data and historical identity information to obtain a dynamic context vector. This dynamic context vector, obtained through information integration, unifies information scattered across multiple modalities and data sources into a feature representation that the model can process, providing comprehensive input for subsequent risk level classification.

[0039] Step S104: Input the dynamic context vector into the pre-trained risk level classification model to obtain the risk score output by the risk level classification model.

[0040] In this context, a risk level classification model refers to a pre-trained machine learning model used to score a user's risk level based on a dynamic context vector. By learning the mapping relationship between dynamic context vectors and risk levels from a large number of labeled samples, the risk level classification model can output a risk score for a new input dynamic context vector. For example, a risk level classification model can be a classification model based on a multilayer perceptron or gradient boosting decision tree, with the dynamic context vector as input and a risk score between 0 and 1 as output.

[0041] In this context, the risk score is a numerical value output by the risk level classification model, used to represent the degree of risk a user faces. A higher risk score indicates a greater likelihood of fraud. For example, the risk score can be a floating-point number between 0 and 1, where 0 represents no risk and 1 represents the highest risk; alternatively, it can be an integer between 1 and 100.

[0042] In this step, the server inputs the dynamic context vector obtained in step S103 into the pre-trained risk level classification model. The model performs a comprehensive analysis based on the video features, motion features, spatiotemporal features, behavioral features, and risk label features contained in the dynamic context vector, and outputs a risk score. During the training phase, the risk level classification model learns the differences in the dynamic context vectors of normal users and fraudulent users in historical verification data, enabling it to quantify the risk of new users' dynamic context vectors during the inference phase.

[0043] Step S105: Determine the risk level based on the comparison between the risk score and the preset risk threshold.

[0044] The preset risk threshold refers to a pre-defined numerical boundary used to classify risk levels. The preset risk threshold can be configured according to the security level requirements of the business scenario; the higher the security level requirement, the lower the preset risk threshold can be set. For example, the preset risk threshold could be 0.5. When the risk score is less than 0.5, the risk level is determined to be the first risk level; when the risk score is greater than or equal to 0.5, the risk level is determined to be the second risk level.

[0045] Risk level refers to the classification based on the comparison between the risk score and a preset risk threshold. Risk level guides the selection of subsequent identity verification strategies, with different risk levels corresponding to different levels of verification strategy intensity. For example, risk levels may include a first risk level and a second risk level, with the first risk level corresponding to a lower risk level and the second risk level corresponding to a higher risk level.

[0046] In this step, the server compares the risk score obtained in step S104 with a preset risk threshold and determines the risk level based on the comparison result. When the risk score is less than the preset risk threshold, the risk level is determined to be the first risk level, indicating that the user is in a low-risk scenario; when the risk score is greater than or equal to the preset risk threshold, the risk level is determined to be the second risk level, indicating that the user is in a high-risk scenario. Through the classification of risk levels, the system can dynamically select verification strategies according to the user's risk status, thereby achieving a reasonable allocation of detection resources.

[0047] Step S106: Execute the corresponding identity verification strategy according to the risk level to obtain the user's identity verification result.

[0048] In this context, identity verification strategy refers to a scheme implemented based on risk level to further verify user identity. Different risk levels correspond to different identity verification strategies; the higher the risk level, the higher the detection accuracy and the richer the detection dimensions of the corresponding identity verification strategy. For example, the first risk level may correspond to a first verification strategy based on keyframe facial recognition, while the second risk level may correspond to a second verification strategy based on action commands and multimodal fusion analysis.

[0049] The identity verification result refers to the final verification conclusion obtained after executing the identity verification strategy. The identity verification result can include two states: verification passed and verification failed. It can also carry additional information such as verification confidence level and risk warnings. For example, when the matching degree between the multimodal fusion features of the second verification strategy and the baseline features is higher than a preset matching threshold, the identity verification result is verification passed; otherwise, the identity verification result is verification failed.

[0050] In this step, the server calls the corresponding identity verification strategy to verify the multimodal data based on the risk level determined in step S105, and obtains the user's identity verification result.

[0051] The technical solution provided in this application has at least the following beneficial effects: First, by deploying a lightweight liveness detection model on the client side to complete liveness detection, compared with the existing technology's static document information extraction and single-frame facial feature comparison scheme, liveness detection can distinguish real faces from attack methods such as photo re-enactments, screen playback, 3D masks, and deepfake videos based on the dynamic features of continuous video frames, thereby solving the technical problem of high bypass rates for online channel fraud in the background technology. At the same time, by placing liveness detection on the client side, obvious non-liveness attacks can be quickly intercepted on the client side without uploading to the server side and consuming bandwidth and computing resources, thus improving the overall processing efficiency of the system. Second, by integrating multimodal data and historical identity information on the server side to obtain a dynamic context vector, and inputting the dynamic context vector into a risk level classification model to obtain a risk score, it can combine multi-dimensional information such as user video features, motion features, spatiotemporal context, historical behavior, and risk tags to make a comprehensive risk judgment. Compared with the existing technology's lack of context awareness capabilities, it can identify abnormal behavior patterns and gang fraud behavior, improving the security of identity verification. Third, by implementing corresponding identity verification strategies based on risk levels, dynamic allocation of detection resources can be achieved. In low-risk scenarios, lightweight verification strategies can be used to improve efficiency, while in high-risk scenarios, high-precision verification strategies can be used to ensure security. This achieves a balance between security, efficiency, and generalization without additional hardware investment, thus solving the technical problem in the background technology of how to achieve dynamic facial recognition without additional hardware investment.

[0052] To facilitate understanding of the dynamic facial recognition method described in this application, two application examples of this method in the financial and medical fields are provided. The first application example is an example of a user initiating insurance contract verification via a mobile device, and the second application example is an example of a patient initiating medical insurance registration verification via a hospital self-service terminal. Both application examples combine the context-aware mechanism of the model context protocol and the layered liveness detection architecture of edge-cloud collaboration to fully illustrate the technical solution of this application.

[0053] Application Example 1: Insurance application and signing scenario in the financial sector.

[0054] This application example illustrates the process of a user initiating an insurance contract verification via a mobile device. Details are as follows.

[0055] Step 1: The client performs the first layer of lightweight liveness detection.

[0056] In this step, the user opens the insurance company's mobile client on their smartphone, enters the insurance application page, and clicks the identity verification button. The client activates its camera to capture the user's detection video stream, extracts a continuous video frame sequence from the stream, and inputs this sequence into a lightweight liveness detection model pre-deployed on the client. This lightweight liveness detection model uses a MobileNetV3 network structure combined with a time-shifting module and runs on the client to initially filter attacks such as photo manipulation and screen playback. The model outputs a liveness confidence score. When the liveness confidence score is greater than a preset confidence threshold of 0.5, the client determines the identity verification result as live; when the liveness confidence score is less than the preset confidence threshold of 0.5, the client determines the identity verification result as not live and directly prompts the user that verification failed.

[0057] Step 2: The client sends multimodal data and historical identity information to the server.

[0058] In this step, after the client's liveness detection is successful, the client sends the identity verification result, multimodal data, and historical identity information to the server. The multimodal data includes keyframe video streams captured by the front-facing camera (640x480 resolution, 25 frames per second) and device motion data collected by the accelerometer and gyroscope (sampling rate 50 Hz). Historical identity information includes the spatiotemporal context of this verification (timestamp, internet protocol address, device identifier), the user's historical verification records (frequency of past insurance verifications, distribution of verification results), and risk tags (matching results from black and gray market databases). The client encapsulates the above data into a verification data upload message and sends it to the server.

[0059] Step 3: The server integrates information and classifies risk levels.

[0060] In this step, after receiving the data sent by the client, the server integrates multimodal data and historical identity information. Specifically, based on the context-aware engine of the model context protocol, the server extracts video features from keyframe video streams, motion features from device motion data, spatiotemporal features from spatiotemporal context, and behavioral features from historical verification records. It then queries the risk tag database to obtain risk weights and encodes these weights to obtain risk tag features. Finally, the video features, motion features, spatiotemporal features, behavioral features, and risk tag features are concatenated to obtain a dynamic context vector. Subsequently, the server inputs the dynamic context vector into a pre-trained risk level classification model, which outputs a risk score. The server compares the risk score with a preset risk threshold of 0.5. If the risk score is less than 0.5, the risk level is determined to be the first risk level; if the risk score is greater than or equal to 0.5, the risk level is determined to be the second risk level. For example, when a user uses a familiar device to sign up for insurance at a frequently used location and has normal historical verification records, the risk score is low, and the risk level is determined to be the first risk level; as another example, when a user uses a new device to sign up for insurance in a different location or when the recent verification frequency has increased abnormally, the risk score is high, and the risk level is determined to be the second risk level.

[0061] Step 4: In low-risk scenarios, use the fast path to complete the verification.

[0062] In this step, when the risk level is at the highest level, it indicates that the user is in a low-risk context, such as using familiar equipment at a frequently visited location to sign an insurance agreement and having a normal historical verification record. The server automatically reduces computational complexity and uses a fast path to complete the verification, i.e., it uses the first verification strategy to verify multimodal data. Specifically, the server extracts a sequence of 5 keyframes from the keyframe video stream and inputs the keyframe sequence into a pre-built first-precision model, which is a ResNet50 residual network model, outputting a first portrait feature vector. The server calculates the cosine similarity between the first portrait feature vector and the pre-stored baseline portrait feature vector in the insurance system. When the similarity is greater than a preset similarity threshold of 0.8, the first verification result is determined to be successful, and the client continues to complete the insurance signing process. In this low-risk scenario, the system significantly shortens the verification response time through the fast path, reduces user waiting time, and improves the insurance application experience.

[0063] Step 5: Trigger the deep detection process in high-risk scenarios.

[0064] In this step, when the risk level is Level 2, it indicates that the user is in a high-risk context, such as using a new device to log in from a different location to sign an insurance contract or a recent abnormal increase in verification frequency. The server triggers a deep detection process, employing a second verification strategy to verify the multimodal data. Specifically, the server sends action instructions to the client to instruct the user to perform predetermined actions, including blinking and head shaking. The client screen displays an action guidance animation, and the verification video stream during the user's execution of the predetermined actions is captured by the camera and returned to the server. The server extracts facial key point sequences and pulse signal features from the verification video stream. Micro-expression recognition is performed on the facial key point sequences to obtain micro-expression features, and 3D deformation modeling is performed on the facial key point sequences to obtain 3D deformation features. The micro-expression features, 3D deformation features, and pulse signal features are fused to obtain multimodal fusion features. The server generates a second verification result based on the multimodal fusion features. If the matching degree between the multimodal fusion features and the baseline fusion features is higher than a preset matching threshold, the second verification result is considered successful; otherwise, the second verification result is considered unsuccessful, the client refuses the insurance contract, and prompts the user to go to a manual verification window. In this high-risk scenario, the system effectively defends against advanced attack methods such as deepfake videos and 3D masks through a deep detection process.

[0065] Step 6: The server updates the user profile and model parameters in reverse.

[0066] In this step, the server updates the user's behavioral profile data based on the identity verification results. This behavioral profile data includes the user's verification history, behavioral habits, and risk tags. The updated behavioral profile data is used for subsequent identity verification requests. Simultaneously, the server generates feedback signals based on the identity verification results and risk scores. These feedback signals are then used to incrementally update the parameters of the risk level classification model, enabling the model to learn online and continuously adapt to the evolution of new attack methods.

[0067] Application Example 2: Medical insurance registration and treatment scenarios in the medical field.

[0068] This application example illustrates the use of a patient initiating identity verification for medical insurance registration through a hospital's self-service registration terminal. In medical insurance registration and treatment scenarios, the accuracy of identity verification directly impacts the security of the medical insurance fund and patient medication safety. Issues such as impersonation and theft of medical insurance cards have always been challenging for medical institutions. This application example utilizes the dynamic facial recognition identity verification method described in this application, enabling dynamic identity verification without the need for additional dedicated hardware such as infrared cameras or 3D depth cameras, using the visible light camera integrated into the hospital's self-service registration terminal. Details are as follows.

[0069] Step 1: The client performs the first layer of lightweight liveness detection.

[0070] In this step, after arriving at the hospital, the patient inserts their medical insurance card or scans their electronic medical insurance voucher at the self-service registration terminal in the hospital lobby, selects the department and appointment number, and then proceeds to the identity verification stage. The self-service registration terminal activates its top visible light camera to capture the patient's detection video stream, extracts a continuous video frame sequence from the detection video stream, and inputs the continuous video frame sequence into a lightweight liveness detection model pre-deployed on the terminal. The lightweight liveness detection model adopts a network structure combining MobileNetV3 with a time-series shift module, runs locally on the self-service registration terminal, and is used to initially filter attacks such as photo manipulation and screen playback. The model outputs a liveness confidence score. When the liveness confidence score is greater than a preset confidence threshold of 0.5, the terminal determines the identity verification result as live and continues to execute subsequent steps; when the liveness confidence score is less than the preset confidence threshold of 0.5, the terminal determines the identity verification result as not live, prompts the patient that the verification has failed, and guides the patient to the manual window for processing.

[0071] Step 2: The client sends multimodal data and historical identity information to the server.

[0072] In this step, after the terminal passes the liveness detection, the self-service registration terminal sends the identity verification result, multimodal data, and historical identity information to the hospital's information center server. The multimodal data includes keyframe video streams captured by the visible light camera on the top of the terminal (640 x 480 resolution, 20 frames per second); and device vibration data collected by the terminal's accelerometer (sampling rate 50 Hz) to detect whether the terminal has been physically tampered with. Historical identity information includes the spatiotemporal context of this registration (registration timestamp, terminal number, hospital area identifier, department identifier), the patient's historical medical records (past medical records, registration frequency, distribution of departments visited), and risk tags (matching results from the medical insurance blacklist database, markers of abnormal medical behavior). The self-service registration terminal then sends the above data to the hospital's information center server via the hospital's intranet.

[0073] Step 3: The server integrates information and classifies risk levels.

[0074] In this step, after receiving the data sent by the terminal, the server in the hospital information center integrates multimodal data and historical identity information. Specifically, based on the context-aware engine of the model context protocol, the server extracts video features from keyframe video streams, motion features from equipment vibration data, spatiotemporal features from spatiotemporal context, and behavioral features from historical medical records. It then queries the risk label database to obtain risk weights and encodes these weights to obtain risk label features. Finally, the video features, motion features, spatiotemporal features, behavioral features, and risk label features are concatenated to obtain a dynamic context vector. Subsequently, the server inputs the dynamic context vector into a pre-trained risk level classification model, which outputs a risk score. The server compares the risk score with a preset risk threshold of 0.5. If the risk score is less than 0.5, the risk level is determined to be the first risk level; if the risk score is greater than or equal to 0.5, the risk level is determined to be the second risk level. For example, when a patient uses their own medical insurance card to register at a frequently used department in a commonly used hospital and has a normal medical history, the risk score is low and the risk level is determined to be the first risk level; as another example, when a patient uses their medical insurance card to register at a hospital in another city and the frequency of registration has increased abnormally recently, the risk score is high and the risk level is determined to be the second risk level.

[0075] Step 4: In low-risk scenarios, use the fast path to complete the verification.

[0076] In this step, when the risk level is at the highest level, it indicates that the patient is in a low-risk context, such as registering at a frequently used hospital using their own medical insurance card and having a normal medical history. The server automatically reduces computational complexity and uses a fast path to complete the verification, that is, it uses the first verification strategy to verify the multimodal data. Specifically, the server extracts 5 keyframe sequences from the keyframe video stream and inputs the keyframe sequences into a pre-constructed first-precision model, which is a ResNet50 residual network model, and outputs the first portrait feature vector. The server calculates the cosine similarity between the first portrait feature vector and the baseline portrait feature vector pre-stored in the medical insurance system. When the similarity is greater than the preset similarity threshold of 0.8, the first verification result is determined to be successful, and the self-service registration terminal continues to complete the registration and print the registration slip. In this low-risk scenario, the system significantly shortens the verification response time through the fast path, reduces the waiting time for patients at the self-service terminal, and improves the medical experience.

[0077] Step 5: Trigger the deep detection process in high-risk scenarios.

[0078] In this step, a risk level of Level 2 indicates that the patient is in a high-risk context, such as using a medical insurance card to register at a hospital in another city or an abnormally high frequency of recent registrations. The server triggers a deep detection process, employing a second verification strategy to verify the multimodal data. Specifically, the server sends action instructions to the self-service registration terminal to instruct the patient to perform predetermined actions, including blinking and head shaking. The self-service registration terminal displays an action guidance animation on its screen, and captures a verification video stream of the patient performing the predetermined actions via a camera, returning it to the server. The server extracts facial key point sequences and pulse signal features from the verification video stream. It performs micro-expression recognition on the facial key point sequences to obtain micro-expression features, performs 3D deformation modeling on the facial key point sequences to obtain 3D deformation features, and fuses the micro-expression features, 3D deformation features, and pulse signal features to obtain multimodal fusion features. The server generates a second verification result based on the multimodal fusion features. If the matching degree between the multimodal fusion features and the baseline fusion features is higher than a preset matching threshold, the second verification result is considered successful; otherwise, it is considered unsuccessful, the self-service registration terminal refuses registration, and the patient is prompted to go to a manual window for verification. In this high-risk scenario, the system effectively defends against advanced attack methods such as deepfake videos, 3D masks, and high-definition photos through a deep detection process, preventing impersonation for medical treatment and the theft of medical insurance cards.

[0079] Step 6: The server updates the patient profile and model parameters in reverse.

[0080] In this step, the server updates the patient's behavioral profile data based on the identity verification results. This behavioral profile data includes the patient's medical history, registration habits, and risk tags. The updated behavioral profile data is used for subsequent identity verification requests. Simultaneously, the server generates feedback signals based on the identity verification results and risk scores. These feedback signals are then used to incrementally update the parameters of the risk level classification model, enabling the model to learn online and continuously adapt to the evolution of new attack methods. For example, if the server detects multiple recent attempted impersonation incidents at a particular hospital, the model automatically increases the risk weight of relevant contextual features for that hospital, making subsequent identity verification requests initiated at that hospital more sensitive to triggering deep detection processes.

[0081] As demonstrated by the two application examples above, the dynamic facial recognition method provided in this application is applicable not only to business scenarios in the financial sector, such as insurance signing, loan approval, and claims applications, but also to business scenarios in the medical sector, such as medical insurance registration, medical insurance settlement, electronic health card application, and remote consultation. In the financial sector, this application can effectively defend against attacks such as photo manipulation, screen playback, and deepfake videos, ensuring the security of financial transactions. In the medical sector, this application can effectively prevent impersonation for medical treatment and theft of medical insurance cards, ensuring the security of medical insurance funds and patient medication safety. Both applications utilize a lightweight liveness detection model deployed on the client side for initial liveness detection. On the server side, multimodal data and historical identity information are integrated to obtain a dynamic context vector, which is then input into a risk level classification model. Based on the risk level, the corresponding identity verification strategy is executed, achieving a layered liveness detection architecture with end-to-cloud collaboration, balancing real-time performance and accuracy. Compared to traditional process solutions, this application achieves dynamic facial recognition verification without additional hardware investment, improving security, efficiency, and generalization.

[0082] Please see Figure 3 In some embodiments, the liveness detection of the user in step S101 using a lightweight liveness detection model pre-deployed on the client may include, but is not limited to, steps S201 to S203.

[0083] Step S201: Obtain the user's detection video stream.

[0084] The detection video stream refers to a sequence of images consisting of multiple video frames continuously captured by the client's camera during the liveness detection process. The detection video stream serves as the raw input data for the client's liveness detection and records dynamic changes in the user's face. For example, the detection video stream can be a visible light video stream with a resolution of 720P and a frame rate of 25 frames per second, lasting 3 to 5 seconds.

[0085] In this context, "user" refers to a natural person who needs to undergo identity verification. Users must verify their identity to confirm its authenticity when conducting financial or medical transactions. For example, a user can be an insurance policyholder, loan applicant, claims applicant, a patient registering for an appointment, or a medical insurance settlement applicant, etc.

[0086] In this step, after receiving the user's identity verification trigger command, the client starts the camera to capture the user's detection video stream. During the capture process, the client can display guiding prompts on the screen, such as prompting the user to place their face in the center of the camera's field of view and maintain a natural expression.

[0087] Step S202: Extract a continuous video frame sequence from the detected video stream. The continuous video frame sequence is a sequence of multiple video frames including the user's image.

[0088] A continuous video frame sequence refers to a sequence of multiple video frames, including the user's image, extracted chronologically from a detection video stream. This sequence preserves the temporal dynamic information of the face within the video stream and serves as input data for lightweight liveness detection models. For example, a continuous video frame sequence of 125 frames can be extracted from a 5-second detection video stream at 25 frames per second, with each frame having a resolution of 640 x 480.

[0089] In this step, the client extracts the frame sequence from the acquired detection video stream to obtain a continuous video frame sequence. During the extraction process, the client can preprocess the video frames, such as face detection and cropping, image normalization, etc., to reduce the computational load of subsequent model inference.

[0090] Step S203: Input the continuous video frame sequence into the lightweight liveness detection model to obtain the liveness confidence output by the lightweight liveness detection model. The lightweight liveness detection model is trained based on a pre-constructed training set, which includes multiple video frames and the corresponding label for each video frame. The label includes liveness and non-liveness.

[0091] Lightweight liveness detection models refer to client-side liveness detection models with a small number of parameters. These models employ lightweight network structures, enabling real-time operation even with limited client-side computing resources. For example, a lightweight liveness detection model could be based on MobileNetV3 combined with a temporal shift module. This module models the temporal relationships between video frames by shifting channel features over time, allowing the model to capture dynamic changes in facial features.

[0092] The liveness confidence score is the probability value output by the lightweight liveness detection model, indicating that the user is alive. The liveness confidence score is a floating-point number between 0 and 1; the closer the value is to 1, the greater the probability that the user is alive, and the closer the value is to 0, the greater the probability that the user is not alive. For example, a liveness confidence score of 0.85 means that the model determines the user to be alive with an 85% probability.

[0093] The training set refers to the dataset used to train the lightweight liveness detection model. The training set includes multiple video frames and a label for each frame, categorized as either live or non-live. The training set is obtained by labeling live video frames captured from real users and non-live video frames captured through attacks such as photos and screen playback. For example, the training set could include 10,000 live video frames and 10,000 non-live video frames, each labeled with either a live or non-live tag.

[0094] Here, labels refer to the category annotations corresponding to each video frame in the training set. Labels include two categories: live and inactive. A live label indicates that the video frame came from a real user, while an inactive label indicates that the video frame came from an attack method such as photo manipulation or screen playback. For example, a label can be 1 to represent live and 0 to represent inactive.

[0095] In this step, the client inputs the continuous video frame sequence extracted in step S202 into the lightweight liveness detection model. The model performs feature extraction and temporal modeling on the continuous video frame sequence and outputs the liveness confidence score. During the training phase, the lightweight liveness detection model learns the feature differences between live and non-live video frames in the training set, enabling it to output the liveness confidence score for newly input continuous video frame sequences during the inference phase.

[0096] Step S204: Determine the identity verification result based on the comparison between the liveness confidence score and the preset confidence threshold.

[0097] The pre-set confidence threshold refers to a pre-defined confidence boundary value used to determine whether a user is a live entity. The pre-set confidence threshold can be configured according to the security level requirements of the business scenario; the higher the security level requirement, the higher the pre-set confidence threshold can be set. For example, the pre-set confidence threshold could be 0.5. When the liveness confidence is greater than or equal to 0.5, the user is determined to be a live entity; when the liveness confidence is less than 0.5, the user is determined to be a non-live entity.

[0098] The identity verification result refers to the liveness detection conclusion obtained by the client based on the comparison between the liveness confidence score and a preset confidence threshold. The identity verification result indicates whether the user is alive and is a preliminary judgment sent by the client to the server. For example, the identity verification result can be either "live" or "not alive".

[0099] In this step, the client compares the liveness confidence score obtained in step S203 with a preset confidence threshold, and determines the identity verification result based on the comparison result. When the liveness confidence score is greater than or equal to the preset confidence threshold, the identity verification result is determined to be live, and the client sends the identity verification result along with multimodal data and historical identity information to the server; when the liveness confidence score is less than the preset confidence threshold, the identity verification result is determined to be non-live, and the client directly prompts the user that the verification failed, without sending data to the server.

[0100] The technical solution provided in this application has at least the following beneficial effects: First, by extracting a continuous video frame sequence from the detected video stream and inputting it into a lightweight liveness detection model, compared with the existing liveness detection scheme based on single-frame images, the continuous video frame sequence can retain the temporal dynamic information of the face, enabling the model to capture the temporal inconsistencies of attack methods such as photo re-capture and screen playback, thereby improving the accuracy of liveness detection; Second, by using a lightweight liveness detection model to perform liveness detection on the client side, the model has a small number of parameters and fast inference speed, and can run in real time under the limited computing resources of the client side. At the same time, it can intercept obvious non-liveness attacks on the client side, reduce unnecessary data transmission and server-side computing power consumption, and improve the overall processing efficiency of the system; Third, by training the lightweight liveness detection model based on a pre-built training set, which includes video frames labeled with both liveness and non-liveness, the model can learn the feature differences between real faces and attack methods, improving the model's ability to recognize and generalize various attack methods.

[0101] Please see Figure 4 In some embodiments, the process of integrating multimodal data and historical identity information in step S103 to obtain a dynamic context vector may include, but is not limited to, steps S301 to S307.

[0102] Step S301: Extract features from the keyframe video stream to obtain video features.

[0103] In this embodiment, the multimodal data includes the user's device motion data, and the historical identity information includes the spatiotemporal context of the interaction between the client and the user during the liveness detection process, the user's historical verification records, and the user's risk tags.

[0104] Device motion data refers to data collected during liveness detection using motion sensors such as accelerometers and gyroscopes on the client device, reflecting the device's motion state. Device motion data can be used to help determine whether a user is holding the device naturally, thereby identifying attacks such as screen playback on a fixed stand. For example, device motion data may include three-axis acceleration data and three-axis angular velocity data, with a sampling rate of 50 Hz.

[0105] Spatiotemporal context refers to the collection of temporal and spatial information about the interaction between the client and the user during the liveness detection process. It characterizes the environmental state during user identity verification and can help identify risky scenarios such as logins from different locations or verifications occurring during unusual time periods. For example, spatiotemporal context may include the timestamp of the current verification, the client's internet protocol address, device identifier, and geographic coordinates.

[0106] Historical verification records refer to the collection of records of a user's past identity verification behaviors prior to the current verification. These records are used to characterize a user's verification behavior habits and can help identify abnormal behavior patterns such as frequent verifications and bulk registrations. For example, historical verification records may include the number of verifications a user has performed in the past 30 days, the distribution of verification time, the distribution of verification results, and the distribution of verification devices.

[0107] Risk tags refer to labeling information obtained from third-party risk databases to indicate a user's risk level. Risk tags are used to incorporate external risk intelligence into the identity verification process, assisting in the identification of high-risk users such as those involved in black market activities, fraud groups, etc. For example, risk tags may include categories such as blacklisted users, graylisted users, and normal users, and the source of these tags can be matching results from industry-shared black market databases.

[0108] In this context, a keyframe video stream refers to a sequence of key video frames containing the user's image, extracted from the original detection video stream during liveness detection. The keyframe video stream is the source of video feature extraction, preserving the visual feature information of the user's face. For example, a keyframe video stream could be a sequence of 25 keyframes extracted from a 5-second detection video stream at a sampling rate of 5 frames per second.

[0109] Video features refer to the feature representations used to characterize the user's facial visual information obtained after feature extraction from keyframe video streams. Video features can include two dimensions: facial appearance features and temporal dynamic features. For example, video features can be a 512-dimensional feature vector extracted from keyframe video streams using a 3D convolutional neural network, where the first 256 dimensions represent facial appearance features and the last 256 dimensions represent temporal dynamic features.

[0110] In this step, the server extracts features from the received keyframe video stream to obtain video features. Feature extraction can employ a pre-trained human feature extraction network, such as a feature extraction model based on residual networks or 3D convolutional neural networks, to extract features from the keyframe video stream frame by frame or segment by segment and fuse them into a unified video feature vector.

[0111] Step S302: Extract features from the equipment motion data to obtain motion features.

[0112] Motion features refer to the feature representations used to characterize the motion state of client devices after feature extraction from device motion data. Motion features can include two dimensions: acceleration features and angular velocity features. For example, a motion feature can be a 64-dimensional feature vector obtained by concatenating the mean, variance, peak value, and other statistical measures calculated from the three-axis acceleration data and the three-axis angular velocity data.

[0113] In this step, the server extracts features from the received device motion data to obtain motion features. The feature extraction process may include data cleaning, time-domain statistics calculation, and frequency-domain feature extraction, transforming the original time-series motion data into a fixed-dimensional motion feature vector.

[0114] Step S303: Extract features from the spatiotemporal context to obtain spatiotemporal features.

[0115] Spatiotemporal features refer to the feature representations used to characterize the user verification environment state after feature extraction from the spatiotemporal context. Spatiotemporal features can include two dimensions: temporal features and spatial features. For example, temporal features can be one-hot encoded vectors obtained by converting the verification timestamp into time period codes (such as morning, afternoon, and night), and spatial features can be embedding vectors obtained by converting Internet Protocol addresses and geographic coordinates into area codes.

[0116] In this step, the server extracts features from the received spatiotemporal context to obtain spatiotemporal features. The feature extraction process may include steps such as temporal encoding, spatial encoding, and feature fusion, transforming unstructured spatiotemporal context information into fixed-dimensional spatiotemporal feature vectors.

[0117] Step S304: Extract features from historical verification records to obtain behavioral features.

[0118] Among them, behavioral features refer to the feature representations obtained after feature extraction from historical verification records, used to characterize users' verification behavior habits. Behavioral features can include dimensions such as verification frequency features, verification time distribution features, and verification result distribution features. For example, a behavioral feature can be a 128-dimensional feature vector obtained by encoding statistics such as the number of verifications, verification time distribution, and verification pass rate of users in the past 30 days.

[0119] In this step, the server extracts features from the received historical verification records to obtain behavioral features. The feature extraction process may include steps such as record aggregation, statistical calculation, and feature encoding, transforming the historical verification records into fixed-dimensional behavioral feature vectors.

[0120] Step S305: Query the risk label in the pre-stored risk label database to obtain the risk weight associated with the risk label.

[0121] The risk label database refers to a pre-stored data storage system containing the mapping relationship between risk labels and risk weights. The risk label database is used to convert risk labels into numerical risk weights that can participate in feature encoding. For example, the risk label database can be a key-value pair storage system, where the key is the risk label category (e.g., blacklist, graylist, normal), and the value is the corresponding risk weight (e.g., 1.0, 0.5, 0.0).

[0122] In this context, risk weight refers to a numerical value associated with a risk label in the risk label database, used to quantify the degree of risk. A higher risk weight indicates a greater level of risk for the user corresponding to that risk label. For example, a risk weight of 1.0 can be assigned to a blacklist user, 0.5 to a graylist user, and 0.0 to a normal user.

[0123] In this step, the server queries a pre-stored risk tag database based on the received risk tag to obtain the risk weight associated with that risk tag. The query process can be completed quickly using database indexes, ensuring that the acquisition of risk weights does not affect the real-time performance of the overall verification process.

[0124] Step S306: Encode the risk weights to obtain the risk label features.

[0125] Risk label features refer to the feature representations used to characterize external risk intelligence for users, obtained by feature encoding of risk weights. Feature encoding transforms scalar risk weights into feature vector forms that the model can process. For example, risk label features can be 16-dimensional feature vectors transformed from risk weights through one-hot encoding or embedding encoding.

[0126] In this step, the server performs feature encoding on the risk weights obtained in step S305 to obtain risk label features. Feature encoding can be performed using one-hot encoding, embedding encoding, or numerical normalization to transform scalar risk weights into fixed-dimensional risk label feature vectors.

[0127] Step S307: Concatenate the video features, motion features, spatiotemporal features, behavioral features, and risk label features to obtain a dynamic context vector.

[0128] Feature concatenation refers to the operation of joining multiple feature vectors in a predetermined order into a longer feature vector. Feature concatenation can unify features from different dimensions and sources into a single comprehensive feature representation, which can then be used as input to a subsequent risk level classification model. For example, feature concatenation can combine 512-dimensional video features, 64-dimensional motion features, 32-dimensional spatiotemporal features, 128-dimensional behavioral features, and 16-dimensional risk label features into a single 752-dimensional comprehensive feature vector.

[0129] The dynamic context vector (VCV) is a comprehensive feature vector obtained by concatenating video features, motion features, spatiotemporal features, behavioral features, and risk label features. The VVC integrates both biometric and non-biometric information, comprehensively depicting the user's identity and risk status in the current verification session, and is used as input to the risk level classification model.

[0130] In this step, the server concatenates the video features, motion features, spatiotemporal features, behavioral features, and risk label features obtained in steps S301 to S306 in a preset order to obtain a dynamic context vector, which is used to input the risk level classification model for risk scoring.

[0131] The technical solution provided in this application has at least the following beneficial effects: First, by extracting features from keyframe video streams, device motion data, spatiotemporal context, historical verification records, and risk tags and concatenating them into a dynamic context vector, compared to the existing technology that relies solely on single-frame facial feature comparison, the dynamic context vector integrates information from five dimensions: video features, motion features, spatiotemporal features, behavioral features, and risk tag features. This allows for a comprehensive characterization of the user's risk status from both biological and non-biological perspectives, improving the accuracy and comprehensiveness of risk assessment. Second, by querying risk tags in the risk tag database to obtain risk weights and encoding these weights, external risk intelligence can be introduced into the dynamic context vector in a numerical form. This enables the risk level classification model to analyze third-party risk data, improving the ability to identify black market users and fraud gangs. Third, by concatenating features into a dynamic context vector, the feature concatenation operation is computationally simple and highly interpretable, facilitating the unified processing of multi-source heterogeneous features by the risk level classification model. Furthermore, the extraction processes of each feature are independent, facilitating parallel computation and improving the efficiency of information integration.

[0132] Please see Figure 5 In some embodiments, the execution of the corresponding identity verification strategy based on the risk level in step S106 to obtain the user's identity verification result may include, but is not limited to, steps S401 to S403.

[0133] Step S401: When the risk level is the first risk level, the first verification strategy is used to verify the multimodal data to obtain the first verification result.

[0134] The first risk level refers to the risk level determined when the risk score is lower than the preset risk threshold, indicating that the user is in a low-risk scenario. Users corresponding to the first risk level typically have characteristics such as normal historical verification records and using familiar equipment for verification in frequently used locations. For example, when a user's risk score is 0.3 and the preset risk threshold is 0.5, the risk level is determined to be the first risk level.

[0135] The first verification strategy refers to a verification strategy with lower detection accuracy corresponding to the first risk level. This strategy verifies identity by comparing keyframe facial features, which requires less computation and is faster. For example, the first verification strategy could involve extracting a keyframe sequence from a keyframe video stream, inputting it into a residual network model to extract facial feature vectors, calculating the similarity with pre-stored baseline facial feature vectors, and determining the verification result based on the similarity.

[0136] The first verification result refers to the verification conclusion obtained after verifying the multimodal data using the first verification strategy. The first verification result can include two states: verification passed and verification failed. For example, when the similarity calculated by the first verification strategy is greater than a preset similarity threshold, the first verification result is verification passed.

[0137] In this step, when the risk level determined in step S105 is the first risk level, the server uses the first verification strategy to verify the multimodal data and obtains the first verification result.

[0138] Step S402: When the risk level is the second risk level, the second verification strategy is used to verify the multimodal data to obtain the second verification result. The second risk level is higher than the first risk level, and the detection accuracy of the second verification strategy is higher than the detection accuracy of the first verification strategy.

[0139] The second risk level refers to the risk level determined when the risk score is greater than or equal to a preset risk threshold, indicating that the user is in a high-risk scenario. Users corresponding to the second risk level typically have characteristics such as logging in from a different location using a new device for verification, having abnormal historical verification records, or being blacklisted or graylisted. For example, when a user's risk score is 0.7 and the preset risk threshold is 0.5, the risk level is determined to be the second risk level.

[0140] The second verification strategy refers to a verification strategy with higher detection accuracy corresponding to the second risk level. This strategy completes identity verification through action command verification and multimodal fusion analysis, involving significant computational effort but achieving high accuracy. For example, the second verification strategy might involve sending an action command to the client, obtaining a verification video stream of the user performing a predetermined action, extracting facial key point sequences and pulse signal features from the verification video stream, performing micro-expression recognition, 3D deformation modeling, and multimodal fusion analysis, and determining the verification result based on the multimodal fusion features.

[0141] The second verification result refers to the verification conclusion obtained after verifying the multimodal data using a second verification strategy. The second verification result can include two states: verification passed and verification failed. For example, when the matching degree between the multimodal fusion features obtained by the second verification strategy and the baseline fusion features is higher than a preset matching threshold, the second verification result is verification passed.

[0142] In this step, when the risk level determined in step S105 is the second risk level, the server uses the second verification strategy to verify the multimodal data and obtain the second verification result.

[0143] Step S403: Generate an identity verification result based on the first verification result or the second verification result.

[0144] The identity verification result refers to the final verification conclusion generated based on either the first or second verification result. When the risk level is the first risk level, the identity verification result is generated based on the first verification result; when the risk level is the second risk level, the identity verification result is generated based on the second verification result. For example, if the first verification result is "verification passed," the identity verification result is "verification passed"; if the second verification result is "verification failed," the identity verification result is "verification failed."

[0145] In this step, the server generates the final identity verification result based on the verification result obtained in step S401 or step S402, and returns the identity verification result to the client, which then displays the verification conclusion to the user.

[0146] The technical solution provided in this application has at least the following beneficial effects: First, by executing corresponding identity verification strategies according to risk levels, using a first verification strategy at the first risk level and a second verification strategy at the second risk level, dynamic allocation of detection resources can be achieved. A lightweight verification strategy is used in low-risk scenarios to improve efficiency, while a high-precision verification strategy is used in high-risk scenarios to ensure security, thus achieving a balance between security and efficiency. Second, by setting the detection precision of the second verification strategy to be higher than that of the first verification strategy, higher-risk users can be ensured to undergo more stringent verification, effectively defending against advanced attack methods such as deepfake videos and 3D masks, thereby improving the security of identity verification. Third, by generating identity verification results based on the first or second verification results, the generation logic of the verification results is clear, facilitating system implementation and subsequent audit traceability.

[0147] Please see Figure 6 In some embodiments, the first verification strategy used in step S401 to verify the multimodal data and obtain the first verification result may include, but is not limited to, steps S501 to S504.

[0148] Step S501: Extract the keyframe sequence from the keyframe video stream.

[0149] A keyframe sequence refers to a set of video frames selected from a keyframe video stream according to preset rules for facial feature extraction. The selection rules for the keyframe sequence can be determined based on indicators such as face quality score and inter-frame differences, to select video frames with high quality and abundant information. For example, a keyframe sequence could be a sequence of 5 optimal keyframes selected from a 25-frame keyframe video stream based on face quality score.

[0150] In this step, the server extracts a keyframe sequence from the received keyframe video stream for subsequent portrait feature extraction. The extraction process may include steps such as face detection, face quality scoring, and frame selection to ensure that each frame in the selected keyframe sequence contains clear portrait information.

[0151] Step S502: Input the keyframe sequence into the pre-constructed first-precision model to obtain the first portrait feature vector output by the first-precision model. The first-precision model is a residual network model.

[0152] The first-precision model refers to a pre-built machine learning model used to extract portrait feature vectors from keyframe sequences. The first-precision model employs a residual network structure, enabling it to extract high-dimensional feature representations from portrait images. For example, a first-precision model could be a portrait feature extraction model based on a ResNet50 residual network, with a keyframe sequence as input and a 512-dimensional portrait feature vector as output.

[0153] The residual network model refers to a deep convolutional neural network that employs a residual connection structure. By introducing skip connections, residual networks enable the network to learn residual mappings rather than direct mappings, thereby alleviating the vanishing gradient problem in deep network training. For example, a residual network model could be ResNet50, containing 50 convolutional layers, which achieves efficient training of deep networks through residual connections.

[0154] The first portrait feature vector refers to the high-dimensional vector output by the first-precision model, used to characterize the user's portrait features. The first portrait feature vector is the foundational data for portrait identity comparison. For example, the first portrait feature vector can be a 512-dimensional floating-point vector, where each dimension represents a specific visual feature of the portrait.

[0155] In this step, the server inputs the keyframe sequence extracted in step S501 into the pre-constructed first precision model. The model extracts and fuses features of each portrait in the keyframe sequence and outputs the first portrait feature vector.

[0156] Step S503: Calculate the similarity based on the first portrait feature vector and the pre-stored baseline portrait feature vector.

[0157] The baseline facial feature vector refers to the pre-stored facial feature vector collected during the user's registration phase. As the benchmark data for facial comparison, the baseline facial feature vector is used to compare with the facial feature vector extracted during the verification phase. For example, the baseline facial feature vector could be a 512-dimensional facial feature vector extracted and stored using a first-precision model when the user opens an account or registers.

[0158] Similarity refers to the degree of similarity between the feature vector of the first portrait and the feature vector of the reference portrait. Similarity can be calculated using metrics such as cosine similarity and Euclidean distance. For example, cosine similarity can be used, with values ​​ranging from -1 to 1; the closer the value is to 1, the more similar the two feature vectors are.

[0159] In this step, the server calculates the similarity between the first portrait feature vector obtained in step S502 and the pre-stored baseline portrait feature vector. The similarity calculation can use the cosine similarity formula, which uses the cosine of the angle between the two feature vectors as the similarity measure.

[0160] Step S504: Determine the first verification result based on similarity.

[0161] The first verification result refers to the verification conclusion determined by comparing the similarity with a preset similarity threshold. When the similarity is greater than the preset similarity threshold, the first verification result is "verification passed"; when the similarity is less than or equal to the preset similarity threshold, the first verification result is "verification failed". For example, the preset similarity threshold can be 0.8, and when the similarity is 0.92, the first verification result is "verification passed".

[0162] In this step, the server compares the similarity calculated in step S503 with the preset similarity threshold, and determines the first verification result based on the comparison result.

[0163] The technical solution provided in this application has at least the following beneficial effects: First, by extracting keyframe sequences from keyframe video streams and inputting them into a residual network model to extract facial feature vectors, compared with the existing face comparison schemes based on single-frame images, the keyframe sequences contain multi-frame facial information, which can improve the robustness of facial feature extraction and reduce comparison errors caused by poor quality of single-frame images; Second, by using a residual network model as the first precision model, the residual network alleviates the gradient vanishing problem in deep network training through skip connections, and can learn richer facial feature representations, improving the discriminative power and comparison accuracy of facial feature vectors; Third, by determining the first verification result based on similarity, the similarity calculation method is simple and efficient, and identity verification can be completed quickly in low-risk scenarios, improving user experience and system throughput.

[0164] Please see Figure 7 In some embodiments, the second verification strategy used in step S402 to verify the multimodal data and obtain a second verification result may include, but is not limited to, steps S601 to S606.

[0165] Step S601: Send an action instruction to the client to instruct the user to perform a predetermined action, and obtain a verification video stream returned by the client during the user's performance of the predetermined action. The predetermined action includes at least one of blinking and head shaking.

[0166] Action commands are messages sent from the server to the client, instructing the user to perform specific actions. By requiring the user to perform randomly specified actions, action commands can effectively defend against attacks such as screen replay and deepfake videos, because attackers find it difficult to generate fake videos that match the specified actions in real time. For example, an action command could be to instruct the user to blink twice in succession and then shake their head to the left.

[0167] In this context, a predetermined action refers to a specific action specified in the action instruction that the user needs to perform. Predetermined actions include at least one of blinking and head shaking, and may also include other facial movements such as opening the mouth or nodding. For example, a predetermined action could be a combination of blinking and head shaking, requiring the user to blink twice and then shake their head to the left once.

[0168] The verification video stream refers to the video stream captured by the client during the user's execution of a predetermined action. The verification video stream records the dynamic changes in the user's face during the action and serves as the foundational data for subsequent micro-expression recognition, 3D deformation modeling, and pulse signal extraction. For example, the verification video stream could be a 720P resolution video stream with a frame rate of 30 frames per second and a duration of 5 seconds.

[0169] In this step, the server sends an action command to the client. After receiving the action command, the client displays an action guidance prompt on the screen, and at the same time starts the camera to capture the verification video stream during the user's execution of the predetermined action, and returns the verification video stream to the server.

[0170] Step S602: Extract facial landmark sequences and pulse signal features from the verification video stream.

[0171] A facial landmark sequence refers to a sequence of facial landmarks detected from each frame of the verification video stream, arranged in chronological order. Facial landmark sequences can characterize the deformation and motion trajectory of a face during movement. For example, a facial landmark sequence could be a sequence of 68 facial landmarks detected from each frame of the verification video stream, with each landmark containing both horizontal and vertical coordinates.

[0172] Among these, pulse signal features refer to the features extracted from the verification video stream that reflect changes in blood pulsation on the user's face. Based on photoplethysmography (PPG), pulse signal features extract pulse signals by analyzing minute periodic changes in facial skin color, serving as an auxiliary basis for liveness detection. For example, pulse signal features could include pulse frequency and amplitude extracted from the forehead area of ​​the face.

[0173] In this step, the server extracts facial landmark sequences and pulse signal features from the verification video stream returned by the client. The extraction of facial landmark sequences can employ a pre-trained facial landmark detection model, while the extraction of pulse signal features can utilize an algorithm based on facial skin color change analysis.

[0174] Step S603: Perform micro-expression recognition on the facial key point sequence to obtain micro-expression features.

[0175] Micro-expression recognition refers to the process of analyzing and identifying subtle facial expression changes reflected in a sequence of facial landmarks. Micro-expression recognition can capture minute, involuntary movements of facial muscles during a user's actions, serving as a basis for determining liveness. For example, micro-expression recognition can identify micro-expressions such as the contraction of the orbicularis oculi muscle when blinking or the movement of neck muscles when shaking the head.

[0176] Micro-expression features refer to the feature representations used to characterize the dynamics of a user's facial micro-expressions, obtained by performing micro-expression recognition on a sequence of facial key points. For example, micro-expression features can be a 128-dimensional feature vector obtained by encoding the temporal changes of a sequence of facial key points.

[0177] In this step, the server performs micro-expression recognition on the facial key point sequence extracted in step S602 to obtain micro-expression features. Micro-expression recognition can employ micro-expression recognition models based on temporal convolutional networks or recurrent neural networks to classify and extract features from the temporal change patterns of the facial key point sequence.

[0178] Step S604: Perform three-dimensional deformation modeling on the facial key point sequence to obtain three-dimensional deformation features.

[0179] 3D deformation modeling refers to the process of reconstructing a 3D deformation model of a face based on a sequence of facial key points. 3D deformation modeling can infer the deformation state of a 3D face from 2D facial key points, and is used to distinguish the 3D deformation of a real face from the planar motion of 2D attacks such as photographs and screen playback. For example, 3D deformation modeling can use 3D Morphable Model (3DMM) technology to estimate the 3D shape and pose parameters of a face based on facial key points.

[0180] Among them, 3D deformation features refer to the feature representations used to characterize the 3D deformation state of a user's face after performing 3D deformation modeling on the sequence of facial key points. For example, 3D deformation features can be a feature vector composed of the shape parameters, expression parameters, and pose parameters of the 3D deformation model.

[0181] In this step, the server performs 3D deformation modeling on the facial keypoint sequence extracted in step S602 to obtain 3D deformation features. 3D deformation modeling can employ a regression network based on the 3D deformation model to predict the shape, expression, and pose parameters of the 3D face based on the 2D facial keypoints.

[0182] Step S605: Fuse micro-expression features, three-dimensional deformation features, and pulse signal features to obtain multimodal fusion features.

[0183] Multimodal fusion features refer to the comprehensive feature representation obtained by fusing micro-expression features, three-dimensional deformation features, and pulse signal features. Multimodal fusion features combine information from three dimensions: facial micro-expression dynamics, three-dimensional deformation state, and physiological pulse signals, enabling a comprehensive depiction of the user's living state during actions. For example, a multimodal fusion feature can be a 224-dimensional feature vector obtained by concatenating 128-dimensional micro-expression features, 64-dimensional three-dimensional deformation features, and 32-dimensional pulse signal features.

[0184] In this step, the server fuses the micro-expression features obtained in step S603, the three-dimensional deformation features obtained in step S604, and the pulse signal features obtained in step S602 to obtain multimodal fused features. Feature fusion can be achieved through feature concatenation, attention weighting, or neural network fusion.

[0185] Step S606: Generate a second verification result based on the multimodal fusion features.

[0186] The second verification result refers to the verification conclusion determined based on the matching degree between the multimodal fusion feature and the benchmark fusion feature. When the matching degree between the multimodal fusion feature and the benchmark fusion feature is higher than a preset matching threshold, the second verification result is considered successful; otherwise, the second verification result is considered unsuccessful. For example, the preset matching threshold can be 0.85, and when the matching degree is 0.91, the second verification result is considered successful.

[0187] In this step, the server calculates the matching degree between the multimodal fusion features obtained in step S605 and the pre-stored benchmark fusion features, and generates a second verification result based on the comparison result of the matching degree and the preset matching threshold.

[0188] The technical solution provided in this application has at least the following beneficial effects: First, by sending action commands to the client to obtain a verification video stream of the user performing a predetermined action, compared with the passive acquisition of video streams, action command verification can effectively defend against attacks such as screen playback and deepfake videos, because attackers find it difficult to generate fake videos that match randomly specified actions in real time, thus improving the security of identity verification; Second, by extracting facial key point sequences and pulse signal features from the verification video stream, and performing micro-expression recognition and three-dimensional deformation modeling based on the facial key point sequences, the micro-expression features can capture facial muscles. The subtle, involuntary movements of flesh and the three-dimensional deformation features can distinguish the three-dimensional deformation of a real human face from the planar movements of two-dimensional attacks. Pulse signal features can provide physiological evidence of liveness. These three features complement each other, improving the accuracy and robustness of liveness detection. Third, by fusing micro-expression features, three-dimensional deformation features, and pulse signal features into a multimodal fusion feature and generating a second verification result based on it, the multimodal fusion feature combines information from multiple dimensions, which can comprehensively depict the user's liveness state, effectively defend against advanced attack methods such as deepfake videos and 3D masks, and improve the security of identity verification in high-risk scenarios.

[0189] Please see Figure 8 In some embodiments, after executing the corresponding identity verification strategy according to the risk level in step S106 to obtain the user's identity verification result, the above dynamic facial recognition identity verification method may also include, but is not limited to, steps S701 to S703.

[0190] Step S701: Based on the identity verification result, update the user's behavioral profile data to obtain the updated behavioral profile data. The behavioral profile data includes the user's verification history, behavioral habit characteristics, and risk tags. The updated behavioral profile data is used for subsequent identity verification requests.

[0191] Behavioral profiling data refers to a comprehensive dataset used to characterize a user's identity verification behavior habits and risk status. Behavioral profiling data includes a user's verification history, behavioral characteristics, and risk tags, providing historical behavioral evidence for subsequent identity verification. For example, behavioral profiling data can be a structured data object containing fields such as user identifier, verification history list, behavioral characteristic vector, and risk tags.

[0192] Verification history refers to the record information of a user's past identity verifications. Verification history includes information such as verification time, verification result, verification device, and verification location. For example, verification history can be a list of records sorted by time, with each record containing fields such as verification timestamp, verification result (pass or fail), device identifier, and internet protocol address.

[0193] Among these, behavioral habit features refer to the features extracted from a user's historical verification behavior that characterize the user's verification habits. Behavioral habit features can include verification frequency, verification time distribution, verification device preferences, etc. For example, behavioral habit features could be the user's average verification frequency over the past 30 days, a histogram of verification time distribution, and frequently used device identifiers.

[0194] Risk labels are tags used to indicate a user's risk level, determined based on their historical verification behavior and external risk data. Risk labels can be dynamically updated as user behavior changes. For example, if a user experiences multiple verification failures recently, their risk label can be updated from "normal user" to "graylist user."

[0195] In this step, the server updates the user's behavioral profile data based on the identity verification result obtained in step S106, resulting in updated behavioral profile data. This updated behavioral profile data will be used to extract historical identity information in subsequent identity verification requests, enabling subsequent verifications to make risk assessments based on the latest user behavioral profile.

[0196] Step S702: Generate a feedback signal based on the identity verification results and risk score.

[0197] Feedback signals, generated based on identity verification results and risk scores, guide the parameter updates of the risk level classification model. They reflect the consistency between the current risk score output by the model and the actual verification result, guiding the adjustment of model parameters. For example, when the risk score is low but the identity verification result is "verification failed," the feedback signal indicates that the model's dynamic context vector risk score for this type of user is too low, requiring an increase in the weight of the corresponding features.

[0198] In this step, the server generates a feedback signal based on the identity verification result obtained in step S106 and the risk score obtained in step S104. The feedback signal can be generated using preset rule mapping or gradient calculation based on the loss function.

[0199] Step S703: Based on the feedback signal, incrementally update the model parameters of the risk level classification model.

[0200] Incremental updates refer to the process of locally adjusting the model parameters of a risk level classification model based on feedback signals without system downtime. Incremental updates enable the model to continuously learn from new sample information and adapt to the evolution of new attack methods. For example, incremental updates can employ online gradient descent algorithms, calculating the gradient of the loss function based on feedback signals and updating the model parameters in small steps.

[0201] In this step, the server incrementally updates the model parameters of the risk level classification model based on the feedback signal generated in step S702. The incremental update process can be executed asynchronously in the server background without affecting the real-time performance of the normal identity verification process.

[0202] The technical solution provided in this application has at least the following beneficial effects: First, by updating the user's behavioral profile data according to the identity verification results, the behavioral profile data can reflect the user's latest verification behavior and risk status, providing a more accurate historical behavioral basis for subsequent identity verification requests and improving the timeliness and accuracy of risk judgment; Second, by generating feedback signals based on the identity verification results and risk scores, the feedback signals can reflect the consistency between the current output of the risk level classification model and the actual verification results, providing clear guidance signals for updating model parameters; Third, by incrementally updating the model parameters of the risk level classification model according to the feedback signals, the model has online learning capabilities, can continuously adapt to the evolution of new attack methods, and does not need to stop and retrain the entire model, improving the adaptability of the model and the availability of the system.

[0203] Figure 9 This is a schematic diagram of the dynamic facial recognition device provided in this application embodiment. Please refer to [link / reference]. Figure 9 This application embodiment also provides a dynamic facial recognition device 900, which can implement the above-mentioned dynamic facial recognition method. The dynamic facial recognition device 900 includes: Response module 901 is used to respond to the client's identity verification request, which is obtained by performing liveness detection on the user through a lightweight liveness detection model pre-deployed on the client. The acquisition module 902 is used to receive multimodal data and historical identity information sent by the client when the identity verification result indicates that the user is a live object. The multimodal data includes key frame video streams collected during the liveness detection process of the user, and the historical identity information is used to describe the user's historical verification information. Extraction module 903 is used to integrate multimodal data and historical identity information to obtain dynamic context vectors; Classification module 904 is used to input dynamic context vectors into a pre-trained risk level classification model to obtain the risk score output by the risk level classification model. The comparison module 905 is used to determine the risk level based on the comparison result between the risk score and the preset risk threshold; Execution module 906 is used to execute the corresponding identity verification strategy according to the risk level and obtain the user's identity verification result.

[0204] In some embodiments, the response module 901 is specifically used for: Obtain the user's detection video stream; Extract a continuous video frame sequence from the detected video stream. The continuous video frame sequence is a sequence of multiple video frames including the user's image. A continuous sequence of video frames is input into a lightweight liveness detection model to obtain the liveness confidence score output by the lightweight liveness detection model. The lightweight liveness detection model is trained based on a pre-built training set, which includes multiple video frames and the corresponding label for each video frame. The label includes liveness and non-liveness. The identity verification result is determined based on the comparison between the liveness confidence level and the preset confidence threshold.

[0205] In some embodiments, the extraction module 903 is specifically used for: Feature extraction is performed on the keyframe video stream to obtain video features; Feature extraction is performed on the equipment motion data to obtain motion features; Spatiotemporal features are obtained by extracting features from the spatiotemporal context; Feature extraction is performed on historical verification records to obtain behavioral features; Query the risk label in the pre-stored risk label database to obtain the risk weight associated with the risk label; Risk weights are feature-encoded to obtain risk label features; The video features, motion features, spatiotemporal features, behavioral features, and risk label features are concatenated to obtain a dynamic context vector.

[0206] In some embodiments, the execution module 906 is specifically used for: When the risk level is the highest, the first verification strategy is used to verify the multimodal data, and the first verification result is obtained. When the risk level is the second risk level, the second verification strategy is used to verify the multimodal data to obtain the second verification result. The second risk level is higher than the first risk level, and the detection accuracy of the second verification strategy is higher than the detection accuracy of the first verification strategy. An identity verification result is generated based on either the first or second verification result.

[0207] In some embodiments, the execution module 906 is specifically used for: Extracting keyframe sequences from visible light video streams; Input the keyframe sequence into the pre-built first-precision model to obtain the first portrait feature vector output by the first-precision model. The first-precision model is a residual network model. The similarity is calculated based on the first portrait feature vector and the pre-stored baseline portrait feature vector. The first verification result is determined based on similarity.

[0208] In some embodiments, the execution module 906 is specifically used for: Send an action instruction to the client to instruct the user to perform a predetermined action, and obtain a verification video stream returned by the client during the user's performance of the predetermined action. The predetermined action includes at least one of blinking and head shaking. Extract facial landmark sequences and pulse signal features from the verification video stream; Micro-expression recognition is performed on facial key point sequences to obtain micro-expression features; Three-dimensional deformation modeling is performed on the sequence of facial key points to obtain three-dimensional deformation features; By fusing micro-expression features, three-dimensional deformation features, and pulse signal features, multimodal fusion features are obtained. A second verification result is generated based on the multimodal fusion features.

[0209] In some embodiments, the dynamic facial recognition device 900 is further used for: Based on the identity verification results, update the user's behavioral profile data to obtain updated behavioral profile data. The behavioral profile data includes the user's verification history, behavioral habit characteristics, and risk tags. The updated behavioral profile data is used for subsequent identity verification requests. Based on the identity verification results and risk score, a feedback signal is generated; Based on the feedback signals, the model parameters of the risk level classification model are incrementally updated.

[0210] The specific implementation of the dynamic facial recognition device 900 is basically the same as the specific implementation of the dynamic facial recognition method described above, and will not be repeated here.

[0211] This application also provides an electronic device, which includes a memory and a processor. The memory stores a computer program, and the processor executes the computer program to implement the above-described dynamic facial recognition method. This electronic device can be any smart terminal, including tablet computers, in-vehicle computers, etc.

[0212] Please see Figure 10 , Figure 10 The hardware structure of an electronic device according to another embodiment is illustrated. The electronic device includes: The processor 1001 can be implemented using a general-purpose CPU (Central Processing Unit), microprocessor, application-specific integrated circuit (ASIC), or one or more integrated circuits, and is used to execute relevant programs to implement the technical solutions provided in the embodiments of this application. The memory 1002 can be implemented as a read-only memory (ROM), a static storage device, a dynamic storage device, or a random access memory (RAM). The memory 1002 can store the operating system and other applications. When the technical solutions provided in the embodiments of this specification are implemented through software or firmware, the relevant program code is stored in the memory 1002 and is called and executed by the processor 1001 using the dynamic facial recognition method of this application embodiment. Input / output interface 1003 is used to implement information input and output; The communication interface 1004 is used to enable communication and interaction between this device and other devices. Communication can be achieved through wired means (such as USB, network cable, etc.) or wireless means (such as mobile network, WIFI, Bluetooth, etc.). Bus 1005 transmits information between various components of the device (e.g., processor 1001, memory 1002, input / output interface 1003, and communication interface 1004); The processor 1001, memory 1002, input / output interface 1003 and communication interface 1004 are connected to each other within the device via bus 1005.

[0213] This application also provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the above-described dynamic facial recognition method.

[0214] Memory, as a non-transitory computer-readable storage medium, can be used to store non-transitory software programs and non-transitory computer-executable programs. Furthermore, memory may include high-speed random access memory, and may also include non-transitory memory, such as at least one disk storage device, flash memory device, or other non-transitory solid-state storage device. In some embodiments, memory may optionally include memory remotely located relative to the processor, and these remote memories can be connected to the processor via a network. Examples of such networks include, but are not limited to, the Internet, intranets, local area networks, mobile communication networks, and combinations thereof.

[0215] It should be noted that any AI models, software tools, or components not belonging to this company appearing in the embodiments of this application are merely illustrative examples and do not represent actual use. All user personal information involved in the embodiments of this application has been authorized (with the knowledge and consent) by the relevant parties or has been fully authorized by all parties, and the executing entity may obtain it through various legal and compliant means. The collection, storage, use, processing, transmission, provision, and disclosure of the information, data, and signals involved all comply with relevant laws and regulations and do not violate public order and good morals.

[0216] The embodiments described in this application are for the purpose of more clearly illustrating the technical solutions of the embodiments of this application, and do not constitute a limitation on the technical solutions provided by the embodiments of this application. As those skilled in the art will know, with the evolution of technology and the emergence of new application scenarios, the technical solutions provided by the embodiments of this application are also applicable to similar technical problems.

[0217] Those skilled in the art will understand that the technical solutions shown in the figures do not constitute a limitation on the embodiments of this application, and may include more or fewer steps than shown, or combine certain steps, or different steps.

[0218] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs.

[0219] Those skilled in the art will understand that all or some of the steps in the methods disclosed above, as well as the functional modules / units in the systems and devices, can be implemented as software, firmware, hardware, or suitable combinations thereof.

[0220] The terms “first,” “second,” “third,” “fourth,” etc. (if present) in the specification and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of this application described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms “comprising” and “having,” and any variations thereof, are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0221] It should be understood that in this application, "at least one (item)" means one or more, and "more than" means two or more. "And / or" is used to describe the relationship between related objects, indicating that three relationships can exist. For example, "A and / or B" can represent three cases: only A exists, only B exists, and both A and B exist simultaneously, where A and B can be singular or plural. The character " / " generally indicates that the preceding and following related objects are in an "or" relationship. "At least one (item) of the following" or similar expressions refer to any combination of these items, including any combination of single or plural items. For example, at least one (item) of a, b, or c can represent: a, b, c, "a and b", "a and c", "b and c", or "a and b and c", where a, b, and c can be single or multiple.

[0222] In the several embodiments provided in this application, it should be understood that the disclosed apparatus and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of the units described above is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between apparatuses or units may be electrical, mechanical, or other forms.

[0223] The units described above as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0224] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0225] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes multiple instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods of the various embodiments of this application. The aforementioned storage medium includes various media capable of storing programs, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0226] The preferred embodiments of the present application have been described above with reference to the accompanying drawings, but this does not limit the scope of the claims of the present application. Any modifications, equivalent substitutions, and improvements made by those skilled in the art without departing from the scope and substance of the embodiments of the present application shall be within the scope of the claims of the present application.

Claims

1. A method for dynamic facial recognition identity verification, characterized in that, The method includes: In response to the client's identity verification result, which is obtained by performing liveness detection on the user through a lightweight liveness detection model pre-deployed on the client; If the identity verification result indicates that the user is a live entity, the system receives multimodal data and historical identity information sent by the client. The multimodal data includes keyframe video streams collected during the liveness detection process of the user, and the historical identity information is used to describe the user's historical verification information. The multimodal data and historical identity information are integrated to obtain a dynamic context vector; The dynamic context vector is input into a pre-trained risk level classification model to obtain the risk score output by the risk level classification model. The risk level is determined based on the comparison between the risk score and the preset risk threshold. The corresponding identity verification strategy is executed according to the risk level to obtain the user's identity verification result.

2. The dynamic facial recognition method according to claim 1, characterized in that, The process of performing liveness detection on the user using a lightweight liveness detection model pre-deployed on the client includes: Obtain the detection video stream of the user; Extract a continuous video frame sequence from the detected video stream, wherein the continuous video frame sequence is a sequence of multiple video frames including the user's image; The continuous video frame sequence is input into the lightweight liveness detection model to obtain the liveness confidence score output by the lightweight liveness detection model. The lightweight liveness detection model is trained based on a pre-constructed training set, which includes multiple video frames and a label corresponding to each video frame. The label includes liveness and non-liveness. The identity verification result is determined based on the comparison between the liveness confidence score and the preset confidence threshold.

3. The dynamic facial recognition method according to claim 1, characterized in that, The multimodal data includes the user's device motion data, and the historical identity information includes the spatiotemporal context of the interaction between the client and the user during the liveness detection process, the user's historical verification records, and the user's risk tags. The process of integrating the multimodal data and historical identity information to obtain a dynamic context vector includes: Feature extraction is performed on the keyframe video stream to obtain video features; Feature extraction is performed on the motion data of the device to obtain motion features; Feature extraction is performed on the spatiotemporal context to obtain spatiotemporal features; Feature extraction is performed on the historical verification records to obtain behavioral features; The risk label is queried in a pre-stored risk label database to obtain the risk weight associated with the risk label; The risk weights are feature-encoded to obtain risk label features; The video features, motion features, spatiotemporal features, behavioral features, and risk label features are concatenated to obtain the dynamic context vector.

4. The dynamic facial recognition method according to claim 1, characterized in that, The step of executing the corresponding identity verification strategy based on the risk level to obtain the user's identity verification result includes: When the risk level is the first risk level, the multimodal data is verified using the first verification strategy to obtain the first verification result; When the risk level is the second risk level, the multimodal data is verified using a second verification strategy to obtain a second verification result, wherein the second risk level is higher than the first risk level, and the detection accuracy of the second verification strategy is higher than the detection accuracy of the first verification strategy. The identity verification result is generated based on the first verification result or the second verification result.

5. The dynamic facial recognition method according to claim 4, characterized in that, The step of verifying the multimodal data using a first verification strategy to obtain a first verification result includes: Extract keyframe sequences from the keyframe video stream; The keyframe sequence is input into a pre-constructed first precision model to obtain the first portrait feature vector output by the first precision model, where the first precision model is a residual network model. The similarity is calculated based on the first portrait feature vector and the pre-stored baseline portrait feature vector; Based on the similarity, the first verification result is determined.

6. The dynamic facial recognition method according to claim 4, characterized in that, The second verification strategy is used to verify the multimodal data to obtain a second verification result, including: Send an action instruction to the client to instruct the user to perform a predetermined action, and obtain a verification video stream returned by the client during the user's performance of the predetermined action, wherein the predetermined action includes at least one of blinking and head shaking. Extract facial landmark sequences and pulse signal features from the verification video stream; Micro-expression recognition is performed on the facial key point sequence to obtain micro-expression features; Three-dimensional deformation modeling is performed on the facial key point sequence to obtain three-dimensional deformation features; The micro-expression features, the three-dimensional deformation features, and the pulse signal features are fused to obtain multimodal fusion features; The second verification result is generated based on the multimodal fusion features.

7. The dynamic facial recognition method according to claim 1, characterized in that, After obtaining the user's identity verification result by executing the corresponding identity verification strategy according to the risk level, the method further includes: Based on the identity verification result, the user's behavioral profile data is updated to obtain updated behavioral profile data. The behavioral profile data includes the user's verification history, behavioral habit characteristics, and risk tags. The updated behavioral profile data is used for subsequent identity verification requests. Based on the identity verification results and the risk score, a feedback signal is generated; Based on the feedback signal, the model parameters of the risk level classification model are incrementally updated.

8. A dynamic facial recognition device, characterized in that, The device includes: The response module is used to respond to the identity verification result of the client, which is obtained by performing liveness detection on the user through a lightweight liveness detection model pre-deployed on the client. The acquisition module is used to receive multimodal data and historical identity information sent by the client when the identity verification result indicates that the user is a live entity. The multimodal data includes keyframe video streams collected during the liveness detection process of the user, and the historical identity information is used to describe the user's historical verification information. The extraction module is used to integrate the multimodal data and historical identity information to obtain a dynamic context vector; The classification module is used to input the dynamic context vector into a pre-trained risk level classification model to obtain the risk score output by the risk level classification model. The comparison module is used to determine the risk level based on the comparison result between the risk score and the preset risk threshold; The execution module is used to execute the corresponding identity verification strategy according to the risk level and obtain the user's identity verification result.

9. An electronic device, characterized in that, The electronic device includes a memory and a processor. The memory stores a computer program, and when the processor executes the computer program, it implements the dynamic facial recognition method according to any one of claims 1 to 7.

10. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by the processor, it implements the dynamic facial recognition method as described in any one of claims 1 to 7.

Images