Face recognition verification method and device and electronic equipment

Abstract

The invention belongs to the technical field of safety detection, and provides a face recognition verification method and device and electronic device.The method comprises the following steps that when a user to be verified initiates verification operation, a display area for displaying a face image of the user in real time is provided; setting at least two acquisition nodes, and generating effective acquisition domains corresponding to different acquisition nodes on the display area; face image data of a user to be verified in each effective collection domain and image sequences and device data generated between adjacent collection nodes are collected respectively; and extracting behavior characteristic data of the user to be verified according to the image sequence, verifying the matching between the behavior characteristic data and the equipment data, and determining the authenticity of the verification operation of the user to be verified according to the verification result. The method can enhance the security of living body detection, prevents forgery attacks, has the advantage of low computing resource consumption, and is suitable for the mobile terminal.

Description

Technical Field

[0001] This invention belongs to the field of security detection technology, and in particular relates to a face recognition verification method, device and electronic device. Background Technology

[0002] In the field of identity verification technology, facial recognition is a commonly used verification method. Facial recognition verifies a user's identity by identifying facial features. In facial recognition, facial features are digitized descriptions of the user's facial features extracted from their facial images.

[0003] Since facial features are unique for the vast majority of humans, the facial recognition features extracted from a user's facial image are also unique. Based on these features, different users can be accurately distinguished, thus achieving accurate user identification. However, facial recognition presents security risks in certain application scenarios.

[0004] The facial recognition workflow mainly consists of three stages: liveness detection, face capture, and face comparison. In the liveness detection stage, the system verifies whether the subject is a real person, preventing attackers from deceiving the verification system by replaying photos or videos. However, in some scenarios, attackers can bypass liveness detection by sending stolen facial images or videos using private devices, leading to system authentication errors and compromising security. Summary of the Invention

[0005] To address the aforementioned technical problems, this application proposes a face recognition verification method, device, and electronic device. By combining dynamic data acquisition nodes and interactive behavior feature analysis, the security and anti-spoofing capabilities of face recognition are significantly improved. The specific technical solution is as follows: Firstly, this application provides a face recognition verification method, comprising: When a user initiates a verification process, a display area is provided to show the user's facial image in real time. Set at least two acquisition nodes and generate effective acquisition fields on the display area corresponding to different acquisition nodes. The effective acquisition fields generated by any two adjacent acquisition nodes have differences in position and / or size and / or state on the display area. Facial image data of the user to be verified in each valid acquisition domain is collected, as well as image sequences and device data generated between adjacent acquisition nodes. The image sequences record the changes in the facial image data of the user to be verified in the display area between adjacent acquisition nodes, and the device data records the changes in the device position data when the user to be verified operates the verification device to complete the acquisition of facial image data in the valid acquisition domain between adjacent acquisition nodes. Behavioral feature data of the user to be verified is extracted from the image sequence, the matching of behavioral feature data with device data is verified, and the authenticity of the verification operation of the user to be verified is determined based on the verification results.

[0006] In one embodiment, the display area showing the user's facial image also displays image acquisition prompts for different acquisition nodes, prompting the user to acquire facial images within the corresponding valid acquisition area according to the image acquisition prompts.

[0007] In one implementation, after each successful acquisition of facial image data of the user to be verified within the current valid acquisition domain, the next valid acquisition domain is displayed.

[0008] In one implementation, the verification method for the facial image data of the user to be verified within the current valid acquisition domain is as follows: determine whether the facial contour of the user to be verified is completely within the valid acquisition domain, and if so, pass.

[0009] In one implementation, an image sequence is generated using a frame-sampling method, a fixed sampling period is set, and the image of the user to be verified in the display area is sampled between two acquisition nodes.

[0010] In one implementation, the method for extracting behavioral feature data of the user to be verified based on the image sequence is as follows: Mark feature points in the first frame of the image sequence; Track the changes in pixel coordinates of feature points in an image sequence; Calculate the true coordinates of feature points in the image sequence and establish a travel curve, which is the behavioral feature data.

[0011] In one implementation, the method for verifying the matching between behavioral feature data and device data is as follows: Standard feature data is calculated and generated based on equipment data; Calculate the goodness of fit between behavioral feature data and standard feature data; The authenticity of the verification operation of the user to be verified is determined based on the degree of fit.

[0012] Secondly, the facial recognition verification device provided in this application is characterized by comprising: The image acquisition module is used to acquire the facial image data of the user to be verified when the user initiates a verification operation. The node control module is used to set different acquisition nodes and generate a valid acquisition field on the display area set by the image acquisition module according to the acquisition nodes; The image sequence generation module is used to acquire consecutive facial image frames and generate an image sequence. The device status acquisition module is used to acquire the location data of the verification device during the verification operation. The feature extraction module is used to extract behavioral feature data of the user to be verified based on the image sequence; The verification module is used to verify the matching between behavioral feature data and device data, and to determine the authenticity of the verification operation of the user to be verified based on the verification results.

[0013] Thirdly, this application provides an electronic device characterized in that it includes a memory for storing computer program instructions and a processor for executing the program instructions, wherein when the computer program instructions are executed by the processor, the electronic device is triggered to execute the face recognition verification method.

[0014] The beneficial effects of this invention are as follows: (1) Enhance the security of liveness detection and prevent spoofing attacks: By using multiple dynamically changing acquisition nodes (effective acquisition domains with different positions, sizes, and states), the user is forced to adjust the device position, increasing the difficulty for attackers to simulate the attack; combined with image sequence analysis (such as the motion trajectory of facial feature points) and device sensor data (such as gyroscopes and accelerometers), it is ensured that the operation is a real-time interaction with a human, rather than static or pre-recorded data. (2) Implement dual verification using interactive behavior features: The movement of facial feature points is tracked using optical flow and other methods to generate behavioral feature curves (such as natural shaking of the handheld device and changes in facial micro-expressions); device data (such as changes in position and posture) is matched with behavioral features to verify that both conform to the operating rules of real people. Even if attackers forge facial images or videos, it is difficult to simulate real handheld device interaction behaviors (such as natural shaking and responsive adjustment); mechanically driven simulated masks or faces will be identified as forgeries because they lack real human behavioral characteristics (such as micro-expressions and device motion correlation). (3) Dynamic data collection strategies improve user experience and security: Dynamic effective acquisition domain: Changes in the acquisition area (position, size, tilt) of adjacent acquisition nodes guide users to adjust their devices naturally, reducing the feeling of deliberate coordination; Enhanced stealth: The data collection process reduces "command-based" operations, making it difficult for attackers to predict the next data collection request; (4) Low computational resource consumption, suitable for mobile terminals: Initial verification only needs to determine whether the face is within the valid acquisition area, reducing complex calculations; Liveness detection is achieved through behavior trajectory fitting (such as least squares support vector machine), balancing accuracy and efficiency; Wide applicability: It can be integrated into mobile devices such as mobile phones and smartwatches without the need for additional hardware; (5) Technological scalability: It can be combined with other biometrics (such as iris and voiceprint) or multimodal sensor data (such as pressure sensing and touch interaction) to further enhance security. Attached Figure Description

[0015] Figure 1 The diagram shown is a flowchart of the face recognition verification method in an embodiment of this application; Figure 2 The diagram shows the effective acquisition domains generated by different acquisition nodes in the embodiments of this application. Detailed Implementation

[0016] In the following description, certain specific details are set forth in order to provide a thorough understanding of various embodiments. However, those skilled in the art will understand that the invention can be practiced without these details.

[0017] To address the security issues of existing face recognition solutions, this specification proposes a face recognition verification method. To present the method in the embodiments of this specification, the inventors first analyze practical application scenarios of face recognition. Generally, face recognition includes a liveness detection step, and malicious actors can bypass liveness detection by sending stolen facial images or videos using private devices.

[0018] In practical applications, liveness detection is unidirectional; the verification device collects the facial information of the person being verified to perform liveness detection. This image acquisition process, based on instructions or requirements, is well-known, allowing malicious actors to bypass or forge samples that meet the detection requirements. However, facial recognition on mobile terminals (such as smartphones and smartwatches) is widespread and commonplace. By collecting and extracting the interaction between the person being verified and the verification device during the verification process, and further completing the verification, not only can liveness detection be achieved, but the problem of malicious actors bypassing or forging samples that meet the detection requirements can also be overcome. In particular, the process of collecting and extracting the interaction between the person being verified and the verification device during the verification process is not only relatively unique in the entire verification process, but also possesses human behavioral characteristics and the ability to conceal the process.

[0019] See Figure 1 The face recognition verification method provided in this application includes the following steps: S100: Determine whether the user to be verified has initiated a face recognition verification operation; If the determination is negative, return to step S100; if the determination is positive, execute step S110.

[0020] S110, Provides a display area for real-time display of the user's facial image; Generally, during the process of facial recognition verification via mobile terminals, a region matching the facial contour is set up to collect facial feature information. When the user to be verified ensures that their facial image is located within the region matching the facial contour, the verification can be completed. Therefore, a display area that can display the user's facial image in real time is also needed to assist the user to be verified.

[0021] Typically, the display area showing the user's facial image will also simultaneously display image acquisition prompts for different acquisition nodes, prompting the user to acquire facial images within the corresponding valid acquisition area according to the image acquisition prompts.

[0022] S120. Set at least two acquisition nodes and generate effective acquisition fields on the display area corresponding to different acquisition nodes. The effective acquisition fields generated by any two adjacent acquisition nodes have differences in position and / or size and / or state on the display area.

[0023] The specific configuration of the data acquisition node is as follows: The experiment uses two acquisition nodes, whose effective acquisition fields differ in position, size, and status on the display area. The specific configurations are shown in the table below:

[0024] According to the table above: 1. Display area prompts: On the display area, node 1 prompts "Please place the face in the center frame", and node 2 prompts "Please move the face to the left frame".

[0025] 2. Data Acquisition Process: The user to be verified first completes the facial image acquisition within the valid acquisition area of ​​node 1. After successful acquisition, the valid acquisition area of ​​node 2 is displayed.

[0026] Generally speaking, even if a region matching the facial contour (i.e., the effective acquisition area) is set on the display area, the size and position of this region are fixed. Ordinary methods only capture facial images in a specific state (usually frontal). A more secure method is to capture the user's actions (such as nodding, shaking, opening the mouth, blinking, etc.) within the area matching the facial contour. However, these are all executed unidirectionally by the verification device, and the verification purpose is intuitive, making it easier for criminals to break through.

[0027] This application instructs the user to complete facial image data acquisition by setting different acquisition nodes and different effective acquisition fields for different acquisition nodes. The key point is that it changes any one of the positions, sizes, or states of the effective acquisition fields on the display area between two adjacent acquisition nodes, with the following purposes: (1) Collect facial images of users to be verified by setting different valid collection fields at different collection nodes. This data can be directly used for subsequent identity verification. (2) This continuous acquisition emphasizes the image acquisition operation, while relatively masking and weakening other acquisition processes during the acquisition process; (3) Set different collection nodes and corresponding effective collection domains so that the user to be verified can spontaneously change the location of the verification device.

[0028] In this application, the position and size of the effective acquisition field on the display area are literal, while the state mainly refers to the tilt state, such as... Figure 2 The diagram shows the different effective acquisition areas corresponding to different acquisition nodes on the display area. The gray shaded area is the effective acquisition area. (a), (b), and (c) show the effective acquisition areas corresponding to three different acquisition nodes with different positions, sizes, and states, respectively.

[0029] Different acquisition nodes are distinguished by time sequence. That is, at a certain point in time, the acquisition node is determined, and the effective acquisition field corresponding to that acquisition node is determined. There is one and only one effective acquisition field. Only after the user to be verified completes the acquisition of his / her facial image according to the image acquisition prompt information will the next acquisition node be generated, and the effective acquisition field corresponding to that acquisition node will be regenerated.

[0030] In one implementation, the verification method for the facial image data of the user to be verified within the current valid acquisition domain is as follows: determine whether the facial contour of the user to be verified is completely within the valid acquisition domain; if so, the verification passes. This verification method is fast and efficient, does not require processing excessive facial detail information and expression feature information, and does not consume excessive computing resources.

[0031] S130. Collect facial image data of the user to be verified in each effective acquisition domain, as well as image sequences and device data generated between adjacent acquisition nodes. The image sequence records the change status of the facial image data of the user to be verified between adjacent acquisition nodes in the display area. The device data records the change of device position data when the user to be verified operates the verification device to complete the acquisition of facial image data in the effective acquisition domain between adjacent acquisition nodes. As described above, this application sets different acquisition nodes and different effective acquisition fields for different acquisition nodes. The effective acquisition fields of adjacent acquisition nodes will change in position, size and state, thereby guiding the user to be verified to move the mobile terminal during the verification process to change the relative position of the mobile terminal and the body, so as to complete the indicative image acquisition work.

[0032] More importantly, the process simultaneously captures the user's facial images to generate an image sequence and acquires device data; both are crucial for performing liveness detection. Since the user to be verified needs to capture facial images in different states according to verification instructions, the user moves the mobile terminal during verification, changing the relative position of the mobile terminal and their body. This interaction causes changes in device data and simultaneously generates data carrying realistic human behavioral characteristics. By simultaneously acquiring image sequences and device data, this approach overcomes the shortcomings of malicious actors bypassing liveness detection by inputting only images or videos through proprietary devices. Furthermore, some malicious actors use mechanically driven simulation masks or facial makeup to mimic human facial movements; however, these lack the relevant features of interactive operation and realistic human behavioral characteristics. Therefore, this application also overcomes the shortcomings of bypassing liveness detection using this method.

[0033] S140. Extract behavioral feature data of the user to be verified from the image sequence, verify the matching between the behavioral feature data and the device data, and determine the authenticity of the verification operation of the user to be verified based on the verification result.

[0034] As described above, the image sequence itself only records image data, but it carries the real human behavioral characteristics of the user to be verified, as well as the correspondence with the device data during the verification process. Therefore, after extracting the behavioral characteristic data of the user to be verified from the image sequence, the authenticity of the user's verification operation can be verified.

[0035] In step S130, the image sequence consists of several image frames based on time. Therefore, a fixed sampling period is set between two acquisition nodes to form the image sequence. The method for extracting the behavioral feature data of the user to be verified based on the image sequence is as follows: In the first frame of the image sequence, feature points are marked (generally, the corners of the eyes, corners of the mouth, or obvious moles on the face can be selected). Track the pixel coordinate changes of feature points in an image sequence (e.g., optical flow method) to obtain a dataset of feature point coordinates; Based on the coordinate dataset, the motion trajectory of the object is reconstructed by fitting with a least squares support vector machine to obtain the travel curve, which is the behavioral feature data. This curve carries the behavioral features of the user to be verified during the verification operation, for different acquisition nodes to complete image acquisition.

[0036] While the user being verified completes image acquisition at different acquisition nodes during the verification operation, the verification device will also experience changes in location data due to the user's handheld operation. These changes match the behavioral feature data and are unique and continuous. Therefore, the curve generated based on this location data can be used as standard feature data to verify and evaluate the authenticity of the verification operation.

[0037] When the behavioral feature data generated from the image sequence collected by the device has a high degree of fit with the standard feature data, it indicates that it is a real person verification and can pass the liveness detection. Conversely, when it is below a certain threshold, it indicates that the behavioral feature data does not match the standard feature data and cannot pass the liveness detection.

[0038] Generally speaking, the goodness of fit between behavioral feature data and standard feature data ranges from 0 to 1. A goodness of fit of 1 indicates a perfect fit. However, due to the micro-expressions on the human face, behavioral feature data can change due to smiling, making it almost impossible to achieve a goodness of fit of 1. Therefore, a goodness of fit of 0.9 or higher is generally considered to have passed the liveness detection.

[0039] This application, based on a face recognition verification method, further provides a face recognition verification device, which includes: The image acquisition module is used to acquire the facial image data of the user to be verified when the user initiates a verification operation. The node control module is used to set different acquisition nodes and generate a valid acquisition field on the display area set by the image acquisition module according to the acquisition nodes; The image sequence generation module is used to acquire consecutive facial image frames and generate an image sequence. The device status acquisition module is used to acquire the location data of the verification device during the verification operation. The feature extraction module is used to extract behavioral feature data of the user to be verified based on the image sequence; The verification module is used to verify the matching between behavioral feature data and device data, and to determine the authenticity of the verification operation of the user to be verified based on the verification results.

[0040] This application also provides an electronic device, including a memory for storing computer program instructions and a processor for executing the program instructions, wherein when the computer program instructions are executed by the processor, the electronic device is triggered to perform the above-described method steps.

[0041] It is understood that the electronic device may also include a communication bus, a user interface, and at least one external communication interface, wherein the communication bus is configured to enable communication between these components. The user interface may include a control panel, and the external communication interface may include standard wired and wireless interfaces.

[0042] It should be noted that, in the embodiments of this application, if the above-described control method is implemented as a software functional module and sold or used as an independent product, it can also be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the embodiments of this application, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), magnetic disks, or optical disks. Thus, the embodiments of this application are not limited to any specific hardware and software combination.

[0043] Accordingly, embodiments of this application provide a computer-readable storage medium storing a computer program thereon, characterized in that the computer program, when executed by a processor, implements the steps in the control method provided in the above embodiments.

[0044] The descriptions of the above embodiments of the electronic devices and storage media are similar to those of the above method embodiments, and have similar beneficial effects. For technical details not disclosed in the embodiments of the computer devices and storage media of this application, please refer to the descriptions of the method embodiments of this application for understanding.

[0045] It should be noted that the descriptions of the storage medium and device embodiments above are similar to the descriptions of the method embodiments above, and have similar beneficial effects. For technical details not disclosed in the storage medium and device embodiments of this application, please refer to the descriptions of the method embodiments of this application for understanding.

[0046] The above embodiments are only used to illustrate the technical solutions of the present invention, and are not intended to limit it.

Claims

1. A face recognition verification method, characterized in that, Includes the following steps: (1) When a user to be verified initiates a verification operation, a display area is provided to display the user's facial image in real time; (2) Set up at least two acquisition nodes and generate effective acquisition fields corresponding to different acquisition nodes on the display area. The effective acquisition fields generated by any two adjacent acquisition nodes have differences in position and / or size and / or state on the display area. (3) Collect facial image data of the user to be verified in each effective acquisition domain, as well as image sequences and device data generated between adjacent acquisition nodes. The image sequence records the change status of the facial image data of the user to be verified between adjacent acquisition nodes in the display area. The device data records the change of device position data when the user to be verified operates the verification device to complete the acquisition of facial image data in the effective acquisition domain between adjacent acquisition nodes. (4) Extract behavioral feature data of the user to be verified from the image sequence, verify the matching of behavioral feature data with device data, and determine the authenticity of the verification operation of the user to be verified based on the verification results.

2. The face recognition verification method according to claim 1, characterized in that, In step (1), the display area showing the user's facial image also displays image acquisition prompts for different acquisition nodes, prompting the user to acquire facial images within the corresponding valid acquisition area according to the image acquisition prompts.

3. The face recognition verification method according to claim 1, characterized in that, In step (2), after each successful acquisition of facial image data of the user to be verified within the current valid acquisition domain, the next valid acquisition domain is displayed.

4. The face recognition verification method according to claim 1, characterized in that, The verification method for the facial image data of the user to be verified in step (3) is as follows: determine whether the facial contour of the user to be verified is completely within the valid acquisition domain, and if so, pass.

5. The face recognition verification method according to claim 1, characterized in that, In step (4), the frame extraction method is used to generate the image sequence, a fixed sampling period is set, and the image of the user to be verified in the display area is sampled between two acquisition nodes.

6. The face recognition verification method according to claim 1, characterized in that, The method for extracting the behavioral feature data of the user to be verified based on the image sequence in step (4) is as follows: Mark feature points in the first frame of the image sequence; Track the changes in pixel coordinates of feature points in an image sequence; Calculate the true coordinates of feature points in the image sequence and establish a travel curve, which is the behavioral feature data.

7. The face recognition verification method according to claim 1, characterized in that, The method for verifying the matching between behavioral feature data and device data in step (4) is as follows: Standard feature data is calculated and generated based on equipment data; Calculate the goodness of fit between behavioral feature data and standard feature data; The authenticity of the verification operation of the user to be verified is determined based on the degree of fit.

8. A face recognition verification device, applied to the face recognition verification method according to any one of claims 1 to 7, characterized in that, include: The image acquisition module is used to acquire the facial image data of the user to be verified when the user initiates a verification operation. The node control module is used to set different acquisition nodes and generate a valid acquisition field on the display area set by the image acquisition module according to the acquisition nodes; The image sequence generation module is used to acquire consecutive facial image frames and generate an image sequence. The device status acquisition module is used to acquire the location data of the verification device during the verification operation. The feature extraction module is used to extract behavioral feature data of the user to be verified based on the image sequence; The verification module is used to verify the matching between behavioral feature data and device data, and to determine the authenticity of the verification operation of the user to be verified based on the verification results.

9. An electronic device, characterized in that, It includes a memory for storing computer program instructions and a processor for executing the program instructions, wherein when the computer program instructions are executed by the processor, the electronic device is triggered to perform the face recognition verification method as described in any one of claims 1 to 7.

Images