Method, device, equipment and storage medium for detecting face recognition attacks

Abstract

The present application discloses a detection method, device, equipment and storage medium for face recognition attacks, and relates to the field of information security technology. The method includes obtaining sensor data of an electronic device and at least two frames of images in a face acquisition preview control when face detection is turned on; determining change information used to characterize the motion posture of the electronic device in space based on the sensor data; determining the movement information of pixel points in the background area of ​​at least two frames of images based on at least two frames of images, and the background area is a non-face area in the image; determining whether there is a face recognition attack by matching the change information and the movement information. In this way, the detection and interception of face synthesis injection attacks are realized on the electronic device side, which is more in line with the synthetic attack scenario than the detection method through the blacklist background image, realizes targeted detection, effectively improves the detection accuracy, and can detect in real time when the electronic device turns on face detection, thereby improving the detection efficiency.

Description

Technical Field

[0001] The present application belongs to the field of information security technology, and in particular, relates to a method, device, equipment and storage medium for detecting face recognition attacks. Background Art

[0002] With the application of identity authentication in various fields, face recognition has become the main means of identity authentication. In the process of collecting face data, there are criminals who use synthetic attacks, injection attacks and other means to attack face recognition to affect face recognition results. For example, using artificial intelligence synthesis technology, the user's facial features are fitted into a realistic background image, and the corresponding actions are activated to generate a false biopsy action video, attacking the face recognition process and disrupting the face recognition results.

[0003] In the related art, since the background images used for fitting usually have homologous characteristics, an attack background database can be built on the server side based on the blacklist background images that have been attacked and have homologous characteristics, and then the collected biopsy action video can be attacked through the attack background database. However, the blacklist background images are derived from manual screening, which makes the data coverage in the attack background database incomplete and the update speed slow, resulting in low accuracy of attack detection and affecting the face recognition results. Summary of the invention

[0004] The embodiments of the present application provide a method, device, equipment and storage medium for detecting face recognition attacks, which can solve the problem of low accuracy of attack detection caused by incomplete data coverage and slow update speed in the attack background database in the related art.

[0005] In a first aspect, an embodiment of the present application provides a method for detecting a face recognition attack, which may include:

[0006] When face detection is enabled, obtain sensor data of the electronic device and at least two frames of images in the face acquisition preview control;

[0007] Determine change information of the electronic device according to the sensor data, where the change information is used to characterize the motion posture of the electronic device in space;

[0008] Based on at least two frames of images, determining movement information of pixel points in a background area in at least two frames of images, where the background area is a non-face area in the image;

[0009] By matching the change information and movement information, it is determined whether there is a face recognition attack.

[0010] In a second aspect, an embodiment of the present application provides a device for detecting a face recognition attack, which may include:

[0011] An acquisition module, used to acquire sensor data of the electronic device and at least two frames of images in the face acquisition preview control when face detection is detected to be turned on;

[0012] A determination module, used to determine change information of the electronic device according to the sensor data, where the change information is used to characterize the motion posture of the electronic device in space;

[0013] The determination module is further used to determine, based on at least two frames of images, movement information of pixels in a background area in at least two frames of images, where the background area is a non-face area in the image;

[0014] The determination module is also used to determine whether there is a face recognition attack based on the matching of the change information and the movement information.

[0015] In a third aspect, an embodiment of the present application provides a computer device, the computer device comprising: a processor and a memory storing computer program instructions;

[0016] When the processor executes the computer program instructions, it implements the method for detecting face recognition attacks as shown in the first aspect.

[0017] In a fourth aspect, an embodiment of the present application provides a computer storage medium having computer program instructions stored thereon. When the computer program instructions are executed by a processor, the method for detecting face recognition attacks as shown in the first aspect is implemented.

[0018] In a fifth aspect, an embodiment of the present application provides a chip, the chip including a processor and a communication interface, the communication interface and the processor are coupled, and the processor is used to run programs or instructions to implement the method for detecting face recognition attacks as shown in the first aspect.

[0019] In a sixth aspect, an embodiment of the present application provides a computer program product, which is stored in a storage medium and is executed by at least one processor to implement the method for detecting face recognition attacks as shown in the first aspect.

[0020] The detection method, device, equipment and storage medium of the face recognition attack of the embodiment of the present application can obtain the sensor data of the electronic device and at least two frames of images in the face collection preview control when the face detection is turned on, and determine the change information of the electronic device according to the sensor data, and the change information is used to characterize the motion posture of the electronic device in space; and, based on the at least two frames of images, determine the movement information of the pixel points in the background area of ​​at least two frames of images, and the background area is the non-face area in the image. Then, it is determined whether there is a face recognition attack by matching the change information and the movement information. In this way, by introducing the sensor data of the physical state change process of the electronic device to integrate the image collected by the face detection, the consistency detection is performed on the non-face area in the collected image, and the detection and interception of the face synthesis injection attack can be realized on the electronic device side. Compared with the detection method through the blacklist background image, it is more suitable for the synthetic attack scenario, realizes targeted detection, and effectively improves the detection accuracy. Moreover, the present application can screen whether there is a face recognition attack in real time on the electronic device side when the face detection is turned on, avoiding the problem of low accuracy of attack detection caused by the slow data update speed in the attack background database of the server, and improving the detection efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] In order to more clearly illustrate the technical solution of the embodiments of the present application, the following is a brief introduction to the drawings required for use in the embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0022] Figure 1 A flowchart of a method for detecting face recognition attacks is provided;

[0023] Figure 2 A flowchart of a method for detecting face recognition attacks provided in an embodiment of the present application;

[0024] Figure 3 A flowchart of a method for detecting face recognition attacks provided in an embodiment of the present application;

[0025] Figure 4 A schematic diagram of the flow of the interaction between the front end and the back end in a method for detecting face recognition attacks provided in an embodiment of the present application;

[0026] Figure 5 A schematic diagram of a flow chart of determining a second face detection strategy in a method for detecting a face recognition attack provided in an embodiment of the present application;

[0027] Figure 6 A flowchart of a method for detecting face recognition attacks provided in an embodiment of the present application;

[0028] Figure 7 It is a structural schematic diagram of a detection device for face recognition attacks provided by an embodiment of the present application;

[0029] Figure 8 It is a schematic diagram of the structure of a computer device provided by an embodiment of the present application. DETAILED DESCRIPTION

[0030] The features and exemplary embodiments of various aspects of the present application will be described in detail below. In order to make the purpose, technical solutions and advantages of the present application clearer, the present application will be further described in detail below in conjunction with the accompanying drawings and specific embodiments. It should be understood that the specific embodiments described herein are only intended to explain the present application, rather than to limit the present application. For those skilled in the art, the present application can be implemented without the need for some of these specific details. The following description of the embodiments is only to provide a better understanding of the present application by illustrating the examples of the present application.

[0031] It should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the statement "include..." do not exclude the existence of other identical elements in the process, method, article or device including the elements.

[0032] The acquisition, storage, use, and processing of data (including but not limited to the features and information in the text) in the technical solution of this application comply with the relevant provisions of national laws and regulations.

[0033] Liveness detection is a method used to determine the real physiological characteristics of an object in an identity authentication scenario. In face recognition applications, liveness detection can verify whether the user is a real living person through a combination of actions such as blinking, opening the mouth, shaking the head, and nodding, using technologies such as facial key point positioning and face tracking. It can effectively resist common attack methods such as photos, videos, face swapping, masks, occlusions, and screen reshoots, and determine the authenticity of the face, thereby helping users identify fraud and ensure user information security.

[0034] Face recognition has been the main means of identity verification for many years. With the upgrading of artificial intelligence and the use of Internet technology to carry out cyber attacks, steal information, extortion and fraud, attacks in the face collection process have gradually changed from low-tech means such as reshoots and masks to more complex and realistic means such as synthesis and injection. In the face detection process, criminals use synthetic attacks, injection attacks and other means to attack face recognition in order to affect the face recognition results. For example, in the liveness detection stage of the face detection process, the collected image including the face is fitted to the realistic background image provided by the criminals, and the corresponding actions are activated to generate a false liveness action video to attack the face recognition process.

[0035] Since these background images have homologous characteristics, that is, multiple attacked user videos are fitted with the same background image, so that the fitted images show consistent background features, which is particularly evident in attacks with preset synthesis and real-time synthesis injection. Therefore, in order to identify attacks with background images with homologous characteristics, an attack background database can be established based on existing attack data to detect potential homologous background images.

[0036] In related technologies, such as Figure 1 As shown, the process of detecting a face recognition attack may be that the electronic device initiates face detection, and in the liveness detection stage of the face detection process, collects a video including a face image and sends it to the server for further detection. When the server receives the video of the face image sent by the electronic device, it matches each frame of the stream image in the video with the blacklist background image in the attack background database to determine whether each frame of the stream image matches the blacklist background image in the attack background database. If the two match, the result is returned to the detection process for intercepting services related to face detection, such as payment services; if the two do not match, the server can provide services related to face detection.

[0037] However, the establishment of the aforementioned attack background database is only based on the manual regular screening of high-risk blacklist background images. Due to the low efficiency of manual screening, few data sources, slow update speed of the attack background database and single reference factor for screening decision, it is easy to cause the quality of blacklist background images to deteriorate, reduce the accuracy and efficiency of attack detection, and, in the case that the attack background database is not yet well established, it is easy to cause poor interception effect and affect the face recognition results.

[0038] In order to solve the problems in the related art, the embodiments of the present application take into account the characteristics of face detection of electronic devices, and associate the physical state changes of electronic devices such as rotation, acceleration, and deceleration that occur in the preparation stage of the face detection process of real users with the motion of the image collected by face detection. The consistency of the sensor data of the physical state change process of the electronic device and the background area in the collected image is used as a screening condition to efficiently discover face recognition attacks. At the same time, this method can also be used to establish an attack background database, that is, through the method provided by the embodiments of the present application, images of face recognition attacks in a large number of electronic devices can be obtained, and these images can be used as blacklist background images to quickly and efficiently supplement the attack background database to improve the update speed of the attack background database, and can also enrich reference elements and improve the quality of blacklist background images, thereby improving the accuracy of face recognition and attack interception efficiency through the supplemented attack background database, and ensuring the face recognition results.

[0039] Based on this, the following will combine the attached Figures 2 to 5 , the methods, devices, computer equipment and storage media of the embodiments of the present application are described in detail. It should be noted that these embodiments are not intended to limit the scope of the disclosure of the present application.

[0040] first, Figure 2 A schematic diagram of the structure of a system for detecting face recognition attacks provided in an embodiment of the present application.

[0041] like Figure 2 As shown, the detection method for face recognition attacks includes two processes, one is a front-end biopsy process performed by an electronic device, and the other is a back-end HACK process performed by a server.

[0042] The electronic device in the embodiment of the present application can be used to enable face detection and perform consistency detection to determine whether there is a face recognition attack.

[0043] Specifically, in an embodiment of the present application, taking the liveness detection stage of face detection as an example, when the electronic device is performing the liveness detection stage, it will prompt the user to adjust the position of the electronic device so that the face is moved to the coverage range of the face collection preview control displayed by the electronic device. The user holding the electronic device will cause the electronic device to undergo obvious deflection or acceleration changes, so that the electronic device collects the transformation information of the electronic device. When the user adjusts the position of the electronic device so that it can collect the user's complete face image, at least two frames of images in the face collection preview control can be collected. The at least two frames of images in the embodiment of the present application can be preview images.

[0044] Since the non-face areas in the images actually captured by the electronic device are obviously different from the blacklist background images used by criminals for attacks, the electronic device in the embodiment of the present application can associate the physical state changes such as rotation, acceleration and deceleration of the electronic device that occur to the real user in the preparation stage of the face detection process with the movement of the image captured by the face detection, and use the consistency between the sensor data of the physical state change process of the electronic device and the background area in the captured image as a screening condition to efficiently detect face recognition attacks.

[0045] Based on this, the embodiments of the present application can integrate the images collected by face detection by introducing sensor data of the physical state change process of the electronic device, and perform consistency detection on the non-face areas in the collected images, so as to effectively discover and identify potential face recognition attacks.

[0046] In this way, the electronic device can be used to collect sensor data of the electronic device when face detection is turned on, wherein the sensor data may include sensor data of at least one of the following stages: sensor data of the process in which the user holds the electronic device and moves the face to the designated detection area, and sensor data of the process in which at least two frames of images in the face collection preview control are collected. The sensor data in the embodiment of the present application may be data collected by multiple types of sensors in the electronic device, including but not limited to gyroscope sensors, acceleration sensors, gravity sensors, and magnetometer sensors.

[0047] Furthermore, the electronic device can also be used to determine, based on the aforementioned sensor data, change information used to characterize the motion posture of the electronic device in space, and based on at least two frames of images, determine the movement information of pixels in the background area of ​​at least two frames of images, where the background area is a non-face area in the image, and determine whether there is a face recognition attack by matching the change information and the movement information. In this way, screening the biopsy image using background consistency detection is more efficient than simple manual operation.

[0048] In addition, in some embodiments of the present application, the electronic device can also be used to send relevant detection data to the server so that the server can re-detect the face recognition attack based on the detection data and determine the face recognition result, or so that the server can update or establish an attack background database based on the detection data.

[0049] Specifically, in some embodiments, the electronic device may also send first detection data to the server when it is determined that there is no face recognition attack. The first detection data includes at least two frames of images in the face acquisition preview control during the face detection process. The first detection data is used by the server to detect face recognition attacks and determine face recognition results.

[0050] Based on this, the server in the embodiment of the present application can also match each frame image in the first detection data with the blacklist background image in the attack background database when receiving the first detection data sent by the electronic device to determine whether each frame image in the first detection data matches the blacklist background image in the attack background database. If the two match, the result is returned to the detection process to intercept services related to face detection such as payment services; if the two do not match, the server can provide services related to face detection such as payment services.

[0051] Therefore, the embodiment of the present application can use the consistency between the sensor data of the physical state change process of the electronic device and the background area in the collected image as a screening condition on the electronic device side, add a process for detecting face recognition attacks on the electronic device side, and when the change information and the movement information match, that is, when it is determined that there is no face recognition attack, it is detected again through the blacklist background image in the attack background database of the server. Through double detection at both ends, the accuracy of face recognition can be effectively improved and the face recognition result can be guaranteed.

[0052] In some other embodiments, the electronic device can also be used to send second detection data to the server when it is determined that a face recognition attack exists, the second detection data including a device identification of the electronic device, at least two frames of images in a face acquisition preview control during the face detection process, sensor data of the electronic device during the face detection process, and liveness detection motion feature parameters of the user in the liveness detection stage of the face detection process.

[0053] Based on this, when receiving the second detection data, the server can match at least two frames of images in the second detection data with the blacklist background images in the attack background database to determine whether each frame of image matches the blacklist background image in the attack background database. If the two match, the result is returned to the detection process to intercept services related to face detection, such as payment; if the two do not match, the server can provide services related to face detection, such as payment services. In addition, the server can also be used to update the blacklist background images in the attack background database based on the second detection data.

[0054] Here, it should be noted that, in the embodiment of the present application, the second detection data may also include an identification of the electronic device that provides the second detection data, sensor data of the electronic device during the face detection process, and liveness detection motion feature parameters of the user in the liveness detection stage of the face detection process. Based on this, the attack background database of the server may store, in addition to storing the blacklist background image, the identification of the electronic device that provides the second detection data, sensor data of the electronic device during the face detection process, and liveness detection motion feature parameters of the user in the liveness detection stage of the face detection process, so that the server can determine whether each frame of the image matches the blacklist background image in the attack background database based on the second detection data, so as to improve the accuracy by detecting whether the image matches the blacklist background image in the attack background database through multiple types of data.

[0055] Therefore, the above method can be used to obtain images of face recognition attacks in massive electronic devices, and these images can be quickly and efficiently added to the attack background database as blacklist background images to increase the update speed of the attack background database. It can also enrich reference elements and improve the quality of blacklist background images, thereby improving the accuracy of face recognition and attack interception efficiency through the supplemented attack background database, and ensuring the face recognition results.

[0056] In addition, in order to reduce the occurrence of false detections during the establishment and update process of the server attack background database, outlier analysis can be performed on the data before it is stored in the attack background database. After feature extraction of images of electronic devices that have been clearly identified as attack backgrounds, they are clustered with the blacklist background images currently stored in the attack background database. If the image is within the clustering range, it is stored in the attack background database. If the image exceeds the clustering range, the data can only be included in the attack background database after manual verification.

[0057] Based on this, the server can have a high confidence level in at least two frames of images acquired by the electronic device in the attack database established through outlier analysis and manual screening. On the one hand, the server can match the image sent by the electronic device with the blacklist background image to obtain a matching result, so as to guide the hack interception in the identification process based on the matching result; on the other hand, the server can utilize the electronic device and user data that are marked as risky based on the second detection data to adjust the thresholds and strategies in the live motion detection stage on the electronic device side. For example, the number of motion biopsies can be increased, the dazzling pupil biopsy process can be increased, etc.

[0058] Therefore, the embodiment of the present application provides an interception method for synthetic face injection attacks based on attack background detection, which can introduce the consistency judgment of the change information of the electronic device's pre-motion and the movement information of the background image in the collected image into the recognition process. Compared with only server detection, it is more in line with the synthetic injection scenario and more effective in improving data quality. In addition, unsupervised learning methods can be used to clean outlier data, which can improve the quality of data in the attack background database, achieve targeted detection, and effectively improve detection accuracy. At the same time, the device source of the successfully stored data is used for positioning, and for these potential risk devices, the corresponding face biopsy threshold is appropriately increased to control the risk, thereby improving detection efficiency.

[0059] It should be noted that the method for detecting face recognition attacks provided in the embodiments of the present application can be applied to scenarios requiring identity verification, such as financial payment, access control, account login authentication, and other scenarios that require verification of the user's true identity.

[0060] Figure 3 A flowchart of a method for detecting face recognition attacks provided in an embodiment of the present application.

[0061] like Figure 3 As shown, the method for detecting face recognition attacks can be applied to electronic devices, and the detection method can specifically include the following steps:

[0062] Step 310, when face detection is turned on, obtain sensor data of the electronic device and at least two frames of images in the face acquisition preview control; Step 320, determine change information of the electronic device based on the sensor data, and the change information is used to characterize the motion posture of the electronic device in space; Step 330, based on at least two frames of images, determine movement information of pixel points in the background area of ​​at least two frames of images, and the background area is the non-face area in the image; Step 340, determine whether there is a face recognition attack through the matching of the change information and the movement information.

[0063] In this way, by introducing sensor data of the physical state change process of the electronic device, integrating the images collected by face detection, and performing consistency detection on the non-face areas in the collected images, detection and interception of face synthesis injection attacks can be achieved on the electronic device side. Compared with the detection method through blacklist background images, it is more in line with the synthetic attack scenario, realizes targeted detection, and effectively improves the detection accuracy. Moreover, the detection method provided by the present application can screen in real time on the electronic device side whether there is a face recognition attack when face detection is turned on, avoiding the problem of low attack detection accuracy due to slow data update speed in the attack background database of the server, thereby improving detection efficiency.

[0064] The above steps are described in detail below, as shown below.

[0065] First, regarding step 310 , in some embodiments of the present application, before step 310 , the detection method may further include step 4101 and step 4202 .

[0066] Step 4101, receiving a first input from a user confirming to enable a face detection control in a first interface.

[0067] Step 4202, in response to the first input, start face detection.

[0068] Based on this, step 310 may specifically include step 3101 and step 3102 .

[0069] Step 3101, when face detection is turned on or liveness detection (referred to as liveness detection) in face detection is turned on, sensor data is collected through at least two sensors in the electronic device. The sensor data in the embodiment of the present application may include data in the face detection process. Specifically, the sensor data may include sensor data in at least one of the following stages: sensor data in the process of the user holding the electronic device to move the face to the designated detection area, and sensor data in the process of collecting at least two frames of images in the face collection preview control. The sensor data in the embodiment of the present application may be data collected by multiple types of sensors in the electronic device, including but not limited to gyroscope sensors, acceleration sensors, gravity sensors, and magnetometer sensors.

[0070] Take the example of multiple types of sensors, gyroscope sensors and acceleration sensors. If the sensor is a gyroscope sensor, it can be used to measure the acceleration data of the electronic device in three-dimensional space to sense the motion state of the electronic device. Based on this, the sensor data can be the angular velocity of the gyroscope sensor. If the sensor is an acceleration sensor, it can be used to measure the acceleration of the electronic device. In this way, the motion posture of the electronic device in space can be determined through these sensor data.

[0071] Therefore, the motion posture of the electronic device can be determined through sensor data, providing rich data support, so as to accurately identify the motion state of the device.

[0072] Step 3102: When the face collection preview control in the first interface includes the user's face, at least two frames of images in the face collection preview control are collected according to the face detection strategy. The face detection strategy may include at least one of the following strategies: the first face detection strategy in the embodiment of the present application, and the second face detection strategy in the embodiment of the present application.

[0073] It should be noted that the trigger condition for collecting sensor data is to enable face detection or enable liveness detection in face detection until at least two frames of images are collected to determine whether there is a face recognition attack. Figure 4 As shown, the current face detection process is to enable face detection, collect at least two frames of images, and send at least two frames of images to the server. In the embodiment of the present application, after enabling face detection, a new process of collecting sensor data is added, and the collection process is imperceptible to the user, that is, silent collection.

[0074] Secondly, regarding step 320, in some embodiments of the present application, the sensor data includes gyroscope sensor data and acceleration sensor data within N time periods, where N is an integer greater than 1. Based on this, step 320 may specifically include step 3201 and step 3202.

[0075] Step 3201, performing data fusion processing on the gyro sensor data and the acceleration sensor data in each of the N time periods to obtain the attitude angle of the electronic device in each time period, where N is an integer greater than 1. Step 3202, generating change information of the electronic device according to the attitude angle in each of the N time periods.

[0076] Exemplarily, still taking the above example as an example, based on the gyroscope sensor data and the acceleration sensor data, the motion vector is calculated to obtain the transformation information of the electronic device in each time period, or the transformation information in each time period of N time periods can be spliced ​​to obtain the transformation information of the electronic device in N time periods.

[0077] Therefore, the embodiment of the present application starts from an attack background that is difficult to forge, introduces multi-sensor data integration and background consistency verification of the physical state change process of electronic devices, constructs an efficient background detection attack interception method, and comprehensively judges the attack risk in each face transaction process, which is an important supplement and improvement to the existing solutions.

[0078] In some embodiments of the present application, the gyroscope sensor data in the embodiments of the present application includes a first angular velocity, and the acceleration sensor data includes a second angular velocity. The second angular velocity is an angular velocity obtained by processing the acceleration data measured by the acceleration sensor with an inverse tangent function. Based on this, step 3201 may specifically include step 32011:

[0079] The first angular velocity and the second angular velocity are weighted and summed by using a first weighting coefficient corresponding to the gyroscope sensor data and a second weighting coefficient corresponding to the acceleration sensor data to obtain the attitude angle of the electronic device in each time period.

[0080] For example, since the angular velocity given by the gyro sensor may have measurement errors, noise and drift, after integration operations, a cumulative error will be formed. This error will become larger and larger over time, and eventually lead to a deviation that is too large to be used. The acceleration sensor is also easily interfered by high-frequency noise, has a slow dynamic response, and also has the problem of cumulative drift error. To eliminate the error, the embodiment of the present application fuses the gyro sensor data collected by the gyro sensor and the acceleration sensor data collected by the acceleration sensor. Taking complementary filtering as an example, the attitude angle can be calculated by the following formula (1):

[0081] Angle=0.98*(Angle+Gyro*dt)+0.02*Acc (1)

[0082] Among them, Angle is the attitude angle obtained after complementary filtering, Gyro is the first angular velocity obtained by the gyroscope sensor, Acc is the second angle of the acceleration sensor after the unit is converted through the inverse tangent function atan(), dt is the operation period of the filter, 0.98 is the first weighting coefficient α and 0.02 is the second weighting coefficient (1-α). Here, the calculation of the first weighting coefficient α depends on the acquisition interval (10ms) and the time constant (490ms) to determine, among which the time constant needs to be determined through experiments to minimize the fusion error.

[0083] Based on this, the calculation of the attitude angle has already begun during the acquisition cycle, and the calculation results of each acquisition point are temporarily stored and summarized before entering the motion biopsy stage.

[0084] It should be noted that the attitude angle of the electronic device in the embodiment of the present application refers to the direction and position of the electronic device in three-dimensional space, which is measured by sensors such as gyroscopes and accelerometers. The attitude angle may include at least a pitch angle (Pitch), a yaw angle (Yaw) and a roll angle (Roll), which describe the specific attitude of the electronic device in space.

[0085] Furthermore, regarding step 330, in some embodiments of the present application, step 330 may specifically include step 3301 and step 3302.

[0086] Step 3301, based on at least two frames of images, determine the motion information and correlation information of the pixel points in the background area of ​​two adjacent frames of images in at least two frames of images in the time domain, the motion information is used to represent the degree of motion of the pixel points in the two adjacent frames of images, and the correlation information is used to represent the moving direction and distance of the pixel points within a reference time period, and the reference time period is the time interval for obtaining two adjacent frames of images.

[0087] Step 3302: Determine movement information of pixels in the background area of ​​at least two frames of images based on the motion information and the correlation information.

[0088] Further, the motion information may be determined by the following steps. Based on this, the motion information in the embodiment of the present application includes the motion vector of the pixel point in two adjacent frames of images. The above step 3301 may specifically include steps 33011 to 33013.

[0089] Step 33011, based on at least two frames of images, determine the grayscale value of the pixel points in the background area of ​​each frame of at least two frames of images.

[0090] Step 33012, constructing a residual function according to the grayscale values ​​of the pixels in the background area of ​​two adjacent frames of images in at least two frames of images.

[0091] Step 33013, based on the residual function, calculate the motion vector of the pixel point in two adjacent frame images through the least squares method and reference constraints.

[0092] For example, the motion vector of a pixel point in two adjacent frames of images, i.e., the pixel offset, is calculated, and the basic calculation formula flow is as follows:

[0093] Consider a pixel I(x,y,t), which moves a distance (dx,dy) to the next frame of image within a time interval of dt. Because it is the same pixel point, the tracked pixel target remains basically unchanged between consecutive frames, that is, it satisfies the following formula (2):

[0094] I(x,y,t)=I(x+dx,y+dy,t+dt) (2)

[0095] After Taylor expansion, we can get the following formula (3):

[0096]

[0097] The equation can be simplified and expressed as follows by formula (4):

[0098] I x u+I y v+I t =0 (4)

[0099] Among them, the optical flow equations are the motion vectors along the X-axis and Y-axis, such as the velocity vectors u and v respectively. Ix, Iy, It can be obtained by pixel convolution, and the convolution kernels are At this time, additional reference constraints are introduced to obtain the values ​​of u and v. Assuming that adjacent pixels have the same motion vector, the pixels in the n×n area are substituted into the equation to solve it. Of course, the reference constraint can also be any one of matching, energy, phase, neurokinematics, or sparse or dense optical flow. Based on the optical flow equation, a reference constraint is added to solve the motion vector. The overdetermined equation is obtained by calculating the nine pixels in the 3×3 window as a whole. The optimal value is obtained using the least squares method, and then the motion change of the motion vector of the pixel points in the area in two adjacent frames is obtained. The values ​​of u and v can be calculated by the following formula (5).

[0100]

[0101] This process is similar to Harris corner detection, and the overall moving direction of the pixels is evaluated by the mean of the corner motion direction.

[0102] Further, the correlation information can be determined through the following steps. Based on this, the correlation information in the embodiment of the present application includes the movement vector of the pixel point in two adjacent frames of images, and the above step 3301 can specifically include steps 33014 to 33016.

[0103] Step 33014, according to the moving direction and distance of the pixel point in two adjacent frame images, respectively determine the pixel matching blocks in the two adjacent frame images, the two adjacent frame images include the first frame image and the second frame image, and the pixel matching blocks include the first pixel matching block of the first frame image and the second pixel matching block of the second frame image.

[0104] For example, the moving direction of pixel point A in the first frame image and the second frame image is toward the right, and the distance is one pixel point. At this time, the pixel matching blocks corresponding to the pixel point can be determined on the first frame image and the second frame image respectively, that is, the first pixel matching block of the first frame image is A', and the second pixel matching block of the second frame image is A".

[0105] Step 33015, calculating the instantaneous motion vector of the image of two adjacent frames within the reference time period according to the matching results of the first pixel matching block and the second pixel matching block.

[0106] Exemplarily, the moving direction 1 and distance 1 from pixel point A to the first pixel matching block A' are calculated, and the moving direction 2 and distance 2 from pixel point A to the second pixel matching block A" are calculated. The moving direction 1 and distance 1, as well as the moving direction 2 and distance 2, i.e., the instantaneous motion vector of the image, are superimposed to obtain the instantaneous motion vector of the image of two adjacent frames of images within the reference time period.

[0107] Step 33016, superimpose the instantaneous motion vectors of the images of each two adjacent frames of at least two frames of images in a reference time period corresponding to each two adjacent frames of images to obtain the motion vector of the pixel point in the two adjacent frames of images.

[0108] Exemplarily, still taking the above example, the instantaneous motion vectors of at least two groups of two adjacent frames of images within a reference time period are superimposed to obtain the motion vectors of the pixels in the two adjacent frames of images.

[0109] Then, involving step 340, in some embodiments of the present application, the change information includes the motion vectors of the pixels of the background area in two adjacent frames of at least two frames, and the movement information includes the motion vectors of the pixels of the background area in two adjacent frames. Based on this, step 340 may specifically include step 3401 and step 3402.

[0110] Step 3401, based on the motion vector and the movement vector, determine the angle between the electronic device and the displacement vector of at least two frames of images.

[0111] Step 3402, determine whether there is a face recognition attack based on the angle between the displacement vectors.

[0112] For example, since it is a relatively long-term status assessment, the displacement direction of the electronic device needs to be cumulatively calculated based on the sensor readings within the acquisition period, and the projection vectors of the inclination angle and acceleration direction in the x- and y-axis planes are obtained to finally determine it, and a consistency judgment is made based on whether the angle between the image and the displacement vector of the device is within the tolerance range.

[0113] It should be noted that in order to save computing resources, only part of the images and the collected data of the equipment are often selected for analysis in at least two frames of images, or consistency calculations are performed at fixed intervals as sampling points on a sparser scale. After the calculation is completed, the data is discarded and only the calculation results are retained. The interval can be dynamically adjusted according to business requirements.

[0114] Here, the consistency judgment method of background change and sensor data provided by the embodiment of the present application can be based on optical flow measurement. The optical flow method uses the change of pixels in the image sequence in the time domain and the correlation between adjacent frames to find the corresponding relationship between the previous frame and the current frame, thereby calculating the motion information of the object between adjacent frames.

[0115] Furthermore, step 3402 may specifically include step 34021 or step 34022.

[0116] Step 34021, when the displacement vector angle is smaller than the reference displacement vector angle, it is determined that there is no face recognition attack.

[0117] Step 34022, when the displacement vector angle is greater than or equal to the reference displacement vector angle, it is determined that there is a face recognition attack.

[0118] Exemplarily, when the displacement vector angle is less than 10° of the reference displacement vector angle, it is determined that there is no face recognition attack. When the displacement vector angle is greater than or equal to 10° of the reference displacement vector angle, it is determined that there is a face recognition attack.

[0119] It should be noted that in actual use, in order to enhance the credibility of the attack background as an interception factor, text information in the background image, shooting scene judgment, shooting equipment differences and other information can also be introduced to determine whether there is a face recognition attack.

[0120] Therefore, based on the angle of the displacement vector, combined with the text information in the background image, the shooting scene judgment, the difference in shooting equipment and other information, multi-dimensional factors can be comprehensively considered to determine whether there is a face recognition attack, thereby improving the accuracy of determining whether there is a face recognition attack.

[0121] In addition, in some embodiments of the embodiments of the present application, after step 340, the detection method may further include step 4201, sending first detection data to the server when it is determined that there is no face recognition attack, the first detection data including at least two frames of images in the face acquisition preview control during the face detection process, and the first detection data is used by the server to detect face recognition attacks and determine face recognition results.

[0122] Here, the server can also match each frame image in the first detection data with the blacklist background image in the attack background database when receiving the first detection data sent by the electronic device to determine whether each frame image in the first detection data matches the blacklist background image in the attack background database. If the two match, the result is returned to the detection process to intercept services related to face detection, such as payment services; if the two do not match, the server can provide services related to face detection, such as payment services.

[0123] Therefore, the embodiment of the present application can use the consistency between the sensor data of the physical state change process of the electronic device and the background area in the collected image as a screening condition on the electronic device side, add a process for detecting face recognition attacks on the electronic device side, and when the change information and the movement information match, that is, when it is determined that there is no face recognition attack, it is detected again through the blacklist background image in the attack background database of the server. Through double detection at both ends, the accuracy of face recognition can be effectively improved and the face recognition result can be guaranteed.

[0124] In some embodiments of the present application, after step 340, the detection method may further include step 4202, where, when it is determined that a face recognition attack exists, at least one of the following operations is performed:

[0125] Sending second detection data to a server, not responding to face detection and sending an alarm message to the server, counting the number of times that face recognition attacks are determined to exist on the electronic device and disabling the face recognition function if the number is greater than or equal to a preset number;

[0126] The second detection data includes a device identifier of the electronic device, at least two frames of images in the face collection preview control during the face detection process, sensor data of the electronic device during the face detection process, and liveness detection action feature parameters of the user during the liveness detection phase of the face detection process. The second detection data is used by the server to update the attack data in the attack background database, and the attack background database is used by the server to detect face recognition attacks.

[0127] The warning message is used to indicate that the electronic device has been attacked by facial recognition.

[0128] Exemplarily, the electronic device sends the second detection data to the server. Based on this, the operation in the embodiment of the present application is to send the second detection data to the server. The second detection data is also used by the server to verify the second detection data through an outlier analysis algorithm. If the verification is passed, the attack data in the attack background database is updated according to the second detection data.

[0129] Exemplarily, upon receiving the second detection data, the server may match at least two frames of images in the second detection data with the blacklist background images in the attack background database to determine whether each frame of image matches the blacklist background image in the attack background database. If the two match, the result is returned to the detection process for intercepting services related to face detection, such as payment; if the two do not match, the server may provide services related to face detection, such as payment services. Furthermore, the server may also be used to update the blacklist background images in the attack background database based on the second detection data.

[0130] In addition, it should be noted that in the embodiment of the present application, the second detection data may also include an identification of the electronic device that provides the second detection data, sensor data of the electronic device during the face detection process, and liveness detection motion feature parameters of the user in the liveness detection stage of the face detection process. Based on this, the attack background database of the server, in addition to storing the blacklist background image, may also store the identification of the electronic device that provides the second detection data, sensor data of the electronic device during the face detection process, and liveness detection motion feature parameters of the user in the liveness detection stage of the face detection process, so that the server can determine whether each frame of the image matches the blacklist background image in the attack background database based on the second detection data, so as to improve accuracy by detecting whether the image matches the blacklist background image in the attack background database through multiple types of data.

[0131] Therefore, the above method can be used to obtain images of face recognition attacks in massive electronic devices, and these images can be quickly and efficiently added to the attack background database as blacklist background images to increase the update speed of the attack background database. It can also enrich reference elements and improve the quality of blacklist background images, thereby improving the accuracy of face recognition and attack interception efficiency through the supplemented attack background database, and ensuring the face recognition results.

[0132] It should be noted that in order to reduce the occurrence of false detections during the establishment and update process of the server attack background database, outlier analysis can also be performed on the data before it is stored in the attack background database. After feature extraction of images of electronic devices that have been clearly identified as attack backgrounds, they are clustered with the blacklist background images currently stored in the attack background database. If the image is within the clustering range, it is stored in the attack background database. If the image exceeds the clustering range, the data can only be included in the attack background database after manual verification.

[0133] Based on this, you can refer to Figure 5 The 5 embodiments of the present application provide the following steps to construct an attack background database.

[0134] Specifically, Figure 5 As shown, S5011, manual screening of risk control raw data. Among them, the server can obtain potential attack data in the long-term detection process of the full-volume scenario. The initial construction of the attack background database relies on the manually screened risk control raw data, which can be clearly defined as attack data. In order to ensure that the attack data is relevant, the attack data can be analyzed for outliers, and high-quality cleaning can be completed through successive iterations to obtain a blacklist background image, and its blacklist background image is added to the attack background database.

[0135] S5012, receiving second detection data provided by the electronic device.

[0136] S502: Perform an outlier check on the second detection data.

[0137] Specifically, if the image in the second detection data is clustered with the blacklist background image currently stored in the attack background database and the clustering result is within the clustering range, it indicates that the outlier check is unsuccessful, and S503 is executed, that is, the image in the second detection data is stored in the attack background database; conversely, if the image in the second detection data is clustered with the blacklist background image currently stored in the attack background database and the clustering result is not within the clustering range, it indicates that the outlier check is successful, and S5011 can be executed for manual screening to determine whether to expand the cluster boundary, and the feature data of the outlier check of the data in the attack background database is mainly based on the biopsy image and the second detection data after data enhancement. The enhancement of the image involving the biopsy in the second detection data can focus more on the traces of the synthetic boundary to enhance the detection accuracy.

[0138] Therefore, the number of false detections can be reduced during the establishment and update of the server attack background database. On the basis of these data, the data from the second detection data source can be gradually added as supplementary data to the attack background database, and the outlier data can be cleaned using unsupervised learning methods, which can improve the quality of the data in the attack background database, achieve targeted detection, and effectively improve detection accuracy. At the same time, the device source of the successfully stored data is used for positioning, and for these potential risk devices, the corresponding face biopsy threshold is appropriately raised to control the risk, thereby improving detection efficiency.

[0139] It should be noted that since the second detection data includes multi-dimensional electronic equipment hardware and software, operating system and other data, the purpose of image enhancement here is to make the common synthesis traces of the blacklist background images in the attack background database more obvious, and the use of the second detection data is to make the outlier analysis of feature data closer to the human screening process. The outlier verification method can be performed using machine learning algorithms such as OC-SVM.

[0140] In addition, in some embodiments of the embodiments of the present application, the electronic device includes a first face detection strategy, and at least two frames of images are acquired according to the first face detection strategy. Based on this, the detection method may also include steps 4301 and 4302.

[0141] Step 4301, receiving a second face detection strategy sent by the server. The second face detection strategy is sent by the server when it is determined that there is a face recognition attack on the electronic device. The detection difficulty level of the second face detection strategy is higher than the detection difficulty level of the first face detection strategy.

[0142] Step 4302: When it is detected that face detection is turned on next time, at least two frames of images in the face acquisition preview control are obtained according to the second face detection strategy.

[0143] For example, based on the reliable construction of the attack background database, the electronic device and user data can be used to guide the front-end biopsy service, such as whether increasing will reduce the number of action biopsies, whether it is necessary to increase the dazzling pupil biopsy process, etc. Before the face service actually occurs, the server is requested to respond to the first face detection strategy corresponding to the electronic device and the user. If the electronic device or the user hits the data in the attack background database, a higher-level second face detection strategy is issued, in which the threshold and biopsy action sequence set by the second face detection strategy, as well as the addition of biopsies of other modalities, have a higher priority or complexity than the content involved in the first face detection strategy, so as to put forward higher cooperation requirements to the user during the face detection process. In addition, in the subsequent recognition process, it is also possible to count whether the user has hit the attack background multiple times and has been intercepted to further disable the recognition service.

[0144] It should be noted that the detection difficulty in the embodiments of the present application can be reflected in the harshness of the acquisition environment, the number of images collected, and the complexity of the detection process. Then, higher-level thresholds and biopsy action sequences are issued, and biopsy of other modalities is added, which puts forward higher cooperation requirements on users.

[0145] For ease of understanding, the face recognition detection process including the face recognition attack detection method applied to an electronic device in an embodiment of the present application is described below.

[0146] Figure 6 A flowchart of an example of a face detection process provided in an embodiment of the present application is shown as follows: Figure 6 As shown, the face detection process includes steps f1 to f10. In step f1, sensor data of the electronic device is collected. For details, please refer to the relevant description of step S310 in the above embodiment, which will not be repeated here.

[0147] Specifically, in application scenarios where identity authentication is the purpose of identification, such as receiving the first operation such as login and payment, and triggering the specific function of liveness detection, the detection task for the electronic device environment has begun. During this process, the user picks up the phone and adjusts the position in a continuous manner until the face in the preview screen is in the target detection area and remains there. During this process, sensor data of the electronic device, such as gyroscope data and accelerometer data, is collected in real time.

[0148] In step f2, the change information of the electronic device is determined according to the sensor data. For details, please refer to the relevant description of step S320 in the above embodiment, which will not be repeated here.

[0149] In step f3, the position detection of the face area in the image frame is performed. If the position detection passes, step f4 is executed; if the position detection fails, it is determined that there is a face recognition attack, the face recognition is intercepted, and the process ends.

[0150] In step f4, the motion trajectory of the face area is obtained.

[0151] In step f5, the user is prompted for actions that the live action detection requires.

[0152] In step f6, the action consistency judgment is performed, that is, whether the action of the face in the captured image is consistent with the prompted action. If the consistency judgment is passed, step f7 is executed; if the consistency judgment is not passed, it is determined that there is a face recognition attack, the face recognition is intercepted, and the process ends.

[0153] In step f7, dazzle detection is performed.

[0154] In step f8, at least two frames of images may be collected during the process from f3 to f7 to determine the movement information. The specific content of step f8 may refer to the specific content involved in step 330 in the above embodiment, which will not be repeated here. Here, at least two frames of images may be acquired at intervals of 5 preview frames.

[0155] In step f9, the matching status of the change information and the movement information is determined. The specific content of step f9 can be found in the specific content involved in step 340 in the above embodiment, which will not be repeated here.

[0156] In step f10, background detection is performed, which can be implemented as a back-end hack detection.

[0157] The above steps f1 to f9 may be performed by the electronic device, and step f10 may be performed by the server.

[0158] Therefore, in an embodiment of the present application, the electronic device can be used to collect sensor data of the electronic device when face detection is turned on, wherein the sensor data may include sensor data of at least one of the following stages: sensor data in the process of a user holding the electronic device and moving the face to a designated detection area, and sensor data in the process of collecting at least two frames of images in the face collection preview control. The electronic device is used to determine the change information used to characterize the motion posture of the electronic device in space based on the aforementioned sensor data, and to determine the movement information of the pixel points in the background area of ​​at least two frames of images based on at least two frames of images, the background area being the non-face area in the image, and to determine whether there is a face recognition attack by matching the change information and the movement information. In this way, screening the biopsy image using background consistency detection is more efficient than simple manual operation.

[0159] The present application also provides a detection device for face recognition attacks, specifically combining Figure 7 Provide detailed explanation.

[0160] Figure 7 It is a structural diagram of a detection device for face recognition attacks provided by an embodiment of the present application.

[0161] In some embodiments of the present application, Figure 7 The device for detecting facial recognition attacks shown can be arranged in the computer device provided in the embodiment of the present application.

[0162] like Figure 7 As shown, the detection device 70 for face recognition attacks may specifically include:

[0163] An acquisition module 701 is used to acquire sensor data of the electronic device and at least two frames of images in the face acquisition preview control when face detection is detected to be turned on;

[0164] A determination module 702 is used to determine change information of the electronic device according to the sensor data, where the change information is used to characterize the motion posture of the electronic device in space;

[0165] The determination module 702 is further configured to determine, based on at least two frames of images, movement information of pixels in a background area in at least two frames of images, where the background area is a non-face area in the image;

[0166] The determination module 702 is also used to determine whether there is a face recognition attack based on the matching of the change information and the movement information.

[0167] In this way, the detection device for face recognition attacks in the embodiment of the present application can integrate the images collected by face detection by introducing sensor data of the physical state change process of the electronic device, and perform consistency detection on the non-face area in the collected image, so as to realize the detection and interception of face synthesis injection attacks on the electronic device side. Compared with the detection method through blacklist background images, it is more in line with the synthetic attack scenario, realizes targeted detection, and effectively improves the detection accuracy. Moreover, the detection method provided by the present application can screen whether there is a face recognition attack in real time on the electronic device side when face detection is turned on, avoiding the problem of low accuracy of attack detection due to slow data update speed in the attack background database of the server, thereby improving detection efficiency.

[0168] The following is a detailed description of the face recognition attack detection device 70 in the embodiments of the present application.

[0169] In some embodiments of the present application, the determination module 702 may be specifically configured to, when the sensor data includes gyroscope sensor data and acceleration sensor data in N time periods, where N is an integer greater than 1, perform data fusion processing on the gyroscope sensor data and acceleration sensor data in each time period of the N time periods to obtain the attitude angle of the electronic device in each time period;

[0170] The change information of the electronic device is generated according to the posture angle in each of the N time periods.

[0171] In some embodiments of the present application, the determination module 702 can be specifically used to, when the gyroscope sensor data includes a first angular velocity, the acceleration sensor data includes a second angular velocity, and the second angular velocity is the angular velocity obtained by processing the acceleration data measured by the acceleration sensor with an inverse tangent function, perform weighted summation of the first angular velocity and the second angular velocity using a first weighting coefficient corresponding to the gyroscope sensor data and a second weighting coefficient corresponding to the acceleration sensor data to obtain the attitude angle of the electronic device in each time period.

[0172] In some embodiments of the present application, the determination module 702 may be specifically used to determine, based on at least two frames of images, motion information and correlation information of pixels in the background area of ​​two adjacent frames of images in at least two frames of images in the time domain, where the motion information is used to indicate the degree of motion of the pixels in the two adjacent frames of images, and the correlation information is used to indicate the moving direction and distance of the pixels in a reference time period, where the reference time period is the time interval for obtaining the two adjacent frames of images;

[0173] Based on the motion information and the correlation information, movement information of pixel points in the background area of ​​at least two frames of images is determined.

[0174] In some embodiments of the present application, the determination module 702 may be specifically configured to determine, based on at least two frames of images, a grayscale value of a pixel in a background area in each of the at least two frames of images when the motion information includes a motion vector of a pixel in two adjacent frames of images;

[0175] Constructing a residual function according to the grayscale values ​​of the pixels in the background area of ​​two adjacent frames of images in at least two frames of images;

[0176] Based on the residual function, the motion vector of the pixel in two adjacent frames is calculated through the least squares method and reference constraints.

[0177] In some embodiments of the present application, the determination module 702 may be specifically configured to, when the correlation information includes a movement vector of a pixel point in two adjacent frames of images, determine pixel matching blocks in the two adjacent frames of images according to a movement direction and a distance of the pixel point in the two adjacent frames of images, respectively, where the two adjacent frames of images include a first frame of image and a second frame of image, and the pixel matching blocks include a first pixel matching block of the first frame of image and a second pixel matching block of the second frame of image;

[0178] Calculating the instantaneous motion vector of two adjacent frames of images within a reference time period according to the matching results of the first pixel matching block and the second pixel matching block;

[0179] The instantaneous motion vectors of the images of each two adjacent frames of images in at least two frames of images in a reference time period corresponding to each two adjacent frames of images are superimposed to obtain the motion vectors of the pixel points in the two adjacent frames of images.

[0180] In some embodiments of the present application, the detection device 70 for face recognition attacks may further include a receiving module and an opening module; wherein,

[0181] A receiving module, used to receive a first input from a user confirming to start a face detection control in the first interface;

[0182] An enabling module, configured to enable face detection in response to a first input;

[0183] The acquisition module 701 may be specifically configured to collect sensor data through at least two sensors in the electronic device when face detection is enabled;

[0184] And, when the face collection preview control in the first interface includes the user's face, at least two frames of images in the face collection preview control are collected according to the face detection strategy.

[0185] In some embodiments of the present application, the determination module 702 may be specifically configured to determine the angle between the displacement vectors of the electronic device and the at least two frames of images based on the motion vector and the movement vector, when the change information includes the motion vector of the pixel point of the background area in two adjacent frames of images in the at least two frames of images, and the movement information includes the movement vector of the pixel point of the background area in two adjacent frames of images;

[0186] Determine whether there is a face recognition attack based on the angle between the displacement vectors.

[0187] In some embodiments of the present application, the determination module 702 may be specifically configured to determine that there is no face recognition attack when the displacement vector angle is smaller than the reference displacement vector angle.

[0188] When the displacement vector angle is greater than or equal to the reference displacement vector angle, it is determined that a face recognition attack exists.

[0189] In some embodiments of the present application, the determination module 702 can be specifically used to send first detection data to the server when it is determined that there is no face recognition attack. The first detection data includes at least two frames of images in the face acquisition preview control during the face detection process. The first detection data is used by the server to detect face recognition attacks and determine face recognition results.

[0190] In some embodiments of the present application, the face recognition attack detection device 70 may further include an execution module; wherein, when it is determined that a face recognition attack exists, at least one of the following operations is performed:

[0191] Sending second detection data to a server, not responding to face detection and sending an alarm message to the server, counting the number of times that face recognition attacks are determined to exist on the electronic device and disabling the face recognition function if the number is greater than or equal to a preset number;

[0192] The second detection data includes a device identifier of the electronic device, at least two frames of images in the face collection preview control during the face detection process, sensor data of the electronic device during the face detection process, and liveness detection action feature parameters of the user during the liveness detection phase of the face detection process. The second detection data is used by the server to update the attack data in the attack background database, and the attack background database is used by the server to detect face recognition attacks.

[0193] The warning message is used to indicate that the electronic device has been attacked by facial recognition.

[0194] In some embodiments of the present application, the operation is to send second detection data to a server, and the second detection data is also used by the server to verify the second detection data through an outlier analysis algorithm. If the verification is passed, the attack data in the attack background database is updated according to the second detection data.

[0195] In some embodiments of the present application, the detection device 70 for face recognition attacks may further include a receiving module, which is used to receive a second face detection strategy sent by the server when the electronic device includes a first face detection strategy and at least two frames of images are obtained according to the first face detection strategy. The second face detection strategy is sent by the server when it is determined that the electronic device has a face recognition attack, and the detection difficulty level of the second face detection strategy is higher than the detection difficulty of the first face detection strategy;

[0196] The acquisition module 701 is further configured to acquire at least two frames of images in the face acquisition preview control according to the second face detection strategy when it is detected that face detection is turned on next time.

[0197] Based on the same inventive concept, the present application also provides a computer device. Figure 8 Provide detailed explanation.

[0198] Figure 8 It is a schematic diagram of the structure of a computer device provided by an embodiment of the present application.

[0199] like Figure 8 As shown, the computer device may include at least one of the following involved in the embodiments of the present application: an electronic device, a server. The computer device may include a processor 801 and a memory 802 storing computer program instructions.

[0200] Specifically, the processor 801 may include a central processing unit (CPU), or an application specific integrated circuit (Application Specific Integrated Circuit (ASTC), or may be configured to implement one or more integrated circuits of the embodiments of the present application.

[0201] The memory 802 may include a large capacity memory for data or instructions. By way of example and not limitation, the memory 802 may include a hard disk drive (HDD), a floppy disk drive, a flash memory, an optical disk, a magneto-optical disk, a magnetic tape, or a universal serial bus (USB) drive or a combination of two or more of these. In appropriate cases, the memory 802 may include a removable or non-removable (or fixed) medium. In appropriate cases, the memory 802 may be inside or outside the integrated gateway disaster recovery device. In a specific embodiment, the memory 802 is a non-volatile solid-state memory. In a specific embodiment, the memory 802 includes a solid-state storage (ROM). In appropriate cases, the ROM may be a mask-programmed ROM, a programmable ROM (PROM), an erasable PROM (EPROM), an electrically erasable PROM (EEPROM), an electrically rewritable ROM (EAROM) or a flash memory or a combination of two or more of these.

[0202] The processor 801 implements any one of the methods for detecting face recognition attacks in the above embodiments by reading and executing computer program instructions stored in the memory 802 .

[0203] In one example, the computer device may further include a communication interface 803 and a bus 810. Figure 8 As shown, the processor 801, the memory 802, and the communication interface 803 are connected via a bus 810 and communicate with each other.

[0204] The communication interface 803 is mainly used to implement communication between various modules, devices, units and / or equipment in the embodiments of the present application.

[0205] Bus 810 includes hardware, software or both, and the parts of flow control device are coupled to each other.For example, but not limitation, bus may include accelerated graphics port (AGP) or other graphics bus, enhanced industry standard system (ETSA) bus, front side bus (FSB), hypertransport (HT) interconnection, industry standard system (TSA) bus, infinite bandwidth interconnection, low pin count (LPC) bus, memory bus, micro channel system (MCA) bus, peripheral component interconnection (PCT) bus, PCT-Express (PCT-X) bus, serial advanced technology attachment (SATA) bus, video electronics standard association local (VLB) bus or other suitable bus or two or more of these combinations. In appropriate cases, bus 810 may include one or more buses. Although the present application embodiment describes and shows a specific bus, the application considers any suitable bus or interconnection.

[0206] The face recognition attack detection device can execute the face recognition attack detection method in the embodiment of the present application, thereby realizing the combination of Figures 2 to 8 A method and apparatus for detecting attacks on face recognition are described.

[0207] In addition, in combination with the method for detecting face recognition attacks in the above embodiments, the embodiments of the present application may provide a computer-readable storage medium for implementation. The computer-readable storage medium stores computer program instructions; when the computer program instructions are executed by the processor, any one of the methods for detecting face recognition attacks in the above embodiments is implemented.

[0208] It should be clear that the present application is not limited to the specific configuration and processing described above and shown in the figures. For the sake of simplicity, a detailed description of the known method is omitted here. In the above embodiments, several specific steps are described and shown as examples. However, the method process of the present application is not limited to the specific steps described and shown, and those skilled in the art can make various changes, modifications and additions, or change the order between the steps after understanding the spirit of the present application.

[0209] The functional blocks shown in the above block diagram can be implemented as hardware, software, firmware or a combination thereof. When implemented in hardware, it can be, for example, an electronic circuit, an application specific integrated circuit (ASIC), appropriate firmware, a plug-in, a function card, etc. When implemented in software, the elements of the present application are programs or code segments that are used to perform the required tasks. The program or code segment can be stored in a machine-readable medium, or transmitted on a transmission medium or a communication link by a data signal carried in a carrier wave. "Machine-readable medium" can include any medium capable of storing or transmitting information. Examples of machine-readable media include electronic circuits, semiconductor memory devices, ROM, flash memory, erasable ROM (EROM), floppy disks, CD-ROMs, optical disks, hard disks, optical fiber media, radio frequency (RF) links, etc. The code segment can be downloaded via a computer network such as the Internet, an intranet, etc.

[0210] It should also be noted that the exemplary embodiments mentioned in this application describe some methods or systems based on a series of steps or devices. However, this application is not limited to the order of the above steps, that is, the steps can be performed in the order mentioned in the embodiment, or in a different order from the embodiment, or several steps can be performed simultaneously.

[0211] The above are only specific implementation methods of the present application. Those skilled in the art can clearly understand that for the convenience and simplicity of description, the specific working processes of the systems, modules and units described above can refer to the corresponding processes in the aforementioned method embodiments, and will not be repeated here. It should be understood that the protection scope of the present application is not limited to this. Any technician familiar with the technical field can easily think of various equivalent modifications or replacements within the technical scope disclosed in this application, and these modifications or replacements should be included in the protection scope of this application.

Claims

1. A method for detecting a face recognition attack, comprising: When face detection is enabled, obtain sensor data of the electronic device and at least two frames of images in the face acquisition preview control; Determining change information of the electronic device according to the sensor data, wherein the change information is used to characterize a motion posture of the electronic device in space; Based on the at least two frames of images, determining movement information of pixel points in a background area in the at least two frames of images, the background area being a non-face area in the images; Whether there is a face recognition attack is determined based on the matching between the change information and the movement information.

2. The detection method according to claim 1, wherein: The sensor data includes gyroscope sensor data and acceleration sensor data in N time periods, where N is an integer greater than 1; The determining, according to the sensor data, change information of the electronic device includes: Performing data fusion processing on the gyroscope sensor data and the acceleration sensor data in each of the N time periods to obtain the attitude angle of the electronic device in each of the N time periods; The change information of the electronic device is generated according to the posture angle in each of the N time periods.

3. The detection method according to claim 2, wherein: The gyroscope sensor data includes a first angular velocity, and the acceleration sensor data includes a second angular velocity, where the second angular velocity is an angular velocity obtained by processing the acceleration data measured by the acceleration sensor with an inverse tangent function; The step of performing data fusion processing on the gyroscope sensor data and the acceleration sensor data in each of the N time periods to obtain the attitude angle of the electronic device in each time period includes: The first angular velocity and the second angular velocity are weightedly summed by using a first weighting coefficient corresponding to the gyro sensor data and a second weighting coefficient corresponding to the acceleration sensor data to obtain the attitude angle of the electronic device in each time period.

4. The detection method according to claim 1, wherein: The determining, based on the at least two frames of images, movement information of pixels in the background area of ​​the at least two frames of images comprises: Based on the at least two frames of images, determine motion information and correlation information in the time domain of pixel points in the background area of ​​two adjacent frames of images in the at least two frames of images, the motion information is used to represent the degree of motion of the pixel point in the two adjacent frames of images, and the correlation information is used to represent the moving direction and distance of the pixel point in a reference time period, and the reference time period is the time interval for obtaining the two adjacent frames of images; Based on the motion information and the correlation information, movement information of pixel points in the background area of ​​the at least two frames of images is determined.

5. The detection method according to claim 4, wherein: The motion information includes a motion vector of the pixel point in the two adjacent frames of images; The step of determining, based on the at least two frames of images, motion information and correlation information of pixel points in the background area of ​​two adjacent frames of images in the at least two frames of images in the time domain includes: Determine, according to the at least two frames of images, the grayscale value of the pixel point in the background area of ​​each frame of the at least two frames of images; Constructing a residual function according to the grayscale values ​​of the pixels in the background area of ​​two adjacent frames of images in the at least two frames of images; Based on the residual function, the motion vector of the pixel point in the two adjacent frames of images is calculated by the least square method and reference constraints.

6. The detection method according to claim 4 or 5, wherein: The correlation information includes a motion vector of the pixel point in the two adjacent frames of images; and determining the motion information and correlation information of the pixel points in the background area of ​​the two adjacent frames of images in the at least two frames of images in the time domain based on the at least two frames of images includes: Determine pixel matching blocks in the two adjacent image frames respectively according to the moving direction and distance of the pixel point in the two adjacent image frames, the two adjacent image frames include a first image frame and a second image frame, and the pixel matching blocks include a first pixel matching block in the first image frame and a second pixel matching block in the second image frame; Calculating the instantaneous motion vector of the image of the two adjacent frames within the reference time period according to the matching result of the first pixel matching block and the second pixel matching block; The instantaneous motion vectors of the images of each two adjacent frames of the at least two frames of images within a reference time period corresponding to the each two adjacent frames of images are superimposed to obtain the motion vector of the pixel point in the two adjacent frames of images.

7. The detection method according to claim 1, wherein: Before acquiring sensor data of the electronic device and at least two frames of images in the face acquisition preview control when face detection is enabled, the method further includes: Receiving a first input from a user confirming to start a face detection control in the first interface; In response to the first input, start face detection; The method of obtaining sensor data of the electronic device and at least two frames of images in the face acquisition preview control when face detection is turned on includes: When face detection is enabled, collecting the sensor data through at least two sensors in the electronic device; And, when the face collection preview control in the first interface includes the user's face, at least two frames of images in the face collection preview control are collected according to the face detection strategy.

8. The detection method according to claim 1, wherein: The change information includes motion vectors of pixels of the background area in two adjacent frames of the at least two frames of images, and the movement information includes motion vectors of pixels of the background area in the two adjacent frames of images; The determining whether there is a face recognition attack based on the matching of the change information and the movement information includes: Determine the angle between the displacement vectors of the electronic device and the at least two frames of images based on the motion vector and the movement vector; It is determined whether there is a face recognition attack according to the angle between the displacement vectors.

9. The detection method according to claim 8, wherein: The determining whether there is a face recognition attack according to the displacement vector angle includes: When the displacement vector angle is smaller than the reference displacement vector angle, it is determined that there is no face recognition attack; When the displacement vector angle is greater than or equal to the reference displacement vector angle, it is determined that a face recognition attack exists.

10. The detection method according to any one of claims 1, 8 and 9, wherein: The method further comprises: When it is determined that there is no face recognition attack, first detection data is sent to the server, where the first detection data includes at least two frames of images in the face acquisition preview control during the face detection process, and the first detection data is used by the server to detect face recognition attacks and determine face recognition results.

11. The detection method according to any one of claims 1, 8 and 9, wherein: The method further comprises: If a face recognition attack is detected, perform at least one of the following actions: Sending second detection data to a server, not responding to the face detection and sending an alarm message to the server, counting the number of times that face recognition attacks are determined to exist related to the electronic device, and disabling the face recognition function when the number is greater than or equal to a preset number; The second detection data includes a device identifier of the electronic device, at least two frames of images in the face collection preview control during the face detection process, sensor data of the electronic device during the face detection process, and liveness detection action feature parameters of the user during the liveness detection phase of the face detection process. The second detection data is used by the server to update attack data in an attack background database, and the attack background database is used by the server to detect face recognition attacks. The warning information is used to indicate that the electronic device is attacked by face recognition.

12. The detection method according to claim 11, wherein: The operation is sending the second detection data to the server, and the second detection data is also used by the server to verify the second detection data through an outlier analysis algorithm. If the verification is passed, the attack data in the attack background database is updated according to the second detection data.

13. The detection method according to any one of claims 1, 8 and 9, wherein: The electronic device includes a first face detection strategy, and the at least two frames of images are acquired according to the first face detection strategy; the method further includes: receiving a second face detection strategy sent by the server, where the second face detection strategy is sent by the server when it is determined that the electronic device is subject to a face recognition attack, and a detection difficulty level of the second face detection strategy is higher than a detection difficulty level of the first face detection strategy; When it is detected that face detection is turned on next time, at least two frames of images in the face acquisition preview control are obtained according to the second face detection strategy.

14. A detection device for face recognition attacks, comprising: An acquisition module, used to acquire sensor data of the electronic device and at least two frames of images in the face acquisition preview control when face detection is detected to be turned on; A determination module, used to determine change information of the electronic device according to the sensor data, wherein the change information is used to characterize the motion posture of the electronic device in space; The determination module is further used to determine, based on the at least two frames of images, movement information of pixel points in a background area in the at least two frames of images, wherein the background area is a non-face area in the images; The determination module is further configured to determine whether a face recognition attack occurs based on a match between the change information and the movement information.

15. An electronic device, comprising: a processor and a memory storing computer program instructions; When the processor executes the computer program instructions, the steps of the method for detecting face recognition attacks as described in any one of claims 1-13 are implemented.

16. A storage medium having computer program instructions stored thereon, wherein the computer program instructions, when executed by a processor, implement the steps of the method for detecting face recognition attacks as described in any one of claims 1 to 13.

17. A computer program product, characterized in that The program product is stored in a storage medium, and the program product is executed by at least one processor to implement the steps of the method for detecting face recognition attacks as described in any one of claims 1-13.

Images