Face authentication method and electronic equipment

Abstract

The invention relates to the technical field of face recognition, and discloses a face authentication method and electronic equipment, which are used for improving the safety of face authentication. In the method, the electronic equipment can obtain a first image, the first image is an image of a first space region, and the first image comprises a set face of a first user. The electronic device can also obtain first data collected by the event camera for the first spatial area, wherein the first data is used for determining a first object moving in the first spatial area. When the electronic equipment determines that the face of the first user comprises the first object according to the first image and the first data, the electronic equipment can display the first information and / or execute the first operation. Therefore, after the electronic equipment determines that the face detection of the acquired image is passed, the electronic equipment needs to determine whether the face of the first user comprises the first object again, and the face detection is assisted, so that the safety of face authentication can be improved.

Description

Technical Field

[0001] The embodiments of the present application relate to the technical field of face recognition, and in particular, to a face authentication method and an electronic device. Background Art

[0002] Face recognition technology is a biometric technology for identity recognition based on facial feature data of a human face. The application scenarios of face recognition technology are very extensive, including scenarios such as mobile phone unlocking, identity verification, and work attendance checking, and it has been popularized in fields such as finance, medical care, payment, and culture and entertainment. An electronic device can recognize a human face through face recognition technology to complete face authentication. However, with the development of artificial intelligence (AI) technology, there are also risks in face authentication. For example, situations such as replacing a face with a video for face authentication will result in a low accuracy rate of face authentication.

[0003] Currently, most traditional face recognition technologies use two-dimensional (2D) face recognition. However, in actual use, 2D information has the limitation of losing depth data and cannot completely represent a real human face, resulting in a low accuracy rate of face recognition and the security of face authentication. Summary of the Invention

[0004] The embodiments of the present application provide a face authentication method and an electronic device to improve the accuracy and security of face authentication.

[0005] In a first aspect, the embodiments of the present application provide a face authentication method applied to an electronic device. In this method, the electronic device can obtain a first image, where the first image is an image of a first spatial region and includes the face of a set first user. The electronic device can also obtain first data collected by an event camera for the first spatial region, and the first data is used to determine a first object that moves in the first spatial region. When it is determined according to the first image and the first data that the face of the first user includes the first object, the electronic device can display first information and / or perform a first operation.

[0006] Optionally, the first object is an object that moves continuously within the first spatial region, the first information is used to indicate that face authentication is successful, and the first operation is an operation to be performed when face authentication is successful.

[0007] In this method, when the electronic device determines that the image of the first spatial region collected includes the face of the first user, it can determine that the user for face authentication has access rights. In this case, the electronic device also needs to perform a secondary detection on the first data collected by the event camera for the first spatial region to further confirm whether the face of the first user includes a moving first object, so as to determine that the person for authentication is a real person rather than a replaced face, and thus can assist in improving the accuracy and security of face authentication.

[0008] In a possible design, when the electronic device determines that the face of the first user does not include the first object based on the first image and the first data, the electronic device can display the second information and / or perform the second operation.

[0009] Optionally, the second information is used to indicate that the face authentication fails, and the second operation is an operation to be performed when the face authentication fails.

[0010] Through this design, when the electronic device determines that the face of the first user does not include the first object, it can determine that the face of the first user has not moved during the authentication process, and thus can determine that the person for authentication is not a real person. Therefore, the electronic device can perform subsequent processing with the result of face authentication failure, thereby improving the security of face authentication.

[0011] In a possible design, the event camera includes a pixel array, and the pixel array includes a plurality of pixels; the first data includes a plurality of event data collected at a plurality of time points, and the plurality of event data includes: one or more event data of the plurality of pixels.

[0012] Optionally, the collection times of the plurality of event data of the same pixel are different, and one event data of any pixel is used to indicate: at the time point when the event data is collected, the brightness change event at that pixel.

[0013] In a possible design, the electronic device can also determine the first image region occupied by the face of the first user in the first image. The electronic device can also determine the second data corresponding to the first image region in the first data, and determine that the face of the first user includes the first object based on the second data.

[0014] Through this design, the electronic device can determine the second data according to the image region occupied by the face of the first user in the first image, without performing recognition processing on the first data. And the electronic device can also determine whether the face region of the first user includes the first object based on the second data to determine whether it is a real scene or a synthetic video for authentication, thereby improving the security of face authentication.

[0015] In a possible design, the electronic device can also determine a first target area corresponding to the first image area in the pixel array according to the mapping relationship between the first image and the pixel array. The electronic device can select second data from the first data according to the first target area, and the second data includes event data of pixels in the first target area among the pixels.

[0016] With this design, the electronic device can obtain the interested second data from the first data based on the established mapping relationship between the first image and the pixel array, without performing operations such as recognition processing on the first data, thereby reducing power consumption.

[0017] In a possible design, when the electronic device determines that the second data includes event data corresponding to the first object, it can determine that the face of the first user contains the first object.

[0018] With this design, the electronic device can determine that the first target area corresponding to the first image area contains the moving first object by determining that the second data includes event data corresponding to the first object. Furthermore, the electronic device can determine that the face of the first user contains the moving first object, enabling the electronic device to determine that the user for authentication is a real person, thereby improving the security of face authentication.

[0019] In a possible design, the electronic device can also determine the first image area occupied by the face of the first user in the first image. The electronic device can also determine target pixels according to the first data, and the target pixels include pixels corresponding to the first object among multiple pixels. The electronic device can determine that the face of the first user contains the first object according to the first image area and the target pixels.

[0020] With this design, the electronic device can determine the target pixels corresponding to the first object from the first data, and by comparing the spatial area corresponding to the first image area with the target pixels, determine whether the face of the first user contains the moving first object, thereby enabling live detection of the face of the first user, and further assisting in improving the security of face authentication.

[0021] In a possible design, the time points when the event camera captures the event data of the target pixels include: consecutive time points among multiple time points arranged in chronological order.

[0022] With this design, the electronic device can determine the target pixels corresponding to the moving object according to the continuity of the event data of the pixels in the time domain.

[0023] In a possible design, the electronic device may also divide the first data into multiple groups of data according to the chronological order of the collected data. Among the multiple groups of data, each group of data includes at least one event data among the multiple event data, and the time points at which the event camera captures at least one event data are the same. When the electronic device determines that there are consecutive multiple groups of data among the multiple data that respectively include different event data of the same pixel, the same pixel is used as the target pixel.

[0024] In a possible design, the electronic device may also determine, according to the mapping relationship between the first image and the pixel array, the second image area corresponding to the second target area in the pixel array in the first image. The second target area is the area occupied by the target pixel. When the electronic device determines that there is an intersection between the first image area and the second image area, it determines that the face of the first user includes the first object.

[0025] Through this design, the electronic device can determine the second image area occupied by the first object in the first image based on the mapping relationship between the first image and the pixel array, and perform liveness detection on the face of the first user by determining whether there is an intersection between the first image area and the second image area, so as to determine whether the person for authentication is a real person, and further improve the security of face authentication.

[0026] In a possible design, the first object includes at least one of the following: the face of the first user, the eyes of the first user, and the lips of the first user.

[0027] In a possible design, the electronic device may also acquire a second image, where the second image is an image of the first spatial area. When the electronic device determines that the second image does not include the face of the first user, it displays the second information and / or performs a second operation.

[0028] Through this design, when the electronic device determines that the acquired second image does not contain the face of the first user, it can determine that the first detection of face authentication fails, and can directly display that the face authentication fails, or perform the operation that needs to be performed when the face authentication fails, avoiding secondary detection, thereby saving power consumption.

[0029] In a second aspect, the present application provides an electronic device, including an event camera, a camera, one or more processors, and one or more memories. The event camera is used to detect an object that moves in a spatial area, the camera is used to acquire an image, the one or more memories are used to store one or more computer programs and data information, and the one or more processors are used to execute the computer programs stored in the one or more memories, so that the electronic device executes the method described in any aspect of the first aspect above. Optionally, the electronic device may be the electronic device of the first aspect.

[0030] In a third aspect, the present application provides a computer-readable storage medium storing a computer program or instruction. When the computer program or instruction is executed by a computing device, the computing device executes the method in any aspect of the first aspect above and any possible implementation manner of any aspect.

[0031] In a fourth aspect, the present application provides a chip system including a processor and a memory, where an instruction is stored in the memory; when the instruction is executed by the processor, the method described in the first aspect above or any possible design of the first aspect is implemented. The chip system may be composed of chips or may include chips and other discrete devices.

[0032] In a fifth aspect, the present application provides a computer program product including a computer program or instruction. When the computer program or instruction is executed by a computing device, the computing device executes the method in any aspect of the first aspect above and any possible implementation manner of any aspect. Description of the Drawings

[0033] Figure 1 It is a schematic diagram of a face authentication scenario;

[0034] Figure 2 It is a schematic diagram of the hardware structure of a possible electronic device provided exemplarily by the present application;

[0035] Figure 3 It is a schematic diagram of the software structure of a possible electronic device provided exemplarily by the present application;

[0036] Figure 4 It is a schematic flowchart of a face authentication method provided by the present application;

[0037] Figure 5 It is a schematic diagram of a collection area provided by the present application;

[0038] Figure 6 It is another schematic diagram of a collection area provided by the present application;

[0039] Figure 7 It is a schematic flowchart of a face authentication use case provided by the present application;

[0040] Figure 8 It is a schematic diagram of the structure of an electronic device provided by the present application. Detailed Embodiments

[0041] Next, the technical solutions in the embodiments of the present application will be described in detail with reference to the accompanying drawings in the following embodiments of the present application.

[0042] First, concepts related to the embodiments of the present application will be explained.

[0043] (1) An electronic device, which can be a device with an authentication function.

[0044] In some embodiments of the present application, the electronic device may be a portable device, such as a mobile phone, a tablet computer, a wearable device with wireless communication function (such as a watch, a bracelet, etc.), a vehicle-mounted terminal device, an augmented reality (AR) / virtual reality (VR) device, a laptop computer, an ultra-mobile personal computer (UMPC), a netbook, a personal digital assistant (PDA), a smart home device (such as a smart TV, a smart speaker, etc.), a smart robot, a workshop device, a wireless terminal in self-driving, a wireless terminal in remote medical surgery, a wireless terminal in smart grid, a wireless terminal in transportation safety, a wireless terminal in smart city, or a wireless terminal in smart home, a flying device (such as a smart robot, a drone, an airplane), etc.

[0045] Among them, the wearable device is a portable device that the user can directly wear on the body or integrate into the user's clothes or accessories.

[0046] In some embodiments of the present application, the electronic device may also be a portable terminal device that further includes other functions such as a personal digital assistant and / or a music player function. Exemplary embodiments of the portable terminal device include, but are not limited to, those equipped with or other operating systems. The above portable terminal device may also be other portable terminal devices, such as a laptop with a touch-sensitive surface (such as a touch panel), etc. It should also be understood that in some other embodiments of the present application, the above electronic device may not be a portable terminal device, but a desktop computer with a touch-sensitive surface (such as a touch panel), or other electronic devices with an access control function such as a building access control.

[0047] (2) Artificial intelligence (AI) face-swap refers to replacing a face in a picture or video with another face to synthesize new media. AI face-swap is the most well-known application form of deepfake technology. Deepfake technology can also be called visual deepfake technology. Deepfake refers to creating or synthesizing audiovisual content, such as images, audio and video, text, etc., based on machine learning methods such as deep learning. From a visual perspective, deepfakes can generally be divided into four categories: reenactment, replacement, editing, and synthesis. Among them, reenactment and replacement have huge potential risks and can be exploited by attackers to control identities and deceive.

[0048] (3) An event camera, a new type of vision sensor inspired by biology, can also be called an event vision sensor (EVS), a dynamic vision sensor (DVS), a dynamic and active-pixel vision sensor (DAVIS), or an asynchronous time-based image sensor (ATIS). Instead of outputting intensity image frames at a constant rate, an event camera only outputs information related to local pixel-level brightness changes (pixel-level brightness changes can be called events). Since it is good at capturing brightness changes, it can output valid data even in darker and strong-light scenarios. Event cameras have the advantages of high frame rate (>5000 frames per second (fps)), low latency, high dynamic range, low power consumption, less data redundancy, and low transmission bandwidth. An event camera includes a pixel array, which contains multiple pixels. At each pixel in the multiple pixels, there is an independent optoelectronic sensing module. When the optoelectronic sensing module senses that the brightness change at this pixel exceeds a set threshold (for example, the difference between the brightness of this pixel at the current moment and the brightness at the previous moment is greater than or equal to the set threshold), it will generate and output the event data (also called pulse data) of this pixel. The event data of a pixel can be used to represent the event of the brightness change of this pixel in the time domain, the brightness change situation of this pixel in the time domain, etc. The event data of a pixel includes the pixel coordinates of the pixel, the timestamp when the event occurs, and the polarity. Among them, the pixel coordinates are used to indicate the coordinates of the pixel in the pixel array, the timestamp is used to indicate the time when the event data of the pixel is obtained, and the polarity is used to indicate whether the brightness change of the pixel is from low to high (i.e., getting brighter) or from high to low (i.e., getting darker). Polarity is also often called polarity or negative event or on or off event. Polarity can also be understood as the event type. Stacking the event data together according to the pixel positions can obtain an image.

[0049] (4) Region of interest (ROI) refers to the area that needs to be processed outlined in a box, circle, ellipse, irregular polygon, etc. in the image to be processed in machine vision and image processing. Various operators and functions are often used in machine vision software such as Halcon, OpenCV, and Matlab to obtain the region of interest ROI for the next step of image processing.

[0050] (5) Image registration refers to finding the spatial mapping relationship from the pixels of one image to the pixels of another image. These images can be acquired (multi-modal registration) by different sensors (or cameras or imaging devices) at different times (multi-temporal registration), different positions, or different conditions (such as weather, illumination, camera position and angle, etc.). That is to say, image registration is the process of matching and superimposing multiple images obtained by different sensors at different times, different positions, or different conditions. Among them, the process of image registration technology can include: extracting feature points from two images respectively; finding matching feature point pairs among the extracted feature points through similarity measurement; obtaining image spatial coordinate transformation parameters through the matching feature point pairs; and performing image registration through the coordinate transformation parameters.

[0051] (6) Face authentication refers to an authentication method for verifying whether a user has the right to access. Among them, the access right can be the right to access a system, a building, etc. The premise of this method is that each user whose face is recorded has been authorized. The face authentication method verifies whether a user can access by using whether the face of the authenticated user is authorized.

[0052] In the embodiments of the present application, "at least one" means one or more, and "multiple" means two or more. "And / or" describes the association relationship of associated objects, indicating that there can be three relationships. For example, A and / or B can represent: A exists alone, A and B exist simultaneously, and B exists alone, where A and B can be singular or plural. The character " / " generally represents an "or" relationship between the front and back associated objects. "At least one (item)" or its similar expression refers to any combination of these items, including any combination of single item (item) or plural items (items). For example, at least one (item) of a, b, or c can represent: a, b, c, a and b, a and c, b and c, or a and b and c, where a, b, and c can be single or multiple.

[0053] In addition, unless otherwise stated, the ordinal numbers such as "first" and "second" mentioned in the embodiments of the present application are used to distinguish multiple objects, and are not used to limit the size, content, order, time sequence, priority, or importance of multiple objects. For example, the first file and the second file are only used to distinguish different files, rather than indicating differences in the size, content, priority, or importance of these two files.

[0054] Currently, most traditional face recognition technologies use 2D face recognition. However, in actual use, 2D information has the limitation of losing depth data and cannot fully represent a real face, resulting in low accuracy of face recognition and low security of face authentication.

[0055] To improve the security of face authentication, an embodiment of the present application provides a face authentication method applied to an electronic device. In this method, when the electronic device determines that the face detection result of the captured image is successful, it can use the first data captured by the event camera for secondary detection to further confirm whether the user undergoing face authentication is a live body. In this way, when the electronic device performs face authentication, it needs to perform double authentication through one image feature detection and one live body detection, which can accurately detect the authenticity of the live face, thereby improving the security of face authentication.

[0056] The embodiments of the present application will be described in detail below with reference to the accompanying drawings.

[0057] The embodiments of the present application can be applied to various scenarios where relevant functions are performed through face authentication. For example, as Figure 1 shown, the user can perform face authentication through the electronic device. The electronic device acquires an image containing the user's face by collecting images of the surrounding environment to perform face authentication on the user. Among them, the electronic device can include sensors, and the sensors can be a camera and an event camera. Exemplarily, the camera can be an RGB camera, and the RGB camera is used to collect RGB images, and the event camera is used to capture moving objects.

[0058] In some embodiments, the electronic device can collect RGB images of the user undergoing face authentication by using the camera for the surrounding environment. The electronic device can capture the moving objects in the surrounding environment through the event camera to obtain the first data. The electronic device can perform face detection on the RGB image to complete one face authentication. When the electronic device determines that the face detection result of the RGB image is successful, the electronic device can determine whether the user undergoing face authentication is a live body according to the RGB image and the first data, thereby realizing secondary face authentication. In this way, after the face detection of the electronic device is successful, it still needs to perform a live body detection to further determine that the user undergoing face authentication is a live body rather than a synthetic video, thereby improving the security of face authentication.

[0059] In some examples, the electronic device can collect images of the surrounding environment through a camera to obtain the user's RGB image. Exemplarily, the user can stand within the collection area of the electronic device so that the electronic device can collect the user's RGB image through the RGB camera. The electronic device can also use an event camera to dynamically capture the collection area to obtain first data, which is used to determine the object that moves within the collection area. After collecting the RGB image, the electronic device can perform face detection on the RGB image to determine whether the face of the user undergoing the authentication operation is the face of a preset user. When the electronic device determines that the face detection result is that the RGB image includes the face of the preset user, it can perform a secondary detection on the user based on the RGB image and the first data. Exemplarily, the electronic device can determine that the user undergoing the authentication is a live body by determining that the face area of the user in the RGB image includes the continuously moving object captured by the event camera. In this way, the electronic device can determine that the face authentication result of the user is authentication passed. When the face authentication result is authentication passed, the electronic device can execute related functions, such as waking up the electronic device and opening the door.

[0060] Figure 2 The schematic diagram of the hardware structure of a possible electronic device is shown. The electronic device 100 can be Figure 1 the electronic device in. As Figure 2 shown, the electronic device 100 can include a processor 110, an external memory interface 120, an internal memory 121, a universal serial bus (USB) interface 130, a charging management module 140, a power management module 141, a battery 142, an antenna 1, an antenna 2, a mobile communication module 15, a wireless communication module 16, an audio module 17, a speaker 170A, a receiver 170B, a microphone 170C, a headphone interface 170D, a sensor module 18, a key 19, a motor 191, an indicator 192, a camera 193, a display screen 194, and a subscriber identification module (SIM) card interface 195, etc. Among them, the sensor module 18 can include a pressure sensor 180A, a gyroscope sensor 180B, a barometric pressure sensor 180C, a magnetic sensor 180D, an acceleration sensor 180E, a distance sensor 180F, a proximity light sensor 180G, a fingerprint sensor 180H, a temperature sensor 180J, a touch sensor 180K, an ambient light sensor 180L, a bone conduction sensor 180M, etc.

[0061] The processor 110 may include one or more processing units. For example, the processor 110 may include an application processor (AP), a microcontroller unit (MCU), a modem processor, a graphics processing unit (GPU), an image signal processor (ISP), a controller, a memory, a video codec, a digital signal processor (DSP), a baseband processor, and / or a neural-network processing unit (NPU), etc. Among them, different processing units may be independent devices or integrated in one or more processors. Among them, the controller may be the nerve center and command center of the electronic device 100. The controller can generate operation control signals according to the instruction operation code and timing signals to complete the control of fetching and executing instructions. A memory may also be provided in the processor 110 for storing instructions and data. In some embodiments, the memory in the processor 110 is a cache memory. This memory can save the instructions or data that the processor 110 has just used or recycled. If the processor 110 needs to use the instruction or data again, it can be directly called from the memory. This avoids repeated accesses, reduces the waiting time of the processor 110, and thus improves the efficiency of the system.

[0062] In the embodiments of the present application, the processor 110 may save one or more application programs, may acquire a first image, and may acquire first data collected by the event camera for a first spatial region; wherein, the first image is an image of the first spatial region, and the first image includes the face of a set first user, and the first data is used to determine a first object that has continuous movement in the first spatial region. In addition, when the processor 110 determines that the face of the first user includes the first object according to the first image and the first data, it may display first information and / or perform a first operation on the display screen 194.

[0063] The USB interface 130 is an interface that complies with the USB standard specification. Specifically, it can be a Mini USB interface, a Micro USB interface, a USB Type C interface, etc. The USB interface 130 can be used to connect a charger to charge the electronic device 100, or to transfer data between the electronic device 100 and peripheral devices. The charging management module 140 is used to receive the charging input from the charger. The power management module 141 is used to connect the battery 142, the charging management module 140, and the processor 110. The power management module 141 receives the inputs from the battery 142 and / or the charging management module 140, and supplies power to the processor 110, the internal memory 121, the external memory, the display screen 194, the camera 193, the wireless communication module 160, etc.

[0064] The wireless communication function of the electronic device 100 can be implemented by the antenna 1, the antenna 2, the mobile communication module 150, the wireless communication module 160, the modulation and demodulation processor, and the baseband processor, etc. The antenna 1 and the antenna 2 are used to transmit and receive electromagnetic wave signals. Each antenna in the electronic device 100 can be used to cover a single or multiple communication frequency bands. Different antennas can also be multiplexed to improve the utilization rate of the antennas. For example, the antenna 1 can be multiplexed as the diversity antenna of the wireless local area network. In some other embodiments, the antenna can be used in combination with a tuning switch.

[0065] The mobile communication module 150 can provide solutions for wireless communications including 2G / 3G / 4G / 5G, etc. applied to the electronic device 100. The mobile communication module 150 can include at least one filter, switch, power amplifier, low noise amplifier (LNA), etc. The mobile communication module 150 can receive the electromagnetic wave by the antenna 1, and perform filtering, amplification, etc. on the received electromagnetic wave, and transmit it to the modulation and demodulation processor for demodulation. The mobile communication module 150 can also amplify the signal modulated by the modulation and demodulation processor, and convert it into an electromagnetic wave through the antenna 1 and radiate it out. In some embodiments, at least some functional modules of the mobile communication module 150 can be provided in the processor 110. In some embodiments, at least some functional modules of the mobile communication module 150 and at least some modules of the processor 110 can be provided in the same device.

[0066] The wireless communication module 160 may provide wireless communication solutions applied to the electronic device 100, including wireless local area network (WLAN) (such as Wi-Fi network), Bluetooth (BT), global navigation satellite system (GNSS), frequency modulation (FM), near field communication (NFC), infrared (IR), etc. The wireless communication module 160 may be one or more devices integrating at least one communication processing module. The wireless communication module 160 receives electromagnetic waves via the antenna 2, performs frequency modulation and filtering processing on the electromagnetic wave signals, and sends the processed signals to the processor 110. The wireless communication module 160 may also receive the signals to be sent from the processor 110, perform frequency modulation and amplification on them, and convert them into electromagnetic waves through the antenna 2 for radiation.

[0067] In some embodiments, antenna 1 of electronic device 100 is coupled to mobile communication module 150, and antenna 2 is coupled to wireless communication module 160, such that electronic device 100 can communicate with a network and other devices through wireless communication technologies. The wireless communication technologies may include global system for mobile communications (GSM), general packet radio service (GPRS), code division multiple access (CDMA), wideband code division multiple access (WCDMA), time-division code division multiple access (TD-CDMA), long term evolution (LTE), the fifth generation (5G) mobile communication system, future communication systems such as the sixth generation (6G) system, etc., BT, GNSS, WLAN, NFC, FM, and / or IR technologies. The GNSS may include global positioning system (GPS), global navigation satellite system (GLONASS), beidou navigation satellite system (BDS), quasi-zenith satellite system (QZSS), and / or satellite based augmentation systems (SBAS).

[0068] The display screen 194 is used to display images, videos, etc. The display screen 194 includes a display panel. The display panel can adopt a liquid crystal display (LCD), an organic light-emitting diode (OLED), an active-matrix organic light-emitting diode (AMOLED), a flexible light-emitting diode (FLED), a Miniled, a MicroLed, a Micro-oLed, a quantum dot light-emitting diode (QLED), etc. In some embodiments, the electronic device 100 may include one or N display screens 194, where N is a positive integer greater than 1. In the embodiments of the present application, the display screen 194 can be used to display the main interface, the application interface, the granularity adjustment control, etc.

[0069] The camera 193 is used to capture static images or videos. The camera 193 can include a front camera and a rear camera. In the embodiments of the present application, the camera 193 includes at least one RGB camera and at least one event camera.

[0070] The internal memory 121 can be used to store computer-executable program codes, and the executable program codes include instructions. The processor 110 executes various functional applications and data processing of the electronic device 100 by running the instructions stored in the internal memory 121. The internal memory 121 can include a program storage area and a data storage area. Among them, the program storage area can store the operating system and software codes of at least one application program (such as Huawei Video, Changlian, etc.). The data storage area can store the data generated during the use of the electronic device 100 (such as images, videos, etc.). In addition, the internal memory 121 can include a high-speed random access memory, and can also include a non-volatile memory, such as at least one disk storage device, a flash memory device, a universal flash storage (UFS), etc.

[0071] The external memory interface 120 can be used to connect an external memory card, such as a Micro SD card, to expand the storage capacity of the electronic device 100. The external memory card communicates with the processor 110 through the external memory interface 120 to achieve the data storage function. For example, files such as pictures and videos are saved in the external memory card.

[0072] The electronic device 100 can implement audio functions through an audio module 170, a speaker 170A, a receiver 170B, a microphone 170C, a headphone jack 170D, and an application processor, etc. For example, music playback, recording, etc.

[0073] It can be understood that Figure 2 the components shown do not constitute a specific limitation on the electronic device. The electronic device may also include more or fewer components than shown, or combine certain components, or split certain components, or have different component arrangements. The various components shown in the figure can be implemented in hardware, software, or a combination of hardware and software including one or more signal processing and / or application specific integrated circuits.

[0074] The software system of the electronic device 100 can adopt a layered architecture, an event-driven architecture, a microkernel architecture, a microservices architecture, or a cloud architecture. The layered architecture divides the software into several layers, and each layer has a clear role and division of labor. The layers communicate with each other through software interfaces. Exemplarily, as Figure 3 shown, this software architecture can be divided into four layers, from top to bottom are the application layer, the application framework layer (framework, FWK), the runtime and system libraries, and the (Linux) kernel layer.

[0075] The application layer is the top layer of the operating system, including the native applications of the operating system, such as the camera, gallery, calendar, Bluetooth, music, video, information, etc., and may also include third-party applications. The application involved in the embodiments of the present application is abbreviated as an application (application, APP), which is a software program capable of implementing one or more specific functions. Generally, multiple applications can be installed in the electronic device, such as a camera application, a gallery application, etc. The applications mentioned below can be system applications pre-installed when the electronic device leaves the factory, or third-party applications downloaded from the network or obtained from other electronic devices by the user during the use of the electronic device.

[0076] Of course, for developers, they can write application programs and install them in this layer. In one possible implementation, the application program can be developed using the Java language and completed by calling the application programming interface (application programming interface, API) provided by the application framework layer. Developers can interact with the underlying layer of the operating system (such as the kernel layer, etc.) through the application framework to develop their own application programs.

[0077] The application framework layer provides APIs and programming frameworks for the application layer. The application framework layer may include some predefined functions. The application framework layer may include a window manager, a content provider, a view system, a telephone manager, a resource manager, a notification manager, etc.

[0078] The window manager is used to manage window programs. The window manager can obtain the display screen size, determine whether there is a status bar, lock the display screen (or screen), capture the display screen, etc.

[0079] The content provider is used to store and obtain data, and make this data accessible to application programs. The data may include information such as files (e.g., documents, videos, images, audio), text, etc.

[0080] The view system includes visual controls, such as controls for displaying text, pictures, documents, and other content. The view system can be used to build application programs. The interface in the display window can be composed of one or more views. For example, a display interface including a text message notification icon may include a view for displaying text and a view for displaying pictures.

[0081] The telephone manager is used to provide the communication function of the electronic device. The notification manager enables application programs to display notification information in the status bar, can be used to convey notification-type messages, and can automatically disappear after a short stay without user interaction.

[0082] The runtime includes a core library and a virtual machine. The runtime is responsible for the scheduling and management of the system.

[0083] The core library of the system consists of two parts: one part is the functional functions that need to be called by the Java language, and the other part is the core library of the system. The application layer and the application framework layer run in the virtual machine. Taking Java as an example, the virtual machine executes the Java files of the application layer and the application framework layer as binary files. The virtual machine is used to perform functions such as object lifecycle management, stack management, thread management, security and exception management, and garbage collection.

[0084] The system library may include multiple functional modules. For example: a surface manager, a media library, a 3D graphics processing library (e.g., OpenGL ES), a 2D graphics engine (e.g., SGL), an image processing library, etc. The surface manager is used to manage the display subsystem and provides the fusion of 2D and 3D layers for multiple application programs. The media library supports the playback and recording of multiple common audio and video formats, as well as static image files, etc. The media library can support multiple audio and video coding formats, such as: MPEG4, H.564, MP3, AAC, AMR, JPG, PNG, etc. The 3D graphics processing library is used to implement 3D graphics drawing, image rendering, synthesis, and layer processing, etc. The 2D graphics engine is the drawing engine for 2D drawing.

[0085] The kernel layer provides the core system services of the operating system. Services such as security, memory management, process management, network protocol stack, and driver model are all implemented based on the kernel layer. The kernel layer also serves as an abstraction layer between the hardware and software stacks. There are many driver programs related to electronic devices in this layer. The main drivers include: display driver; keyboard driver as an input device; Flash driver based on memory technology devices; camera driver; audio driver; Bluetooth driver; WiFi driver, etc.

[0086] It should be understood that the functional services described above are only examples. In actual applications, electronic devices can also be divided into more or fewer functional services according to other factors, or the functions of each service can be divided in other ways, or the functional services can be not divided but work as a whole.

[0087] The present application will be described below in conjunction with specific embodiments.

[0088] Figure 4 It is a schematic diagram of a face authentication method provided by an embodiment of the present application. As Figure 4 shown, the method includes:

[0089] S401: The electronic device acquires a first image, where the first image is an image of a first spatial region and includes the face of a set first user.

[0090] Among them, the first spatial region can be the acquisition region of the camera of the electronic device, and the first user is the user granted access rights by the electronic device.

[0091] In some embodiments, the electronic device can collect images of the surrounding environment through the camera to acquire the first image.

[0092] Exemplarily, the camera can be an RGB camera. Correspondingly, the first image is an RGB image.

[0093] The electronic device can collect images of the surrounding environment through the camera to obtain an RGB image. The electronic device can perform face recognition on the RGB image to obtain a face recognition result. When the face recognition result is that the RGB image includes the face of the first user, the electronic device can use the RGB image as the first image. When the face recognition result is that the RGB image does not include the face of the first user, the electronic device can determine that the first image has not been acquired. Exemplarily, the electronic device can use 2D face recognition technology to perform face recognition on the RGB image, or can also use a neural network to perform face recognition on the RGB image, which is not limited herein.

[0094] In a possible solution, the electronic device can perform real-time monitoring and image acquisition on the surrounding environment to obtain an RGB image. Exemplarily, during the real-time acquisition of the RGB image, the electronic device can perform image recognition on the acquired RGB image to determine whether a user enters the acquisition area of the electronic device. In a possible case, when the RGB image does not contain a user, the electronic device can determine that there is no user in the acquisition area for permission verification and continue to acquire images. In another possible case, when the RGB image contains a user, the electronic device can determine that a user has entered the acquisition area. When the user enters the acquisition area, the electronic device can perform face recognition on the real-time acquired RGB image to determine whether the face of the set first user is included in the RGB image. Exemplarily, the electronic device can perform face recognition on the RGB image through the method described in steps A1 to A3 below.

[0095] A1: The electronic device can perform face recognition on the RGB image to determine the face area in the RGB image.

[0096] In some embodiments, the electronic device can determine whether the user included in the RGB image is the first user by performing face recognition on the RGB image. In a possible case, if the electronic device determines that the first image does not contain a face area, the electronic device can determine that there is no user for authentication operation and continue to acquire the RGB image to obtain the first image.

[0097] A2: The electronic device extracts features from the face area of the RGB image to obtain face features.

[0098] A3: The electronic device matches the face features with the preset face information to obtain a face detection result.

[0099] In some embodiments, the preset face information can be pre-stored in the electronic device, and the preset face information can be the face information of the first user. Exemplarily, when the user enables the face authentication function of the electronic device, the face information of the first user can be stored in the electronic device as the authentication standard. In this way, the electronic device can determine whether the user performing the authentication operation is the first user with permission to access according to the pre-stored face information.

[0100] Exemplarily, the electronic device can determine the similarity between the facial features of a face and preset face information, and determine whether the facial features of the face match the preset face information according to the similarity. For example, when the similarity is greater than or equal to a set threshold, the electronic device can determine that the facial features of the face match the preset face information successfully. When the facial features of the face match the preset face information successfully, the electronic device can determine that the face detection result is that the RGB image contains the face of the first user, and use this RGB image as the first image. Also for example, when the similarity is less than the set threshold, the electronic device can determine that the facial features of the face do not match the preset face information successfully. After the facial features of the face do not match the preset face information successfully, the electronic device can determine that the face detection result is that the RGB image does not contain the face of the first user, and continue to perform image acquisition to obtain the first image. The electronic device can also notify the user that the authentication has failed before continuing to perform image acquisition and real-time monitoring of the surrounding environment. For example, the electronic device can inform the user that the authentication has failed through voice broadcast, or can also display a message indicating that the authentication has failed on the display interface to inform the user that the authentication has failed. The electronic device can also inform the user that the authentication has failed in other ways, which are not limited herein.

[0101] For example, taking the electronic device as a mobile phone as an example, as Figure 5 shown, the sensors on the electronic device can perform image acquisition on the acquisition area in real time. When the user needs to perform face authentication, they can enter the acquisition area of the electronic device. In this way, the electronic device can collect an RGB image containing the user through the sensors. When the electronic device determines that the RGB image containing the user includes the face of the first user, it can use this RGB image as the first image.

[0102] In another possible solution, the electronic device can also collect an RGB image including the user through the camera in response to an authentication operation triggered by the user. Among them, the authentication operation can be an operation that enables the electronic device to start the authentication process triggered by the user on the electronic device, such as an authentication start operation. Exemplarily, the user can trigger an authentication operation on the electronic device to obtain the access permission of the electronic device. The electronic device can collect an RGB image including the user in response to the authentication operation triggered by the user. When collecting the RGB image, the electronic device can display the collection area of the camera of the electronic device in the display interface. The user can place the face into the collection area of the electronic device according to the collection area displayed by the electronic device. In this way, the electronic device can collect an RGB image including the user's face. The electronic device can perform face recognition on the RGB image to determine whether the face of the first user is included in the RGB image. That is, the electronic device determines whether the user performing the authentication is the first user through face recognition. Among them, the process of the electronic device performing face recognition is the same as the process of performing face recognition in the above steps A1 to A3, and will not be described in detail here. When the RGB image includes the face of the first user, the electronic device can use the RGB image as the first image. When the RGB image does not include the face of the first user, the electronic device can inform the user that the authentication fails and end the authentication operation.

[0103] For example, taking the electronic device as a mobile phone, as Figure 6 shown, the user can trigger the authentication operation by clicking the face verification button on the first display interface of the electronic device. The electronic device can display the second display interface in response to the user's authentication operation and collect the user's RGB image through the camera. The user can place the face into the collection area of the electronic device according to the face area in the second display interface. The electronic device can perform face recognition on the RGB image to determine whether the face of the first user is included in the RGB image. When the electronic device determines that the face of the first user is included in the RGB image, it uses the RGB image as the first image. When the electronic device determines that the face of the first user is not included in the RGB image, it can display an authentication failure message to inform the user that the authentication fails, or can also inform the user that the authentication fails through operations such as voice broadcast.

[0104] S402: The electronic device obtains the first data collected by the event camera for the first spatial region. Among them, the first data is used to determine the first object that has continuous movement in the first spatial region. Exemplarily, the first object includes but is not limited to at least one of the following: the face of the first user, the eyes of the first user, the lips of the first user.

[0105] An electronic device can use an event camera to dynamically capture a first spatial region in real time, obtain event data of multiple pixels whose brightness changes, and obtain first data. Among them, one event data of any one of the multiple pixels is used to indicate: at the time point when the event data is collected, the brightness change event at this pixel. The first data includes multiple event data collected at multiple time points, and the multiple event data includes one or more event data of each pixel among the multiple pixels, and the collection times of the multiple event data of the same pixel are different. Exemplarily, when the electronic device determines that the brightness of a pixel in the pixel array changes and the degree of change exceeds a set threshold, the electronic device will generate one event data of this pixel. Among them, the event data includes the pixel coordinates (x, y) of the triggered event, the timestamp t, and the polarity p of the event. That is, the event data triggered by the pixel can be expressed as e = (x, y, t, p), where x and y represent the pixel coordinates of the pixel (i.e., the coordinates of the pixel in the pixel array), t represents the time when the event is triggered (which can also be understood as the time when the event data is collected), and p represents the type of the pixel-triggered event (such as getting brighter or darker).

[0106] The electronic device can divide the first data into multiple groups of data according to the chronological order of the timestamps t of the event triggers. Among them, each timestamp t corresponds to a group of data, that is, each group of data in the multiple groups of data includes at least one event data among the multiple event data, and the time points when the event camera collects at least one event data are the same. The electronic device can also stack the event data in each group of data together according to the pixel positions respectively, and obtain multiple second images arranged in the chronological order of the timestamps t. Among them, the second image is an EVS image, and the second image can include at least one pixel point. There is a mapping relationship between the second image and the pixel array. Each pixel point in the second image corresponds to a pixel in the pixel array.

[0107] In the embodiments of the present application, step S401 and step S402 can be executed in parallel or asynchronously. The embodiments of the present application do not limit the execution order of step S402 and step S401.

[0108] Based on the content described in the above step S401 and step S402, the electronic device can determine whether the first verification of the user's face authentication is passed by determining whether the first image is obtained. In a possible case, when the electronic device obtains the first image, that is, when the face recognition is successful, the electronic device can determine that the user has passed the first verification. After the first verification is passed, the electronic device can also perform a second verification on the user based on the first data obtained in step S402 to assist in determining the face recognition, so as to improve the security of face authentication. In this way, the electronic device can determine through the second verification that the user performing the face authentication is a live body rather than a synthesized video, thereby avoiding counterfeiting means such as video AI face swapping and providing the security of face authentication.

[0109] In some embodiments, the electronic device can perform a liveness detection on the user according to the first image and multiple second images to obtain a verification result.

[0110] In a possible solution, the electronic device can perform liveness detection by the method described in the following steps B1 to B3.

[0111] B1: The electronic device can establish a spatial mapping relationship between the pixel points in the first image and the pixel points in the second image.

[0112] In some embodiments, the electronic device can perform image registration on the first image and the second image to obtain a spatial mapping relationship between the pixel points in the first image and the pixel points in the second image. Exemplarily, the electronic device can perform feature extraction on the first image and the second image respectively to obtain the feature points in the first image and the feature points in the second image. The electronic device can determine the similarity between each feature point in the first image and each pixel point in the second image. The electronic device can determine the feature points in the first image that match the feature points in the second image according to the similarity. The electronic device can establish a spatial mapping relationship between the pixel points in the first image and the pixel points in the second image according to the pairs of mutually matching feature points in the first image and the second image. Among them, the process of the electronic device performing image registration on the first image and the second image is the same as the process of performing image registration in the prior art. In addition, since the pixel points in the second image correspond one by one to the pixels in the pixel array, the spatial mapping relationship between the first image and the second image can also be used to indicate the spatial mapping relationship between the first image and the pixel array.

[0113] B2: The electronic device can determine the core area in the second image according to the spatial mapping relationship. Among them, the core area is the ROI area.

[0114] In some examples, an electronic device may determine a core region according to a set face authentication policy. The face authentication policy is used to indicate actions that a user needs to perform during the authentication process. The face authentication policy includes, but is not limited to, the following data: the blinking frequency of human eyes, lip movement, and the relative movement between the electronic device and the face. The relative movement between the face and the electronic device may be turning the head, nodding, or other movements, which are not limited here. A person blinks 15 - 20 times per minute under normal circumstances. For example, when the face authentication policy includes a head-turning action, the electronic device may determine the core region as the user's face region. Another example is that when the face authentication policy includes a blinking action, the core region may be the eye region. Still another example is that when the face authentication policy does not specify the face part that needs to move, the core region may be the entire face region, or the eye region, or the lip region. Exemplarily, when the core region is the entire face region, the first object may be the face of the first user; when the core region is the eye region, the first object may be the eyes of the first user; when the core region is the lip region, the first object may be the lips of the first user; when the core region is other regions, the first object may also be other features of the first user.

[0115] The electronic device may perform image recognition on the first image to determine the first image region occupied by the face of the first user in the first image. The electronic device may determine the core region in the event image according to the pixel coordinates of the first image region and the spatial mapping relationship.

[0116] B3: The electronic device may perform liveness detection based on the core region of the second image to obtain a verification result.

[0117] The electronic device can determine whether there is a live body in the core area by detecting the continuity of pixel points in the time-domain spectrum in the core area. That is, the electronic device determines whether there is a live body in the core area by the continuity of the acquisition time of the event data carried by the pixel points in the core area in multiple second images in the time-domain spectrum. In this way, the electronic device can determine whether the user for authentication is a live body. Exemplarily, the electronic device can implement live body detection by detecting the motion continuity of pixel points in the core area. Moreover, during the process of performing live body detection on the core area, the electronic device does not need to perform feature extraction and recognition on the second image. It only needs to detect the continuity of the acquisition time of the event data carried by the pixel points in the core area in the time-domain spectrum to complete the detection. In addition, the electronic device can also implement live body detection through a pre-trained model based on neural networks (NN), convolutional neural networks (CNN), recurrent neural network (RNN), spiking neural network (SNN), and other neural networks, and can also use traditional computer vision (CV) algorithms to implement live body detection. Among them, the motion captured by the event camera is continuous, the time resolution can reach 1 us, and it has the characteristic of high frame rate (>5K fps). The frame rate of synthetic videos such as AI face swapping generally does not exceed 120 fps. Utilizing the ultra-high frame rate characteristic of the event camera can greatly improve the detection accuracy of the motion continuity of objects. The electronic device uses the core area of the second image captured by the event camera for live body detection, which can improve the security of face authentication. Exemplarily, the electronic device can perform live body detection in the following manner.

[0118] The electronic device can perform motion continuity detection on the core area of the second image within a set time period to obtain a verification result. Among them, the start time of the set time period can be the time when the electronic device obtains the first image, or a period of time before the time when the electronic device obtains the first image. Exemplarily, the motion continuity detection performed by the electronic device can be motion detection such as human eyes, lips, and the relative motion between the electronic device and the human face. The electronic device can obtain the live body detection result in the following manner.

[0119] In some embodiments, the electronic device may perform motion continuity detection on the core region in the second image within a set time period to obtain detection data. Among them, the detection data may include that the pixel points of the core region continuously appearing in multiple second images carry event data, that is, the detection data may represent the continuity in the time domain of the acquisition time of the event data of the pixels corresponding to the pixel points of the core region. Exemplarily, when the core region is the eye region, the electronic device may perform motion continuity detection on the event data carried by the pixel points in the eye region to obtain detection data. When the detection data indicates that the pixel points in the eye region continuously carry event data in multiple second images, the electronic device may determine that there is a first object that moves in the eye region. For example, when the pixel points in the eye region continuously carry event data in 5 second images, the electronic device may determine that there is a first object that moves in the eye region. That is, the first object may be a human eye. In this way, the electronic device may determine that the verification result is that a live body is detected. For another example, when the detection data indicates that there is no continuity in the time domain of the acquisition time of the event data of the pixels corresponding to the pixel points of the core region, the electronic device may determine that there is no first object that moves in the eye region. That is, the electronic device may determine that the verification result is that no live body is detected.

[0120] In another possible solution, the electronic device may also perform live body detection by the method described in the following steps C1 to C5.

[0121] C1: The electronic device may establish a spatial mapping relationship between the first image and the pixel array.

[0122] Optionally, the mapping relationship described in the embodiments of the present application may specifically be a mapping relationship between coordinate systems and / or coordinates. For example, the mapping relationship between the first image and the pixel array may be the mapping relationship between the coordinate system of the first image and the coordinate system of the pixel array, and any coordinate in the coordinate system of the first image corresponds to a coordinate in the coordinate system of the pixel array. Therefore, based on the mapping relationship between the first image and the pixel array, the corresponding positions of any one or any number of pixel points in the first image in the pixel array can be determined, and the corresponding positions of any one or any number of pixels in the pixel array in the first image can also be determined.

[0123] Among them, the process of establishing the spatial mapping relationship in step C1 is the same as the process of establishing the spatial mapping relationship in step B1 above, and will not be described in detail here.

[0124] C2: The electronic device may determine target pixels according to the first data. The target pixels include the pixels corresponding to the first object among multiple pixels, and the time points of the event data of the target pixels collected by the event camera include: consecutive multiple time points among multiple time points arranged in chronological order.

[0125] In some embodiments, the electronic device may select multiple sets of data from the first data according to the set time period in the chronological order of the collection time of the collected data. The electronic device may determine a target pixel according to the collection time of the event data of at least one pixel in the multiple sets of data. Exemplarily, when there are consecutive multiple sets of data in the multiple sets of data that respectively contain different event data of the same pixel, the electronic device may use the same pixel as the target pixel. For example, the five sets of data are data set 1, data set 2, data set 3, data set 4, and data set 5 respectively. The pixels corresponding to the event data in data set 1, data set 2, data set 3, data set 4, and data set 5 are: pixel 1, pixel 2, pixel 3, and pixel 4. The event data of pixel 1 is located in data set 1 and data set 5, the event data of pixel 2 is located in data set 1, data set 2, data set 3, data set 4, and data set 5, the event data of pixel 3 is located in data set 1, data set 2, and data set 3, and the event data of pixel 4 is located in data set 1, data set 2, and data set 3. In this case, the electronic device may use pixels 2, 3, and 4 as the target pixels.

[0126] C3: The electronic device may determine a target area occupied by the target pixel in the pixel array according to the pixel coordinates of the target pixel.

[0127] Wherein, the target area includes a first object, the first object corresponds to the target pixel, and the event data of the target pixel is determined according to the first object. Exemplarily, the electronic device may use the area composed of the target pixels as the target area.

[0128] C4: The electronic device may determine a second image area corresponding to the target area in the first image according to the spatial mapping relationship between the first image and the pixel array.

[0129] C5: The electronic device may determine a verification result according to the second image area and the first image area.

[0130] When the electronic device determines that there is no intersection between the first image area and the second image area, it may determine that the verification result is that the face of the first user does not include the first object. When the electronic device determines that there is an intersection between the first image area and the second image area, it may determine that the verification result is that the face of the first user includes the first object.

[0131] In some other embodiments, after the electronic device finishes executing C1 to C2, it can also obtain second data from the first data. The process by which the electronic device obtains the second data is the same as the process of determining the second data in the above steps B2 to B3, and will not be elaborated here. The electronic device can determine whether the second data includes event data of the target pixel to determine whether the second data includes event data corresponding to the first object. Exemplarily, when the second data includes event data of the target pixel, the electronic device can determine that the face of the first user includes the first object. For another example, when the second data does not include event data of the target pixel, the electronic device can determine that the face of the first user does not include the first object.

[0132] S403: When it is determined that the face of the first user includes the first object, the electronic device can display the first information and / or perform the first operation.

[0133] In a possible solution, when the electronic device determines that the verification result of the secondary verification is that the face of the first user includes the first object, it can determine that the face authentication result of the user is authentication success. When the electronic device determines that the face authentication result is authentication success, it can display the first information and / or perform the first operation. Exemplarily, the first information is used to indicate that the face authentication is successful. For example, the first information can be the information that the user wants to obtain, or the information that informs the user that the authentication is successful. The first operation is the operation to be performed when the face authentication is successful. For example, the first operation can be an operation to inform the user that the authentication is successful, or an operation to execute related functions to provide permissions for the user.

[0134] In a possible case, after the electronic device determines that the face authentication is successful, it can display the first information and / or perform the first operation to inform the user that the authentication is successful. For example, the electronic device can display the first information of successful authentication in the display interface to inform the user that the authentication is successful. For another example, the electronic device can also perform the first operation of voice broadcast to inform the user that the authentication is successful. When the electronic device displays the first information and / or the first operation, the electronic device can also execute related functions to let the user pass. For example, when the user wants to access the APP of the electronic device through face authentication, the electronic device can start the APP and display the application interface of the APP while informing the user that the authentication is successful.

[0135] In another possible case, after the electronic device determines that the face authentication is successful, it can also complete the execution of related functions by displaying the first message and / or performing the first operation. For example, taking the electronic device as a mobile phone, the user wants to access the APP through face authentication. After the electronic device determines that the user authentication is successful, it can directly start the APP and display the display interface of the APP in the display interface. For another example, when the electronic device is a building access control, after the electronic device determines that the user authentication is successful, it can perform the door opening operation.

[0136] In another possible scenario, when the electronic device determines that the verification result of the secondary verification shows that the face of the first user does not include the first object, it can determine that the face authentication result of the user is authentication failure and end the face authentication process. When the electronic device determines that the face authentication result is authentication failure, it can display the second information and / or perform the second operation. Exemplarily, the second information is used to indicate that the face authentication fails. The second operation is the operation to be performed when the face authentication fails. For example, the second operation can be an operation to inform the user of the authentication failure, or an operation to end the face authentication process. When the electronic device performs the second operation to inform the user of the authentication failure, the electronic device can also end the face authentication process. For example, taking the electronic device as a mobile phone, the user wants to access the APP through face authentication. After the electronic device determines that the user authentication fails, it can display the first information of authentication failure in the display interface to inform the user of the authentication failure. Another example is that when the electronic device is a building access control, the electronic device can also voice broadcast "verification failed" to inform the user of the authentication failure.

[0137] Based on the content shown in the above embodiments, the electronic device performs face recognition on the RGB image. While ensuring a high recognition rate, it is paired with an event camera for live detection. Utilizing the ultra-high frame rate characteristic of the event camera, it can greatly improve the detection accuracy of the motion continuity of the live face, thereby accurately detecting the authenticity of the live face and further enhancing the security of face authentication. In addition, the content shown in the above embodiments can also be applied to other scenarios of live detection to prevent the counterfeiting of synthetic videos.

[0138] As Figure 7 shown, the present application provides a face authentication use case. Among them, the core area of interest of the electronic device in this use case is the face area. As Figure 7 shown, this use case includes:

[0139] S701: The electronic device collects an image of the first spatial area through a camera to obtain an RGB image.

[0140] S702: The electronic device performs face recognition on the RGB image to determine the face area of the RGB image.

[0141] S703: The electronic device extracts features from the face area of the RGB image to obtain face features.

[0142] S704: The electronic device matches the face features with the preset face information to obtain a face detection result.

[0143] S705: The electronic device determines whether the face detection result is a successful detection; if so, it executes step S706; if not, it executes step S713.

[0144] S706: The electronic device uses the RGB image as the first image.

[0145] S707: The electronic device obtains a second image captured by the event camera for the first spatial region. The pixel points carrying event data in the second image are used to determine a first object that undergoes continuous motion in the first spatial region.

[0146] Exemplarily, the process of obtaining the second image in step S707 is the same as the process of obtaining the second image in step S302, and will not be elaborated here.

[0147] S708: The electronic device establishes a spatial mapping relationship between the pixel points in the first image and the pixel points in the second image.

[0148] S709: The electronic device determines a core region in the second image corresponding to the face region in the first image according to the spatial mapping relationship.

[0149] S710: The electronic device performs liveness detection on the core region to obtain a verification result.

[0150] Among them, the process of performing liveness detection in step S710 can refer to the content of the above steps B1 - B3 or C1 - C5, and will not be elaborated here.

[0151] S711: The electronic device determines whether the verification result is that the core region contains a live body; if so, it executes step S712; if not, it executes step S713.

[0152] S712: The electronic device determines that the authentication result is authentication success.

[0153] S713: The electronic device determines that the authentication result is authentication failure.

[0154] Based on Figure 7 the content shown above, the electronic device performs face recognition on the RGB image. While ensuring a high recognition rate, it cooperates with the event camera to perform liveness detection. Utilizing the ultra-high frame rate characteristic of the event camera, it can greatly improve the detection accuracy of the continuity of the live face movement, thereby accurately detecting the authenticity of the live face and further improving the security of face authentication.

[0155] Based on the above embodiments and the same technical concept, the embodiment of the present application further provides a face authentication method, as shown in the following steps D1 - D3, and the method may include:

[0156] D1: The electronic device obtains a first image. Among them, the first image is an image of the first spatial region, and the first image includes a set first user face.

[0157] Exemplarily, the first image may be the first image in the foregoing embodiments. Regarding the first image, reference may be made to the description of the first image in the foregoing embodiments, and details will not be repeated here.

[0158] D2: The electronic device acquires first data collected by the event camera for the first spatial region. The event camera includes a pixel array, and the pixel array includes a plurality of pixels. The first data is used to determine a first object that moves in the first spatial region.

[0159] Exemplarily, the first data may be the first data in the foregoing embodiments, and the first object may be the first object in the foregoing embodiments. Regarding the first data, reference may be made to the description of the first data in the foregoing embodiments, and details will not be repeated here. Regarding the first object, reference may be made to the description of the first object in the foregoing embodiments, and details will not be repeated here.

[0160] Among them, there is no order of execution between D1 and D2.

[0161] D3: The electronic device determines whether the face of the first user includes the first object according to the first image and the first data. When the electronic device determines that the face of the first user includes the first object, the electronic device executes step D4; or, when the electronic device determines that the face of the first user does not include the first object, the electronic device executes step D5.

[0162] In some embodiments of the present application, as an alternative implementation, the electronic device may determine a first image area occupied by the face of the first user in the first image, and determine second data corresponding to the first image area in the first data. The electronic device may determine that the face of the first user includes the first object according to the second data. Exemplarily, the electronic device may determine a first target area corresponding to the first image area in the pixel array according to the mapping relationship between the first image and the pixel array; the electronic device may select second data from the first data according to the first target area; among them, the second data includes event data of pixels in the first target area among the plurality of pixels. The first target area may be the core area in the foregoing embodiments. Regarding the first target area, reference may be made to the description of the core area in the foregoing embodiments, and details will not be repeated here. When the electronic device determines that the second data includes event data corresponding to the first object, it determines that the face of the first user includes the first object. Regarding the specific implementation manner of this method, reference may be made to the method for the electronic device to perform live detection on the core area described in the foregoing embodiments, and details will not be elaborated here.

[0163] Optionally, when the electronic device determines that the second data does not include event data corresponding to the first object, it determines that the face of the first user does not include the first object.

[0164] Optionally, after determining the second data, the electronic device may also determine the event data corresponding to the target pixel from the first data. Among them, the target pixel includes the pixels corresponding to the first object among multiple pixels, and the target pixel may be the target pixel described in the foregoing embodiments, which will not be elaborated here. The electronic device may determine whether the first user's face includes the first object by determining whether the second data includes the event data corresponding to the target pixel. If the second data includes the event data corresponding to the target pixel, the first user's face includes the first object; or, if the second data does not include the event data corresponding to the target pixel, the first user's face does not include the first object. The specific implementation manner of this method may refer to the related methods described in the foregoing embodiments and will not be elaborated here.

[0165] As another optional implementation manner, the electronic device may determine the first image area occupied by the first user's face in the first image; the electronic device may also determine the target pixel according to the first data; the electronic device may determine that the first user's face includes the first object according to the first image and the target pixel; among them, the first image area is the first image area described in the foregoing embodiments and will not be elaborated here. Exemplarily, the electronic device may also divide the first data into multiple groups of data in the order of the time of collecting the data; among the multiple groups of data, each group of data includes at least one of the multiple event data, and the time points when the event camera collects at least one event data are the same. The multiple groups of data are the multiple groups of data described in the foregoing embodiments and will not be elaborated here. When the electronic device determines that there are consecutive multiple groups of data in the multiple groups of data that respectively include different event data of the same pixel, the same pixel is used as the target pixel. The electronic device may also determine the second image area corresponding to the second target area in the pixel array in the first image according to the mapping relationship between the first image and the pixel array; among them, the second target area is the area occupied by the target pixel, and the second target area is the target area described in the foregoing embodiments and will not be elaborated here. When the electronic device determines that there is an intersection between the first image area and the second image area, it determines that the first user's face includes the first object. Regarding the specific implementation manner of this method, reference may be made to the method for live detection by the electronic device through C1 to C5 described in the foregoing embodiments, which will not be elaborated here.

[0166] Optionally, when there is an intersection between the first image area and the second image area, the electronic device may determine that the first user's face does not include the first object.

[0167] In some embodiments of the present application, the first object includes but is not limited to at least one of the following: the face of the first user; the eyes of the first user; the lips of the first user.

[0168] D4: If the electronic device determines that the face of the first user includes the first object based on the first image and the first data, the electronic device displays the first information and / or performs the first operation. Wherein, the first information is used to indicate that the face authentication is successful, and the first operation is the operation to be performed when the face authentication is successful. The first information is the first information described in the foregoing embodiments, and the first operation is the first operation described in the foregoing embodiments, which will not be elaborated herein.

[0169] D5: If the electronic device determines that the face of the first user does not include the first object based on the first image and the first data, the electronic device displays the second information and / or performs the second operation. Wherein, the second information is used to indicate that the face authentication fails, and the second operation is the operation to be performed when the face authentication fails. The second information is the second information described in the foregoing embodiments, and the second operation is the second operation described in the foregoing embodiments, which will not be elaborated herein.

[0170] In some embodiments of the present application, before obtaining the first image, the electronic device may further obtain a second image. Wherein, the second image is the RGB image described in the foregoing embodiments, which will not be elaborated herein. When the electronic device determines that the second image does not include the face of the first user, it displays the second information and / or performs the second operation. The specific implementation manner of this method may refer to the relevant methods described in the foregoing embodiments, which will not be elaborated herein.

[0171] In the above method, the specific steps executed by the electronic device can refer to the relevant introductions in the foregoing embodiments, and will not be elaborated herein.

[0172] Based on the above content and the same technical concept, the present application provides an electronic device, including a memory and one or more processors. The memory is used to store computer program code, and the computer program code includes computer instructions; the one or more processors are used to execute the computer program instructions stored in the memory, so that the electronic device executes the steps executed by the electronic device in the above method embodiments.

[0173] Optionally, the electronic device may further include a camera, and the camera may be used to collect images.

[0174] Optionally, the electronic device may further include an event camera, and the event camera may be used to detect an object in motion in a spatial region.

[0175] Based on the above embodiments and the same technical concept, an embodiment of the present application further provides an electronic device, and the electronic device is used to implement the face authentication method provided by the embodiment of the present application. As Figure 8As shown in the figure, the electronic device 800 may include: a memory 801, one or more processors 802, and one or more computer programs (not shown in the figure). Each of the above components may be coupled via one or more communication buses 803. Optionally, the electronic device 800 may further include a display screen 804. Optionally, the electronic device may further include a camera, which can be used to capture images.

[0176] Optionally, the electronic device may further include an event camera, which can be used to detect objects that are in motion in a spatial region.

[0177] Among them, one or more computer programs (codes) are stored in the memory 801, and the one or more computer programs include computer instructions; the one or more processors 802 call the computer instructions stored in the memory 801, so that the electronic device 800 executes the face authentication method provided in the embodiments of the present application.

[0178] In a specific implementation, the memory 801 may include a high-speed random access memory, and may also include a non-volatile memory, such as one or more disk storage devices, flash memory devices, or other non-volatile solid-state storage devices. The memory 801 may store an operating system (hereinafter referred to as the system), such as an embedded operating system like ANDROID, IOS, WINDOWS, or LINUX. The memory 801 may be used to store the implementation program of the embodiments of the present application. The memory 801 may also store a network communication program, which may be used to communicate with one or more additional devices, one or more user devices, and one or more network devices.

[0179] One or more processors 802 may be a general-purpose central processing unit (CPU), a microprocessor, an application-specific integrated circuit (ASIC), or one or more integrated circuits for controlling the execution of the program of the solution of the present application.

[0180] The display screen 804 is used to display relevant user interfaces such as application interfaces.

[0181] It should be noted that Figure 8 is merely one implementation manner of the electronic device 800 provided in the embodiments of the present application. In practical applications, the electronic device 800 may further include more or fewer components. Specifically, reference may be made to Figure 2 the specific structure and description shown, which are not limited here.

[0182] Based on the above content and the same concept, the present application provides a computer-readable storage medium. A computer program or instruction is stored in the computer-readable storage medium. When the computer program or instruction is executed by a computing device, the computing device executes the steps executed by the electronic device in the above method embodiment.

[0183] Based on the above content and the same concept, the present application provides a computer program product. The computer program product includes a computer program or instruction. When the computer program or instruction is executed by a computing device, the computing device executes the steps executed by the electronic device in the above method embodiment.

[0184] Those skilled in the art should understand that the embodiments of the present application can be provided as a method, a system, or a computer program product. Therefore, the present application can take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware aspects. Moreover, the present application can take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0185] The present application is described with reference to the flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to the present application. It should be understood that each flow and / or block in the flowchart and / or block diagram, as well as the combination of flows and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to the processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing devices to generate a machine, such that the instructions executed by the processor of the computer or other programmable data processing devices generate a device for implementing the specified functions in Figure 1 one flow or multiple flows and / or blocks Figure 1 one block or multiple blocks.

[0186] These computer program instructions can also be stored in a computer-readable memory that can direct a computer or other programmable data processing device to work in a specific manner, such that the instructions stored in the computer-readable memory generate a manufactured article including an instruction device, and the instruction device implements the specified functions in Figure 1 one flow or multiple flows and / or blocks Figure 1 one block or multiple blocks.

[0187] These computer program instructions can also be loaded onto a computer or other programmable data processing device, such that a series of operation steps are executed on the computer or other programmable device to generate a computer-implemented process. Thus, the instructions executed on the computer or other programmable device provide for implementing the specified functions in Figure 1 one flow or multiple flows and / or blocksFigure 1 Steps of functions specified in one or more boxes.

[0188] Obviously, those skilled in the art can make various changes and modifications to this application without departing from the spirit and scope of this application. Thus, if these modifications and variations of this application fall within the scope of the claims of this application and their equivalent technologies, this application is also intended to include these changes and modifications.

Claims

1. A face authentication method, characterized in that, The method includes: Obtaining a first image, where the first image is an image of a first spatial region and includes the face of a set first user; Obtaining first data collected by an event camera for the first spatial region; wherein the first data is used to determine a first object that moves in the first spatial region; If it is determined according to the first image and the first data that the face of the first user includes the first object, display first information and / or perform a first operation.

2. The method according to claim 1, wherein The method further includes: If it is determined according to the first image and the first data that the face of the first user does not include the first object, display second information and / or perform a second operation.

3. The method according to claim 1 or 2, characterized in that, The event camera includes a pixel array, and the pixel array includes a plurality of pixels; the first data includes a plurality of event data collected at a plurality of time points, and the plurality of event data includes: one or more event data of each pixel in the plurality of pixels.

4. The method according to claim 3, wherein The determining that the face of the first user includes the first object according to the first image and the first data includes: Determining a first image region occupied by the face of the first user in the first image; Determining second data corresponding to the first image region in the first data, and determining that the face of the first user includes the first object according to the second data.

5. The method according to claim 4, wherein The determining the second data corresponding to the first image region in the first data includes: Determining a first target region corresponding to the first image region in the pixel array according to the mapping relationship between the first image and the pixel array; Selecting the second data from the first data according to the first target region; wherein the second data includes the event data of the pixels in the first target region among the plurality of pixels.

6. The method according to claim 4 or 5, characterized in that, The determining that the face of the first user includes the first object according to the second data includes: Determining that the face of the first user includes the first object when it is determined that the second data includes event data corresponding to the first object.

7. The method according to claim 3, wherein The determining that the face of the first user includes the first object according to the first image and the first data includes: Determining a first image region occupied by the face of the first user in the first image; Determining target pixels according to the first data; wherein the target pixels include the pixels corresponding to the first object among the plurality of pixels; Determining that the face of the first user includes the first object according to the first image region and the target pixels.

8. The method according to claim 7, characterized in that The time points when the event camera collects the event data of the target pixels include: a plurality of consecutive time points among the plurality of time points arranged in chronological order.

9. The method according to claim 7 or 8, characterized in that, The determining the target pixels according to the first data includes: Dividing the first data into multiple groups of data in chronological order of data collection; wherein, in the multiple groups of data, each group of data includes at least one of the plurality of event data, and the time points when the event camera collects the at least one event data are the same; When it is determined that there are consecutive multiple sets of data in the multiple sets of data that respectively include different event data of the same pixel, the same pixel is used as the target pixel.

10. The method according to any one of claims 7 to 9, characterized in that The determining that the face of the first user includes the first object according to the first image area and the target pixel includes: Determining a second image area corresponding to a second target area in the pixel array in the first image according to the mapping relationship between the first image and the pixel array; wherein, the second target area is the area occupied by the target pixel; When it is determined that there is an intersection between the first image area and the second image area, it is determined that the face of the first user includes the first object.

11. The method according to any one of claims 1 to 10, characterized in that, The first object includes at least one of the following: The face of the first user; The eyes of the first user; The lips of the first user.

12. The method according to any one of claims 1 to 11, characterized in that, The method further includes: Obtaining a second image, where the second image is an image of the first spatial area; When it is determined that the face of the first user is not included in the second image, displaying second information and / or performing a second operation.

13. An electronic device, characterized in that, Including: One or more processors; one or more memories; The one or more memories are used to store one or more computer programs and data information; wherein the one or more computer programs include instructions; When the instructions are executed by the one or more processors, the terminal device is caused to execute the method according to any one of claims 1 to 12.

14. A computer-readable storage medium, characterized in that, The computer-readable storage medium is used to store a computer program, and when the computer program runs on a computer, the computer is caused to execute the method according to any one of claims 1 to 12.

Images