Face recognition method and electronic device

Abstract

Embodiments of the present application provide a face recognition method and an electronic device. The face recognition method can be applied to an electronic device comprising a TOF camera. The method comprises the following steps: controlling the TOF camera to capture a first image frame based on a first exposure time; in the case that the brightness of the first image frame is greater than a preset brightness threshold, determining a second exposure time, the second exposure time being less than the first exposure time; and controlling the TOF camera to capture a second image frame based on the second exposure time. The method provided by the embodiments of the present application reduces the exposure time of the TOF camera in the case that the brightness of the first image frame captured by the TOF is greater than the preset brightness threshold, thereby reducing the time for which the light signal is irradiated onto the human skin, reducing the energy of the light signal absorbed by the skin, avoiding damage to the human skin caused by the light signal, and protecting the skin safety.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of terminal, and particularly relates to a face recognition method and an electronic device. BACKGROUND

[0002] A time of flight (TOF) camera module can include a transmitter (Tx) and a receiver (Rx), wherein the transmitter is used to emit infrared light or laser pulses, and the receiver is used to receive reflected light and form an image. In a face recognition scenario, the TOF camera can collect depth information of a face, so as to better recognize the three-dimensional structure of the face. Therefore, compared with a traditional 2D image, the image collected by the TOF camera module can enhance the uniqueness of the face, thereby improving the accuracy of recognition.

[0003] When the TOF camera collects an image, the transmitter needs to project a light signal to the face, that is, the light signal projected by the transmitter will irradiate on the skin of the face and the like, and therefore, it is necessary to avoid that the light signal projected by the transmitter causes harm to the skin of the person. SUMMARY

[0004] The face recognition method and the electronic device provided by the embodiments of the present application can ensure that the light signal emitted by the TOF does not cause harm to the skin of the person, protect the safety of the skin, and thereby improve the user experience.

[0005] In a first aspect, the embodiments of the present application provide a face recognition method, which can be applied to an electronic device containing a TOF camera, and the method comprises the following steps: controlling the TOF camera to collect a first image frame based on a first exposure time; in a case where the brightness of the first image frame is greater than a preset brightness threshold, determining a second exposure time, the second exposure time being less than the first exposure time; and controlling the TOF camera to collect a second image frame based on the second exposure time.

[0006] The method provided by the embodiments of the present application reduces the exposure time of the TOF camera in a case where the brightness of the first image frame collected by the TOF is greater than the preset brightness threshold, that is, from a face ID mode to a skin safety mode, thereby reducing the time length of the light signal irradiating on the skin of the person, reducing the energy of the light signal absorbed by the skin, avoiding that the light signal causes harm to the skin of the person, and protecting the safety of the skin.

[0007] In an implementation, in a case where the brightness of the first image frame is greater than a preset brightness threshold, the second exposure duration is determined, including: determining a first brightness of the first image frame, the first brightness being an average brightness of all pixels in a grayscale image corresponding to the first image frame; and in a case where the first brightness is greater than the brightness threshold, determining the second exposure duration. In this way, the electronic device can perform skin safety detection based on the brightness of the grayscale image corresponding to the first image frame, and no longer rely on the PD value of the receiver in the Tx, so that the electronic device can also achieve skin safety detection when equipped with the TOF camera with a baffle, and the adaptability of the face recognition method is improved. Figure 6

[0008] In an implementation, in a case where the first brightness is greater than the brightness threshold, the second exposure duration is determined, including: in a case where the first brightness of n first image frames continuously collected by the TOF camera is greater than the brightness threshold, determining the second exposure duration, n being a positive integer greater than 1. When the first brightness of the first image frame is greater than the brightness threshold and lasts for a certain number of frames, it indicates that the TOF camera may be continuously directed at the user, and thus there is a skin safety risk. In this case, the electronic device reduces the exposure duration of the TOF camera, thereby facilitating improvement of the accuracy of entering the skin safety mode.

[0009] In an implementation, after the TOF camera is controlled to collect the second image frame based on the second exposure duration, the method further includes: in a case where the brightness of the second image frame is less than the brightness threshold, controlling the TOF camera to collect image frames based on the first exposure duration. When the brightness of the second image frame is less than the brightness threshold, it indicates that the skin safety risk is eliminated, and thus the electronic device can control the TOF camera to return to collecting image frames based on the first exposure duration, thereby obtaining image frames containing more information, and facilitating improvement of the speed of face recognition.

[0010] In an implementation, the TOF camera includes a transmitter and a receiver; the first exposure duration includes a first depth exposure duration of the transmitter and a first grayscale exposure duration of the receiver, and the second exposure duration includes a second depth exposure duration of the transmitter and a second grayscale exposure duration of the receiver; the second depth exposure duration is less than the first depth exposure duration, and the second grayscale exposure duration is less than the first grayscale exposure duration. In this way, in the skin safety mode, the intensity of the signal light emitted by the TOF camera can remain unchanged compared with the face ID mode, but the duration of the emitted and received light signals is shortened. In this way, the energy of the light signals absorbed by the skin is reduced to protect the skin safety, the face recognition can be normally performed, and the power consumption during face recognition is also reduced, achieving three goals at once.

[0011] ​In an implementation, the electronic device includes a camera hardware abstraction layer, and the camera hardware abstraction layer controls the TOF camera to capture the first image frame based on the first exposure time, including: the camera hardware abstraction layer configures the TOF camera with first configuration parameters corresponding to a first working mode (i.e., a face ID mode), and the first configuration parameters include the first exposure time and a first working current of the TOF camera; and the TOF camera captures the first image frame based on the first configuration parameters. In this way, the TOF camera can work in the face ID mode based on the first configuration parameters to capture images.

[0012] In an implementation, in a case where the brightness of the first image frame is greater than a preset brightness threshold, the second exposure time is determined, including: in a case where the brightness of the first image frame is greater than the brightness threshold, the camera hardware abstraction layer configures the TOF camera with second configuration parameters corresponding to a second working mode, and the second configuration parameters include the second exposure time and the first working current; and the camera hardware abstraction layer controls the TOF camera to capture the second image frame based on the second exposure time, including: the TOF camera captures the second image frame based on the second configuration parameters. In this way, the TOF camera can work in the skin safety mode based on the second configuration parameters to capture images.

[0013] In an implementation, the electronic device includes a face trust application, and after the camera module captures the first image frame based on the first configuration parameters, the face trust application further obtains a grayscale image corresponding to the first image frame; the face trust application determines a first brightness based on the grayscale image corresponding to the first image frame; and the face trust application obtains a detection result based on the first brightness and the brightness threshold, where, in a case where the first brightness of n frames of the first image frame continuously captured by the TOF camera are all greater than the brightness threshold, the face trust application obtains a first detection result, and in a case where the first brightness of n frames of the first image frame continuously captured by the TOF camera are not all greater than the brightness threshold, the face trust application obtains a second detection result. In this way, the face trust application can obtain different skin safety detection results based on different numerical relationships between the first brightness and the brightness threshold, thereby providing a reference basis for the TOF camera to switch to the skin safety mode.

[0014] In an implementation, the electronic device includes a face recognition control module, and after the face trust application obtains the detection result based on the first brightness and the brightness threshold, the face trust application further sends the detection result to the face recognition control module, and the face recognition control module sends the detection result to the camera hardware abstraction layer.

[0015] In an implementation manner, in a case that the brightness of the first image frame is greater than the brightness threshold, the camera hardware abstraction layer configures the second configuration parameter corresponding to the second working mode for the TOF camera, including: in a case that the camera hardware abstraction layer receives the first detection result, the camera hardware abstraction layer configures the second configuration parameter corresponding to the second working mode for the TOF camera. In this way, the camera hardware abstraction layer can switch the working mode of the TOF camera to the skin safety mode based on the received skin safety detection result.

[0016] In an implementation manner, after the TOF camera collects the second image frame based on the second configuration parameter, the face trust application further includes: the face trust application obtains a gray image corresponding to the second image frame; the face trust application determines a second brightness of the second image frame based on the gray image corresponding to the second image frame, the second brightness being an average value of brightnesses of all pixels in the gray image corresponding to the second image frame; the face trust application obtains a comparison result of the second brightness and the brightness threshold, wherein, in a case that the second brightness is less than the brightness threshold, the face trust application obtains a first comparison result, and in a case that the second brightness is greater than or equal to the brightness threshold, the face trust application obtains a second comparison result. In this way, the face trust application can obtain different comparison results based on different numerical relationships between the second brightness and the brightness threshold, thereby providing a reference basis for whether the TOF camera switches to the face ID mode.

[0017] In an implementation manner, after the face trust application obtains the comparison result of the second brightness and the brightness threshold, the face trust application further includes: the face trust application sends the comparison result to the face recognition control module; the face recognition control module sends the comparison result to the camera hardware abstraction layer; in a case that the camera hardware abstraction layer receives the first comparison result, the camera hardware abstraction layer configures the first configuration parameter corresponding to the first working mode for the TOF camera. In this way, the camera hardware abstraction layer can switch the working mode of the TOF camera to the face ID mode based on the received comparison result.

[0018] In an implementation manner, the first image frame and the second image frame are both used for face recognition. In the embodiment of the present application, the second image frame collected in the face ID mode and the skin safety mode can both be used for face recognition, so that even if the TOF camera works in the skin safety mode, the normal face recognition process will not be affected, and the user experience is improved.

[0019] In a second aspect, the embodiment of the present application provides an electronic device, including a memory, a TOF camera and one or more processors; the memory, the TOF camera and the processor are coupled; wherein the memory stores computer program code, the computer program code includes computer instructions, when the computer instructions are executed by the processor, the electronic device executes the face recognition method provided by the first aspect and any possible design manner thereof.

[0020] In a third aspect, an embodiment of the present application provides a computer readable storage medium, which includes computer instructions, when the computer instructions are executed on an electronic device, cause the electronic device to perform the face recognition method according to the first aspect and any possible design of the first aspect.

[0021] In a fourth aspect, an embodiment of the present application provides a computer program product, when the computer program product is executed on an electronic device, cause the electronic device to perform the face recognition method according to the first aspect and any possible design of the first aspect.

[0022] It can be understood that the beneficial effects achieved by the technical solutions provided in the second aspect to the fifth aspect can refer to the beneficial effects in the first aspect and any possible design of the first aspect, which will not be repeated here. BRIEF DESCRIPTION OF DRAWINGS

[0023] Figure 1 FIG. 1 is a structural schematic diagram of an electronic device 100 provided by an embodiment of the present application;

[0024] Figure 2 FIG. 2 is a schematic diagram of the principle of a TOF imaging technology provided by an embodiment of the present application;

[0025] Figure 3 FIG. 3 is a software architecture diagram of an electronic device provided by an embodiment of the present application;

[0026] Figure 4 FIG. 4 is a schematic diagram of the interaction between software modules involved in the face recognition method provided by an embodiment of the present application;

[0027] Figure 5 FIG. 5 is a structural schematic diagram of Tx of a TOF camera provided by an embodiment of the present application;

[0028] Figure 6 FIG. 6 is a structural schematic diagram of Tx of another TOF camera provided by an embodiment of the present application;

[0029] Figure 7 FIG. 7 is another schematic diagram of the interaction between software modules involved in the face recognition method provided by an embodiment of the present application;

[0030] Figure 8 FIG. 8 is a flowchart of a face recognition method provided by an embodiment of the present application;

[0031] Figure 9 FIG. 9 is another flowchart of a face recognition method provided by an embodiment of the present application;

[0032] Figure 10 FIG. 10 is an interaction schematic diagram of face unlocking provided by an embodiment of the present application;

[0033] Figure 11 is another interaction schematic diagram of face unlocking provided by an embodiment of the present application;

[0034] Figure 12 is still another interaction schematic diagram of face unlocking provided by an embodiment of the present application;

[0035] Figure 13 is still another flowchart of a face recognition method provided by an embodiment of the present application;

[0036] Figure 14 is still another flowchart of a face recognition method provided by an embodiment of the present application;

[0037] Figure 15 is a schematic diagram of a chip system provided by an embodiment of the present application. DETAILED DESCRIPTION

[0038] The technical solutions in the embodiments of the present application will be described below with reference to the drawings in the embodiments of the present application. In the description of the present application, unless otherwise specified, “at least one” refers to one or more, and “a plurality of” refers to two or more than two. In addition, in order to clearly describe the technical solutions in the embodiments of the present application, in the embodiments of the present application, the same items or similar items with basically the same functions and effects are distinguished by using “first”, “second”, etc. The skilled in the art can understand that “first”, “second”, etc. do not limit the quantity and execution order, and “first”, “second”, etc. also do not necessarily mean different.

[0039] In order to make the description of each embodiment below clear and simple, first, a brief introduction of related concepts or technologies is given:

[0040] The rich execution environment (REE), which can also be referred to as a rich execution environment or a normal execution environment or an untrusted execution environment, refers to a system running environment of a mobile terminal, in which an operating system such as Android, IOS and Linux can be run. The REE has good openness and expansibility but low security.

[0041] A trusted execution environment (TEE), which can also be referred to as a secure side or a secure area, is an area that requires authorization to access. The TEE coexists with the REE in the running environment in the electronic device, and is isolated from the REE by the support of hardware, has security capabilities and can resist software attacks that the regular REE side is vulnerable to. The TEE has its own running space and defines strict protection measures, and thus has a higher security level than the REE, and can protect assets (such as data, software, etc.) in the TEE from software attacks and resist specific types of security threats.

[0042] The REE+TEE architecture refers to an architecture in which the TEE and the REE are combined to jointly provide services for an application. That is, the TEE and the REE coexist in the electronic device. For example, the TEE can implement an isolated running mechanism from the REE by the support of hardware. The TEE has its own running space and has a higher security level than the REE, and can protect assets (such as data, software, etc.) in the TEE from software attacks. Only authorized security software can execute in the TEE, and it also protects the confidentiality of the resources and data of the security software. Compared with the REE, the TEE can better protect the security of data and resources due to its protection mechanisms such as isolation and permission control.

[0043] The TA, i.e., the trusted application, is an application running in the TEE and can provide security services for the CA running outside the TEE, such as entering a password, generating a transaction signature, face recognition, etc.

[0044] The CA, i.e., the client application. The CA generally refers to an application running in the REE. The CA can call the TA through a client (Client) application programming interface (application programming interface, API) and instruct the TA to perform a corresponding security operation.

[0045] Software development kit (software development kit, SDK): In a broad sense, it refers to a collection of related documents, examples and tools that assist in the development of a certain type of software.

[0046] RAW Data, i.e., raw data, can be understood as "unprocessed and uncompressed data". In the embodiments of the present application, the RAW Data can refer to the original image data converted by the TOF camera from the light source signal captured. Some metadata (Metadata) generated by the camera shooting are also recorded in the RAW Data.

[0047] Metadata, also known as meta-data, intermediate data or relay data, is data about data, mainly information describing data properties. In the embodiments of the present application, Metadata can indicate the working mode of the camera, the light current value, the working state of the TOF camera device, the exposure value and other information.

[0048] Time of flight (TOF) imaging technology refers to a technology of emitting a group of infrared light (or laser pulse) that cannot be seen by human eyes, reflecting after encountering an object, ending at the camera, calculating the time difference or phase difference from emission to reflection to the camera, and collecting data to form a group of distance depth data, thereby obtaining a three-dimensional 3D model. That is, the TOF imaging technology adds depth information from the Z-axis direction on the basis of the traditional 2D XY-axis imaging, and finally generates 3D image information. In the embodiments of the present application, the TOF technology can refer to indirect TOF (iToF) technology.

[0049] The TOF camera can include a transmitter (Tx) and a receiver (Rx), wherein the transmitter is used to emit infrared light or laser pulse, and the receiver is used to receive reflected light and image. In the face recognition scene, the TOF camera can collect the depth information of the face, so as to better recognize the three-dimensional structure of the face. Therefore, compared with the traditional 2D image, the image collected by the TOF camera module contains depth information, which can enhance the uniqueness of the face, thereby improving the accuracy of recognition.

[0050] When the TOF camera collects images, the transmitter needs to project light signals and the like to the face, that is, the light signals and the like projected by the transmitter will irradiate the skin of the face and the like, therefore, it is necessary to avoid the light signals projected by the transmitter from causing harm to the skin of the person.

[0051] The embodiments of the present application provide a face recognition method, which can avoid the light signals projected by the transmitter from causing harm to the skin of the person, and can improve the user experience.

[0052] Figure 1 A structural schematic diagram of an electronic device 100 provided in the embodiments of the present application is shown.

[0053] As Figure 1As shown, the electronic device 100 can include a processor 110, an external memory interface 120, an internal memory 121, a universal serial bus (USB) interface 130, a charging management module 140, a power management module 141, a battery 142, an antenna 1, an antenna 2, a mobile communication module 150, a wireless communication module 160, an audio module 170, a speaker 170A, a receiver 170B, a microphone 170C, a headset jack 170D, a sensor module 180, a key 190, a motor 191, an indicator 192, a camera 193, a display screen 194, and a subscriber identification module (SIM) card interface 195, etc.

[0054] The processor 110 can include one or more processing units, for example: the processor 110 can include an application processor (AP), a modem processor, a graphics processing unit (GPU), an image signal processor (ISP), a controller, a memory, a video codec, a digital signal processor (DSP), a baseband processor, and / or a neural-network processing unit (NPU), etc. Among them, different processing units can be independent devices, or can be integrated in one or more processors.

[0055] The controller can be the nerve center and command center of the electronic device 100. The controller can generate operation control signals according to instruction operation codes and timing signals, complete the control of fetching instructions and executing instructions.

[0056] The NPU is a neural-network (NN) computing processor, which can quickly process input information by drawing on the structure of a biological neural network, for example, by drawing on the transmission mode between human brain neurons, and can also constantly self-learn. Through the NPU, the electronic device 100 can realize intelligent cognitive applications such as image recognition, face recognition, voice recognition, text understanding, etc.

[0057] Processor 110 may also include a memory for storing instructions and data. In some embodiments, the memory in processor 110 is a cache memory. This memory can store instructions or data that have just been used or are being recycled by processor 110. If processor 110 needs to use the same instruction or data again, it can directly access the memory. This avoids duplicate accesses, reduces processor 110 latency, and thus improves system efficiency.

[0058] In some embodiments, the processor 110 may include one or more interfaces. The interfaces may include an inter-integrated circuit (I2C) interface, an inter-integrated circuits sound (I2S) interface, a pulse code modulation (PCM) interface, a universal asynchronous receiver / transmitter (UART) interface, a mobile industry processor interface (MIPI), a general-purpose input / output (GPIO) interface, a subscriber identity module (SIM) interface, and / or a universal serial bus (USB) interface.

[0059] It is understood that the interface connection relationship between the modules illustrated in this embodiment is merely an illustrative illustration and does not constitute a structural limitation on the electronic device 100. In other embodiments, the electronic device 100 may also adopt different interface connection methods from the above embodiments, or a combination of multiple interface connection methods.

[0060] The external memory interface 120 can be configured to connect an external memory card, such as a micro SD card, to extend the memory capacity of the electronic device 100. The external memory card can communicate with the processor 110 via the external memory interface 120 to store data. For example, the external memory card can store files such as music, videos, and the like.

[0061] The charging management module 140 can be configured to receive a charging input from a charger. The charging management module 140 can supply power to the electronic device via the power management module 141 while charging the battery 142.

[0062] The power management module 141 can be configured to connect the battery 142, the charging management module 140, and the processor 110. The power management module 141 can receive power from the battery 142 and / or the charging management module 140 to supply power to the processor 110, the internal memory 121, the external memory, the display 194, the camera 193, and the wireless communication module 160. In some embodiments, the power management module 141 can be disposed in the processor 110. In some embodiments, the power management module 141 and the charging management module 140 can be disposed in the same device.

[0063] The wireless communication function of the electronic device 100 can be implemented via the antenna 1, the antenna 2, the mobile communication module 150, the wireless communication module 160, a modem processor, and a baseband processor.

[0064] The antenna 1 and the antenna 2 can be configured to transmit and receive electromagnetic wave signals. Each antenna in the electronic device 100 can be configured to cover a single or multiple communication bands. Different antennas can also be multiplexed to improve the utilization of the antennas. For example, the antenna 1 can be multiplexed as a diversity antenna for a wireless local area network.

[0065] The mobile communication module 150 can provide a solution for wireless communication including 2G / 3G / 4G / 5G, etc. applied to the electronic device 100. The mobile communication module 150 can include at least one filter, a switch, a power amplifier, a low noise amplifier (LNA), etc. The mobile communication module 150 can receive electromagnetic waves by the antenna 1, and perform filtering, amplification, etc. on the received electromagnetic waves, and transfer the processed signals to the modem processor for demodulation. The mobile communication module 150 can also amplify the signals modulated by the modem processor, and radiate the signals as electromagnetic waves through the antenna 1.

[0066] The modem processor can include a modulator and a demodulator. The modulator is configured to modulate a low-frequency baseband signal to be transmitted into a medium-high frequency signal. The demodulator is configured to demodulate a received electromagnetic wave signal into a low-frequency baseband signal. The demodulator then transmits the demodulated low-frequency baseband signal to the baseband processor for processing. The low-frequency baseband signal processed by the baseband processor is transmitted to the application processor. The application processor outputs a sound signal through an audio device (not limited to the speaker 170A, the microphone 170B, etc.), or displays an image or a video through the display screen 194.

[0067] The wireless communication module 160 can provide a solution for wireless communication including wireless local area networks (WLAN) (e.g., wireless fidelity (Wi-Fi) network), Bluetooth (BT), global navigation satellite system (GNSS), frequency modulation (FM), near field communication (NFC), infrared (IR) technology, etc. applied to the electronic device 100. The wireless communication module 160 can be one or more devices integrated with at least one communication processing module. The wireless communication module 160 receives electromagnetic waves via the antenna 2, performs frequency modulation and filtering on the electromagnetic wave signals, and transmits the processed signals to the processor 110. The wireless communication module 160 can also receive signals to be transmitted from the processor 110, perform frequency modulation and amplification on the signals, and radiate the signals as electromagnetic waves through the antenna 2.

[0068] In some embodiments, the antenna 1 and the mobile communication module 150 of the electronic device 100 are coupled, and the antenna 2 and the wireless communication module 160 are coupled, so that the electronic device 100 can communicate with a network and other devices through wireless communication technology. The wireless communication technology can include global system for mobile communications (GSM), general packet radio service (GPRS), code division multiple access (CDMA), wideband code division multiple access (WCDMA), time-division code division multiple access (TD-SCDMA), long term evolution (LTE), BT, GNSS, WLAN, NFC, FM, and / or IR technology, etc. The GNSS can include global positioning system (GPS), global navigation satellite system (GLONASS), beidu navigation satellite system (BDS), quasi-zenith satellite system (QZSS), and / or satellite based augmentation systems (SBAS).

[0069] The electronic device 100 can implement an audio function through an audio module 170, a speaker 170A, a receiver 170B, a microphone 170C, a headset interface 170D, and an application processor, etc. For example, music playing, recording, etc.

[0070] The audio module 170 is used to convert digital audio information into analog audio signals output, and is also used to convert analog audio input into digital audio signals. The audio module 170 can also be used to encode and decode audio signals. The speaker 170A, also known as a "loudspeaker", is used to convert audio electrical signals into sound signals. The receiver 170B, also known as a "earpiece", is used to convert audio electrical signals into sound signals. The microphone 170C, also known as a "microphone", "sound transducer", is used to convert sound signals into electrical signals. The headset interface 170D is used to connect a wired headset.

[0071] The sensor module 180 can include a pressure sensor 180A, a gyro sensor 180B, an air pressure sensor 180C, a magnetic sensor 180D, an acceleration sensor 180E, a distance sensor 180F, a proximity light sensor 180G, a fingerprint sensor 180H, a temperature sensor 180J, a touch sensor 180K, an ambient light sensor 180L, a bone conduction sensor 180M, and the like.

[0072] The keys 190 include a power key, a volume key, and the like. The keys 190 can be mechanical keys. The keys 190 can also be touch keys. The electronic device 100 can receive key input and generate key signal input related to user settings and function control of the electronic device 100. The motor 191 can generate a vibration prompt. The motor 191 can be used for incoming call vibration prompts and also for touch vibration feedback. The indicator 192 can be an indicator light and can be used to indicate a charging state, a power change, and also to indicate a message, a missed call, a notification, and the like. The SIM card interface 195 is used to connect a SIM card. The SIM card can be inserted into or removed from the SIM card interface 195 to achieve contact and separation with the electronic device 100. The electronic device 100 can support one or N SIM card interfaces, where N is a positive integer greater than 1. The SIM card interface 195 can support a Nano SIM card, a Micro SIM card, a SIM card, and the like.

[0073] The electronic device 100 implements a display function through a GPU, a display screen 194, an application processor, and the like. The GPU is a microprocessor for image processing, connected to the display screen 194 and the application processor. The GPU is used to perform mathematical and geometric calculations for graphics rendering. The processor 110 can include one or more GPUs that execute program instructions to generate or change display information.

[0074] The display screen 194 is used to display images, videos, and the like. The display screen 194 includes a display panel. The display panel can use a liquid crystal display (LCD), a light-emitting diode (LED), an organic light-emitting diode (OLED), an active-matrix organic light emitting diode (AMOLED), a flexible light-emitting diode (FLED), a Miniled, a MicroLed, a Micro-oLed, a quantum dot light emitting diode (QLED), and the like.

[0075] The electronic device 100 can realize the shooting function through the ISP, camera 193, video codec, GPU, display screen 194 and application processor. The ISP is used to process the data fed back by the camera 193. The camera 193 is used to capture still images or videos. The digital signal processor is used to process digital signals. In addition to processing digital image signals, it can also process other digital signals. The video codec is used to compress or decompress digital videos. The electronic device 100 can support one or more video codecs. In this way, the electronic device 100 can play or record videos in multiple encoding formats, such as: Moving Picture Experts Group (MPEG) 1, MPEG2, MPEG3, MPEG4, etc.

[0076] The number of cameras 193 may be 1 to N. For example, the electronic device may include 2 front cameras and 4 rear cameras, wherein the front camera may include a TOF camera.

[0077] It should be understood that the structure illustrated in this embodiment does not constitute a specific limitation on the electronic device 100. In other embodiments, the electronic device 100 may include more or fewer components than shown, or may combine or separate certain components, or arrange the components differently. The illustrated components may be implemented in hardware, software, or a combination of software and hardware.

[0078] The methods in the following embodiments can all be implemented in the electronic device 100 having the above hardware structure.

[0079] Figure 2 This is a schematic diagram of the principle of a TOF imaging technology provided in an embodiment of the present application;

[0080] A TOF camera includes a transmitter (Tx) and receiver (Rx). The Tx transmitter can be used to transmit light signals (infrared light or laser pulses), while the Rx receiver can be used to receive images. For example, the Tx transmitter can be a laser diode (LD). The Rx receiver can be a complementary metal oxide semiconductor (CMOS) or charge coupled device (CCD) image sensor.

[0081] For example, Figure 2 As shown in (a) of FIG, the transmitter Tx of the TOF camera can continuously send light signals (infrared light or laser pulses) to the target to be measured (for example, the user), and the sensor end Rx of the TOF camera receives the light signal returned by the target to be measured, as shown in FIG. Figure 2As shown in (b) in FIG. 1, the depth information of the measured target can be obtained according to the phase difference (delay) of the emitted and received optical signals.

[0082] The Tx and the Rx can interact information through the bus. For example, the Rx can send a configuration parameter to the Tx through the bus (for example, a serial peripheral interface (SPI) bus), and the configuration parameter is used to indicate the address of the register corresponding to the Tx and the value for the register. For example, the address of the register corresponding to the Tx can be 0x11, and the storage space corresponding to 0x11 can store the current value. The Tx can work at the corresponding current value based on the corresponding configuration parameter, so as to emit an optical signal with a corresponding light intensity. The Rx can receive the reflected light of the optical signal emitted by the transmitter, and obtain corresponding image data based on the reflected light. It should be noted that the Tx can emit optical signals with different light intensities when working at different current values. For example, the Tx can emit an optical signal with a light intensity Q1 when working at a current value I1. The Tx can emit an optical signal with a light intensity Q2 when working at a current value I2. The current value I2 is greater than the current value I1. The light intensity Q2 is greater than the light intensity Q1. The image data obtained by the Rx based on the reflected light of the optical signals with different intensities is also different. For example, when the Tx emits an optical signal with a light intensity Q1 when working at a current value I1, the Rx obtains first image data within a corresponding exposure time; when the Tx emits an optical signal with a light intensity Q2 when working at a current value I2, the Rx obtains second image data within a corresponding exposure time; the second image data is different from the first image data.

[0083] The software system of the electronic device 100 can adopt a layered architecture, an event-driven architecture, a microkernel architecture, a microservice architecture, or a cloud architecture. The embodiment of the present application takes the Android system with a layered architecture as an example to exemplarily illustrate the software structure of the electronic device 100.

[0084] The layered architecture divides the software into several layers, and each layer has a clear role and division of labor. The layers communicate with each other through interfaces. In some embodiments, the Android system can include an application layer, an application framework layer, an Android runtime and a system library, a hardware abstraction layer (HAL), and a kernel layer. It should be noted that the embodiments of the present application are exemplified by the Android system, and as long as the functions of the various functional modules are similar to those of the embodiments of the present application, the schemes of the present application can also be implemented in other operating systems (such as the IOS system, etc.).

[0085] The application layer can include a series of application packages.

[0086] AsFigure 3 As shown, the application package can include applications such as a camera, a gallery, a calendar, a call, a map, a navigation, a WLAN, a Bluetooth, music, a video, a short message, a lock screen application, a settings application, and the like. Of course, the application layer can also include other application packages, such as a payment application, a shopping application, a bank application, a chat application, or a financial application, and the like, without limitation.

[0087] The settings application has a function of inputting a face, and the input face is used for face unlocking. The lock screen application has a function of responding to a user's unlocking operation (e.g., pressing a power key) to perform unlocking. The lock screen application can perform face unlocking, fingerprint unlocking, password unlocking, and the like, and embodiments of the present application are mainly described by taking face unlocking as an example.

[0088] The application framework layer provides an application programming interface (API) and a programming framework for the applications of the application layer. The application framework layer includes some pre-defined functions. For example, the application framework layer can include an activity manager, a window manager, a content provider, a view system, a resource manager, a notification manager, a camera service, a face recognition service, and the like, without limitation.

[0089] The system library can include a plurality of function modules. For example, a surface manager, media libraries, OpenGL ES, SGL, and the like.

[0090] The surface manager is used to manage a display subsystem, and provides fusion of 2D and 3D layers for a plurality of applications.

[0091] The media library supports playback and recording of a plurality of commonly used audio, video formats, and static image files. The media library can support a plurality of audio and video encoding formats, such as MPEG4, H.264, MP3, AAC, AMR, JPG, PNG, and the like.

[0092] The OpenGL ES is used to realize three-dimensional graphics drawing, image rendering, synthesis, and layer processing, and the like.

[0093] The SGL is a drawing engine for 2D drawing.

[0094] The Android runtime includes a core library and a virtual machine. The Android runtime is responsible for scheduling and management of the Android system. The core library includes two parts: one is a function function that the java language needs to call, and the other is the core library of Android. The application layer and the application framework layer run in the virtual machine. The virtual machine executes the java file of the application layer and the application framework layer into a binary file. The virtual machine is used to perform the functions of object lifecycle management, stack management, thread management, security and exception management, and garbage collection.

[0095] The HAL layer is a wrapper for the Linux kernel driver, providing an interface upward and shielding the implementation details of the low-level hardware.

[0096] The HAL layer can include a Wi-Fi HAL, an audio HAL, a camera HAL (Camera HAL), a face recognition control module, and the like.

[0097] The camera HAL is a core software framework of the Camera, and the camera HAL can include a sensor node and an image front end (IFE) node. The sensor node and the IFE node are components (nodes) in a transmission path (also referred to as a transmission pipeline) of image data and control instructions created by the camera HAL.

[0098] The face recognition control module, also referred to as a face client application (Face CA) or a face recognition CA, is a core software framework / application of face recognition. The face recognition control module can communicate with a face trusted application (Face TA).

[0099] The face trusted application, also referred to as a face recognition TA, is an application for face recognition running in a TEE environment.

[0100] The Face TA can include a frame management module, a TOF algorithm module, and a face (Face) ID algorithm module. The frame management module can send RAW Data corresponding to each frame of image to the TOF algorithm module. The TOF algorithm module can convert the RAW Data into a grayscale image (IR image) and a depth image (depth image). The TOF algorithm module can send the grayscale image and the depth image to the face ID algorithm module. The face ID algorithm module can perform face recognition (i.e., determine whether the current user is the owner) based on the grayscale image and perform liveness detection (anti-fake detection) based on the depth image, thereby obtaining a face recognition result.

[0101] The kernel layer is a layer between hardware and software. The kernel layer at least contains display driver, camera driver, audio driver, sensor driver.

[0102] The camera driver is a driving layer of the camera device, and is mainly responsible for interaction with the hardware.

[0103] The hardware layer includes a display, a TOF camera, an IFE module, and a secure buffer.

[0104] The secure buffer is a memory with a security protection function, and can be used to store raw data collected by the TOF camera.

[0105] The TOF camera, which can also be referred to as a TOF sensor, can include a transmitter Tx and a receiver Rx. The Tx is used to emit infrared light or laser pulses, and the Rx is used to receive reflected light and image.

[0106] IFE module (IFE-Lite): can be referred to as an image pre-processing module, and can be used to forward image data without processing the image data in the forwarding process.

[0107] The software modules and the interaction between the modules involved in the face recognition method provided by the embodiments of the present application are described below.

[0108] As shown in FIG. 1, the software modules involved in the face recognition method provided by the embodiments of the present application include a camera driver 101, an IFE module 102, a face recognition module 103, and a display driver 104. Figure 4As shown, the lock screen application in the application layer can interact with the face recognition SDK, the face recognition SDK can interact with the face recognition service in the framework layer by calling a preset application programming interface (API) interface, the face recognition service can interact with the face recognition CA in the HAL layer, the face recognition CA can interact with the camera HAL in the HAL layer through the camera service in the framework layer, or the face recognition CA can directly interact with the camera HAL in the HAL layer. The camera HAL can include a sensor node and an IFE node. The sensor node can interact with the camera driver module in the kernel layer, and the camera driver module can be used to drive the TOF camera in the hardware layer to collect image data in a default working mode (for example, a human eye safety mode). The IFE module can store the image data collected by the TOF camera into a secure memory. The storage location of the image data collected by the TOF camera in the secure memory can be represented using a file descriptor (FD). The IFE module can send the FD of the image data to the camera driver module, the camera driver module can send the FD to the IFE node of the camera HAL, the IFE node can send the FD to the camera service, the camera service can send the FD to the face recognition CA, and the face recognition CA can send the FD to the face recognition TA. The face recognition TA can read the image data from the secure memory according to the FD, process the image data, and can send the processing result to the face recognition CA. The face recognition CA can continue to interact with the camera HAL through the camera service to switch the working mode of the TOF camera. The camera HAL can continue to interact with the camera driver module, so that the camera driver module can drive the TOF camera to collect image data in the switched working mode (for example, a human face ID mode). The FD corresponding to the image data can continue to be sent to the face recognition TA through the IFE module, the camera driver module, the IFE node, the camera service and the face control module. The face recognition TA can read the image data again and process it, and feed back the processing result (face recognition success or face recognition failure) to the face recognition CA. The face recognition CA can feed back the processing result to the lock screen application through the face recognition service and the face recognition SDK, so that the lock screen application determines whether to unlock (if the face recognition is successful, unlock; if the face recognition fails, do not unlock, that is, unlock failure). Wherein, Figure 4 The solid arrow in the figure can be used to represent the control flow, and the dashed arrow can be used to represent the data flow.

[0109] In the embodiments of the present application, the working modes of the TOF camera can include a human eye safety mode, a human face ID mode (a first working mode), a skin safety mode (a second working mode) and a Tx off mode.

[0110] EyeSafe Mode refers to a mode in which the Tx of the TOF camera works at a small current (a current smaller than a preset threshold, for example, a current value I1). The EyeSafe Mode can be used for eye safety detection of the TOF camera, that is, detection of whether the TOF camera is damaged. When the TOF camera is damaged, the light intensity of the light signal emitted from the TOF camera can exceed the eye safety range, thereby causing harm to the human eye.

[0111] Face ID Mode refers to a mode in which the Tx of the TOF camera works at a normal current (that is, a first working current, the first working current being within a preset threshold range, for example, a current value I2). The Face ID Mode is used for scenarios such as secure face unlocking and secure payment. In the Face ID Mode, the electronic device can also perform skin safety detection to determine whether the intensity of the light signal emitted by the Tx is within the skin safety range.

[0112] FaceSafe Mode refers to a mode in which the Tx of the TOF camera works at a normal current (that is, a first working current, the first working current being within a preset threshold range, for example, a current value I2). However, compared with the Face ID Mode, the maximum exposure duration of the emitter of the TOF camera and the maximum exposure duration of the light receiver are smaller, so as to reduce the energy value of the light signal irradiated to the skin, thereby protecting the skin safety. In the FaceSafe Mode, the electronic device can also be used for scenarios such as secure face unlocking and secure payment.

[0113] Tx OFF Mode refers to a mode in which the Tx of the TOF camera is not powered on (and thus does not emit light). The Tx OFF Mode is used in a case where the Tx device of the TOF camera is damaged or cannot work normally. This is because the use of the damaged Tx device while still being powered on can have adverse effects on the human eye. Therefore, when it is detected that the Tx device of the TOF camera is damaged, the Tx OFF Mode is adopted to make the Tx device not powered on, thereby avoiding harm to the human eye.

[0114] Figure 5 is a structural schematic diagram of the Tx of the TOF camera provided in an embodiment of the present application.

[0115] As Figure 5In a, as shown in one implementation, the Tx of the TOF camera can include a light emitter, a light receiver and a light diffuser. The light emitter can be a laser diode (LD) for example, and the light receiver can be a photodiode (PD) for example. The LD and the PD can be arranged side by side under a display screen cover plate (such as a glass cover plate) of the electronic device. The LD projects a light signal, and the PD senses the reflected light of the light signal to generate a corresponding electrical signal, such as a voltage signal or a current signal, the intensity value of which is related to the light intensity of the light signal. The light diffuser is arranged between the display screen cover plate and the LD and the PD, and is located in the orthographic projection area of the LD and the PD, i.e. covers the LD and the PD. When the light signal emitted by the LD passes through the light diffuser, the light diffuser can perform light homogenization and scattering on the light signal, so that the light signal can uniformly and more widely irradiate the target such as a human face through the display screen cover plate and reflect back.

[0116] When the light signal projected by the LD passes through the light diffuser and the display screen cover plate, part of the light signal will be reflected back by the light diffuser and the display screen cover plate. When the PD senses the reflected light, an electrical signal related to the light intensity of the reflected light will be generated, such as a voltage signal and a current signal. In the embodiments of the present application, the voltage value of the voltage signal or the current value of the current signal generated by the PD is referred to as a PD value. It can be understood that the PD value is positively correlated with the light intensity of the reflected light sensed by the PD. Generally speaking, as shown in a, if the light diffuser is not damaged, the light intensity of the reflected light is relatively stable in the Eye Safe Mode, and therefore the PD value is equal to a standard PD value or within a standard PD value range. When the light diffuser is damaged, the light intensity of the reflected light will change, resulting in a change in the PD value. Figure 5

[0117] For example, as shown in b, when the light diffuser is partially damaged and partially undamaged in the orthographic projection area of the LD, and the light diffuser is completely damaged in the orthographic projection area of the PD, the PD value will increase. As shown in c, when the light diffuser is partially damaged and partially undamaged in the orthographic projection area of the LD, and the light diffuser is undamaged in the orthographic projection area of the PD, the PD value will increase. As shown in d, when the light diffuser is completely damaged in the orthographic projection area of the LD, and the light diffuser is undamaged in the orthographic projection area of the PD, the change in the PD value is uncertain, that is, the PD value can increase, decrease or remain unchanged. As shown in e, when the light diffuser is completely damaged, the PD value will increase. Figure 5 Figure 5 Figure 5 Figure 5

[0118] ​​​​​Based on the characteristic that the PD value will change when the light diffuser is damaged, the eye safety detection can be performed based on the PD value in the eye safety mode, that is, whether the TOF camera is damaged is detected. For example, if the PD value is greater than the standard PD value, the light diffuser is damaged, that is, the TOF camera is damaged. However, when the damage of the TOF camera is as shown in d of Figure 5 , the change of the PD value is uncertain, and when the PD value decreases or remains unchanged, it will lead to the failure to detect the damage of the camera, that is, there is the problem of eye safety false detection.

[0119] Figure 6 is another structure diagram of Tx of the TOF camera provided by the embodiment of the present application.

[0120] As shown in a of Figure 6 , in order to solve the problem of eye safety false detection, the TOF camera further includes a baffle, which is arranged in the orthographic projection area of the PD and located between the PD and the light diffuser. Based on the structure shown in a of Figure 6 , if the light diffuser is not damaged, the light intensity of the reflected light is also relatively stable in the eye safety mode (EyeSafe Mode), and therefore, the PD value is equal to a standard PD value or within a standard PD value range. When the light diffuser is damaged, the light intensity of the reflected light will change, leading to the change of the PD value.

[0121] For example, through actual test: as shown in b of Figure 6 , when the light diffuser is partially damaged and partially not damaged in the orthographic projection area of the LD, and the light diffuser is completely damaged in the orthographic projection area of the PD, the PD value will decrease. As shown in c of Figure 6 , when the light diffuser is partially damaged and partially not damaged in the orthographic projection area of the LD, and the light diffuser is not damaged in the orthographic projection area of the PD, the PD value will increase. As shown in d of Figure 5 , when the light diffuser is completely damaged in the orthographic projection area of the LD, and the light diffuser is not damaged in the orthographic projection area of the PD, the PD value will decrease. As shown in e of Figure 5 , when the light diffuser is completely damaged, the PD value will decrease. That is, in the Tx structure shown in Figure 6 , no matter how the light diffuser is damaged, the change of the PD value is certain. Therefore, when the eye safety detection is performed based on the PD value in the eye safety mode, as long as the PD value is greater than or less than the standard PD value, it can be determined that the light diffuser is damaged, that is, the TOF camera is damaged, and the situation of eye safety false detection will not occur.

[0122] In addition, based on Figure 5The structure shown can also perform skin safety detection based on the PD value. For example, in the face ID mode, when the skin of a face or other part blocks the TOF camera, the light signal emitted by the LD is irradiated onto the skin and reflected, and the light receiver receives the reflected light to generate a corresponding electrical signal. In this way, whether there is a skin safety problem can be determined based on the size and changes of the PD value. However, the method of performing skin safety detection based on the PD value cannot be used in the Tx structure shown, because after the baffle is arranged between the PD and the light diffuser, the reflected light from the outside is blocked by the baffle and cannot irradiate onto the PD, so that whether there is a skin safety problem cannot be determined based on the size and changes of the PD value. Figure 6 The Tx structure shown cannot be used, because after the baffle is arranged between the PD and the light diffuser, the reflected light from the outside is blocked by the baffle and cannot irradiate onto the PD, so that whether there is a skin safety problem cannot be determined based on the size and changes of the PD value.

[0123] For example, as shown in Figure 7 The sensor node in the camera HAL can be used to select the working mode of the TOF camera.

[0124] The default initial working mode of the TOF camera can be the eye safety mode. In the eye safety mode, the electronic device can perform eye safety detection, and after obtaining the eye safety detection result, the sensor node in the camera HAL can switch the working mode of the TOF camera. Specifically, if the eye safety detection result is successful (i.e., there is no eye safety problem), the TOF camera can be switched to the face ID mode. If the eye safety detection result fails (i.e., there is an eye safety problem), the TOF camera can be switched to the Tx off mode. In this way, the eye safety can be ensured. In the face ID mode, the electronic device can perform skin safety detection, and after obtaining the skin safety detection result, the sensor node in the camera HAL can switch the working mode of the TOF camera. Specifically, if the skin safety detection result is successful (i.e., there is no skin safety problem), the TOF camera can be kept in the face ID mode. If the skin safety detection result fails (i.e., there is a skin safety problem), the TOF camera can be switched to the skin safety mode. In this way, the skin safety can be ensured.

[0125] For ease of understanding, the method provided by the embodiments of the present application is specifically introduced below with reference to the drawings.

[0126] Figure 8 is a flowchart of a face recognition method provided by an embodiment of the present application.

[0127] As shown in Figure 8 The face recognition method provided by the embodiments of the present application can include the following steps:

[0128] In step S101, the lock screen application calls the face recognition SDK to perform face recognition.

[0129] When the electronic device detects a user's unlock operation (first operation), the lock screen application can call the face recognition SDK to perform face recognition. The user's unlock operation includes the user picking up the electronic device, or pressing the power key, or operating on the screen (clicking, sliding, etc.), or pulling out the charging line, etc.

[0130] In addition, the lock screen application can register a callback to the face recognition SDK. The purpose of registering the callback is that when the face recognition SDK obtains the face recognition result, the face recognition SDK can return the face recognition result to the lock screen application.

[0131] Step S102, the face recognition SDK sends a face recognition request to the face recognition service.

[0132] The face recognition request can carry an identification of a face recognition type, a resolution size of an image, and a data stream format. The face recognition type includes a 2D face recognition type (for example, which can correspond to identification 0) and a 3D face recognition type (for example, which can correspond to identification 1).

[0133] For example, the identification of the face recognition type carried in the face recognition request can be 1 (i.e. 3D face recognition type), the resolution size of the image can be 1280x2898 pixels, and the data stream format can be raw image format (RAW) 16.

[0134] In addition, the face recognition SDK can register a callback to the face recognition service. The purpose of registering the callback is that when the face recognition service obtains the face recognition result, the face recognition service can return the face recognition result to the face recognition SDK.

[0135] Step S103, the face recognition service sends a face recognition request to the face recognition CA.

[0136] The face recognition request can refer to the description of step S102, which will not be repeated here.

[0137] That is, the face recognition SDK can notify the face recognition CA to perform face recognition through the face recognition service. The face recognition service can send the face recognition request received from the face recognition SDK to the face recognition CA.

[0138] In addition, the face recognition service can register a callback to the face recognition CA. The purpose of registering the callback is that when the face recognition CA obtains the face recognition result, the face recognition CA can return the face recognition result to the face recognition service.

[0139] Step S104, in response to receiving the face recognition request, the face recognition CA matches the camera according to the face recognition request.

[0140] Specifically, the face recognition CA can obtain the identification of the face recognition type, the resolution size of the image and the data stream format from the face recognition request, and determine the matched camera by querying the camera capability from the camera service.

[0141] It should be understood that during the booting process of the electronic device, the camera service can send a camera capability query request to the camera HAL, the camera capability query request being used to request to query the camera capability supported by the electronic device. After receiving the camera capability query request, the camera HAL can send the camera capability supported by the electronic device to the camera service, and the camera service can store the received camera capability supported by the electronic device. The camera capability supported by the electronic device includes the camera identity (ID) of each camera, the supported maximum resolution size, the format of the data stream and whether the camera supports collecting depth information, etc.

[0142] For example, it is assumed that three cameras are installed on the mobile phone, and the capability information of the three cameras can be as shown in Table 1:

[0143] Table 1

[0144] Camera ID Mounting position Maximum resolution supported Data stream format Depth information 1 Rear-facing 4096 x 3072 pixels YUV no 2 Front-facing 3264 x 2448 pixels YUV no 3 Front-facing 1280 x 2898 pixels RAW16 yes

[0145] The camera with the camera ID of 3 can be a TOF camera and support collecting depth information. The cameras with the camera IDs of 1 and 2 can be ordinary cameras and do not support collecting depth information. Of course, more front or rear cameras can be installed on the mobile phone, for example, the mobile phone can be installed with 2 front cameras and 4 rear cameras.

[0146] The face recognition CA can send a camera capability query request to the camera service, the camera service can send the camera capability supported by the electronic device to the face recognition CA, and the face recognition CA can determine the camera matched with the face recognition request according to the camera capability supported by the electronic device, for example, it can be determined that the matched camera is the camera with the ID of 3 (i.e. the TOF camera).

[0147] It should be noted that Table 1 is only an example, and the data stream format corresponding to each camera can include multiple types. For example, the camera with the camera ID of 1 can correspond to not only the data stream format of YUV, but also the data stream format of RAW16, which is not limited in the present application.

[0148] S105, the face recognition CA sends a request for starting the camera Camera to the camera service.

[0149] For example, the face recognition CA can send a request for starting the camera to the camera service through a vendor native development kit (VNDK) interface. The request for starting the camera carries information such as a security identifier, a camera ID, a resolution size, and a data stream format. The security identifier is used to indicate that the data is stored in a secure buffer. That is, the security identifier can be used to apply for a piece of secure memory, which is used to store the data collected by the camera later. For example, the security identifier can be 1 or 0, 1 indicating that the data is stored in the secure buffer, and 0 indicating that the data is stored in a non-secure buffer.

[0150] For example, the security identifier carried in the request for starting the camera can be 1 (that is, the data is stored in the secure buffer), the resolution size of the image can be 1280x2898 pixels, the data stream format can be RAW16, and the camera ID can be 3.

[0151] In addition, the face recognition CA can register a callback with the camera service, and the callback is used to notify the face recognition CA that the camera service has completed the camera starting after the camera service completes the camera starting.

[0152] In response to receiving the request for starting the camera, the camera service sends a request for starting the camera to the camera HAL, and the request for starting the camera carries information such as the security identifier, the camera ID, the resolution size, and the data stream format.

[0153] The camera HAL receiving the request for starting the camera can cache the information such as the security identifier, the camera ID, the resolution of the image, and the data stream format for a preset time length.

[0154] In addition, the camera service can register a callback with the camera HAL, and the callback is used to notify the camera service of the result of creating the path by the camera HAL.

[0155] The camera HAL creates a corresponding path according to the camera ID, the resolution of the image, and the data stream format.

[0156] The camera HAL can select available nodes according to the camera ID, the resolution, and the data stream format, and then create a corresponding path according to the available nodes. For example, if the resolution is 1280x2898 pixels, the data stream format is RAW16, and the camera ID is 3, it can be determined that the sensor node and the IFE node are selected. This is because the sensor node and the IFE node can support the transmission of the data collected by the camera with the camera ID of 3, the resolution of 1280x2898 pixels, and the data stream format of RAW16.

[0157] The path corresponding to the sensor node can be a path composed of the sensor node, the camera driver, the TOF camera, the IFE module, and the secure memory. The path corresponding to the IFE node can be a path composed of the IFE module (carrying the FD), the camera driver, and the IFE node. The camera HAL can connect the output port of the sensor node and the input port of the IFE node at the HAL layer. Thus, the path corresponding to the sensor node and the path corresponding to the IFE node can form a closed loop path. After the path is created, the hardware in the path is powered on (i.e., the hardware circuit is powered on) and waits for a data request.

[0158] In step S108, the camera HAL sends the result of creating the path to the camera service.

[0159] The result of creating the path can be success or failure. If the result of creating the path is failure, the camera HAL notifies the camera service that the path creation fails. If the result of creating the path is success, the camera HAL notifies the camera service that the path creation succeeds, and the subsequent steps S109 and the like can be performed.

[0160] In step S109, in response to receiving the notification that the path creation succeeds, the camera service sends a message that the camera startup is completed to the face recognition CA.

[0161] It can be understood that the camera startup completion means that the preparation work (e.g., camera parameter configuration, power-on, and the like) before photographing or video shooting of the camera has been completed.

[0162] In step S110, in response to receiving the message that the camera startup is completed, the face recognition CA sends a data request to the camera service.

[0163] The data request is used to request to obtain the data stream of the camera.

[0164] In step S111, in response to receiving the data request sent by the face recognition CA, the camera service calls the camera HAL to obtain the data stream.

[0165] In step S112, the sensor node of the camera HAL selects the working mode of the camera.

[0166] Specifically, the sensor node can select the working mode of the camera through the camera resolution and the data stream format cached in S106. For example, the sensor node can select the working mode of the camera by looking up a table (e.g., Table 2).

[0167] Table 2

[0168]

[0169]

[0170] Of course, the working mode of the camera can further include more, which is not limited in the present application.

[0171] According to Table 2, when the maximum value of the resolution of the image is 1280x2898 pixels and the data stream format is Raw, the working mode of the camera can include the eye safety mode, the face ID mode, the skin safety mode, the Tx off mode, etc.

[0172] In the embodiment of the present application, the sensor node can take the eye safety mode as the initial working mode of the camera. Therefore, in step S112, the sensor node can select the eye safety mode.

[0173] In step S113, the sensor node sends the configuration parameters of the eye safety mode to the camera driving module.

[0174] The sensor node can store the configuration parameters corresponding to different working modes of the TOF sensor. For example, the configuration parameters corresponding to the eye safety mode can be: the current value is 700mA, the depth exposure duration is 10us, and the IR grayscale exposure duration is 10us. The working parameters corresponding to the face ID mode can be: the current value is 2800mA, the depth exposure duration is 1.8ms, and the IR grayscale exposure duration is 3ms. The configuration parameters of the skin safety mode can be: the current value is 2800mA, the depth exposure duration is 5us, and the IR grayscale exposure duration is 450us. The working parameters corresponding to the Tx off mode can be: the current value is 0mA, the depth exposure duration is 0us, and the IR grayscale exposure duration is 0us. Among them, the depth exposure duration and the IR grayscale exposure duration in each configuration parameter can refer to the maximum exposure duration.

[0175] Among them, the depth exposure duration refers to the time when the TOF camera captures depth information, that is, the time when the laser or infrared light emitter is turned on, that is, the time when the Tx transmits the light signal. This time determines the time window for the sensor to measure the depth. Generally, the longer the depth exposure duration, the more laser or infrared light signals can be received by the Rx, thereby improving the accuracy of depth measurement. The IR grayscale exposure duration refers to the time when the Rx of the TOF camera is in the receiving state.

[0176] In step S114, the camera driving module writes the configuration parameters of the eye safety mode into the register of the TOF camera.

[0177] That is, the camera driving module can send the configuration parameters of the eye safety mode to the TOF camera.

[0178] Exemplarily, the camera driving module can write the configuration parameters of the eye-safety mode into a register of the Rx of the TOF camera through an inter-integrated circuit (I2C) bus. The register can be disposed in the TOF sensor or outside the TOF sensor. The address corresponding to the register of the Rx can be 0x01. The register corresponding to the Rx can include multiple registers, which are not limited in the present application. That is, the camera driving module can send the configuration parameters of the eye-safety mode to the Rx of the TOF camera through the I2C bus. The configuration parameters of the eye-safety mode include configuration parameters for the Rx and the Tx. For example, the configuration parameters for the Tx can be a current value I1 and a depth exposure time. The configuration parameters for the Rx can be an IR grayscale exposure time. The Rx can write the configuration parameters corresponding to the Tx into the register corresponding to the Tx through an SPI bus. The address of the register corresponding to the Tx can be 0x11. The register corresponding to the Tx can include multiple registers, which are not limited in the present application.

[0179] In step S115, the camera driving module sends a stream on command / instruction to the TOF camera.

[0180] The stream on command is used to drive the TOF camera to perform data acquisition.

[0181] In step S116, in response to receiving the stream on command, the TOF camera runs based on the configuration parameters of the eye-safety mode.

[0182] In a specific implementation, the Tx in the TOF sensor can emit a light signal by the light emitter of the Tx based on the depth exposure time and the current value I1 in the configuration parameters of the eye-safety mode. Of course, the present embodiment does not limit the specific value of the current value I1, and any current value that can ensure no harm to the human eye and support the TOF sensor to complete the eye-safety detection can be used. That is, the light signal emitted by the TOF sensor based on the current value I1 is harmless to the human eye. At the same time, the depth exposure time should not be too long, and can support the TOF sensor to complete the eye-safety detection.

[0183] When the light emitter of the Tx emits the light signal, the light receiver of the Tx can sense the reflected light of the light signal, generate an electrical signal related to the light intensity of the reflected light, and the TOF sensor can obtain the eye-safety detection result based on the numerical relationship between the PD value of the electrical signal generated by the light receiver of the Tx and the standard PD value.

[0184] For example, the human eye safety detection result can include safe (normal) or unsafe (abnormal), wherein if the PD value in the human eye safety detection result is equal to or within the standard PD value range, the human eye safety detection result is safe, and if the PD value is greater than or less than the standard PD value or outside the standard PD value range, the human eye safety detection result is unsafe.

[0185] Next, the TOF sensor can also write the human eye safety detection result into a register, wherein the register can be arranged in the TOF sensor. In addition, although the TOF sensor does not output image data externally during the operation based on the configuration parameters of the human eye safety mode, this part of the image data is also data related to user privacy, and therefore, the embodiment can store these image data and the human eye safety detection result in the register together to ensure data security.

[0186] Step S117, the camera driving module reads the human eye safety detection result from the register of the TOF camera.

[0187] For example, the camera driving module can read the human eye safety detection result from the register of the TOF camera through an inter-integrated circuit (I2C) bus.

[0188] Step S118, the camera driving module sends the human eye safety detection result to the camera HAL.

[0189] Step S119, the sensor node of the camera HAL determines the working mode of the TOF camera based on the human eye safety detection result.

[0190] For example, if the human eye safety detection result is safe, the sensor node determines that the working mode of the TOF camera is the face ID mode; if the human eye safety detection result is unsafe, the sensor node determines that the working mode of the TOF camera is the Tx-off mode.

[0191] It should be noted that the execution order of steps S101-S119 is not limited in the embodiment of the present application. In some embodiments, after steps S101-S107 are executed, steps S112-S118 can be directly executed, and then steps S108-S111 can be executed after step S118, and then step S119 can be executed after step S111. Of course, steps S101-S119 can also have other combination orders to ensure that the sensor node of the camera HAL can obtain the human eye safety detection result, so as to determine the working mode of the TOF camera, which will not be described here.

[0192] The following is an example of a sensor node determining the working mode of a TOF camera as a face ID mode. After S119, the method further includes steps S120-S157:

[0193] In step S120, the sensor node sends the configuration parameters of the face ID mode to the camera driving module.

[0194] In step S121, the camera driving module writes the configuration parameters of the face ID mode into the register of the TOF camera.

[0195] That is, the camera driving module can send the configuration parameters of the face ID mode to the TOF camera.

[0196] For example, the camera driving module can write the configuration parameters of the face ID mode into the register of the TOF camera through I2C. That is, the camera driving module can send the configuration parameters of the face ID mode to the TOF camera through I2C.

[0197] In step S122, the TOF camera collects RAW Data 1 based on the face ID mode.

[0198] In a specific implementation, the Tx of the TOF camera can work at a current value I2 (i.e., a first working current) based on the configuration parameters of the face ID mode, and emit a light signal based on the depth exposure time length (i.e., a first depth exposure time length) in the configuration parameters of the face ID mode, and the Rx of the TOF camera can receive reflected light of the light signal and image based on the IR grayscale exposure time length (i.e., a first grayscale exposure time length) in the configuration parameters of the face ID mode, thereby obtaining image data.

[0199] For example, the configuration parameters of the face ID mode include a current value of 2800 mA, a depth exposure time length of 1.8 ms, and an IR grayscale exposure time length of 3 ms. Then, the Tx of the TOF camera can work at a current value of 2800 mA, the depth exposure time length (i.e., the time length of emitting a light signal) is 1.8 ms, and the IR grayscale exposure time length (i.e., the time length of receiving reflected light) of the Rx of the TOF camera is 3 ms.

[0200] It should be noted that in the face ID mode, the Tx and Rx in the TOF camera are periodically exposed, and in each acquisition cycle, the Tx in the TOF camera emits a light signal once in the depth exposure duration, and the Rx in the TOF camera receives reflected light once in the IR grayscale exposure duration and forms an image, so that a frame of image data can be obtained, that is, RAW Data 1 corresponding to this frame of image data is obtained. That is, the TOF camera can obtain RAW Data 1 corresponding to a frame of image data in each acquisition cycle, and can obtain multiple RAW Data 1 corresponding to multiple frames of image data in multiple acquisition cycles, for example, RAW Data 11~RAW Data 1 n , n is an integer greater than 1.

[0201] In addition, the RAW Data 1 can also include Metadata. For example, the Metadata can include the current working mode of the TOF camera (for example, the face ID mode), the configuration parameters of the current working mode (for example, the configuration parameters of the face ID mode), the working state of the TOF camera device, and the like.

[0202] Step S123, the TOF camera sends the RAW Data 1 to the IFE module.

[0203] For example, the TOF camera can transmit the RAW Data 1 to the IFE module through the MIPI interface.

[0204] Step S124, the IFE module stores the RAW Data 1 in the secure memory (Secure Buffer).

[0205] The storage location of the RAW Data 1 in the secure memory can be represented by FD 1.

[0206] Step S125, the IFE module sends the FD 1 to the camera drive module.

[0207] Step S126, the camera drive module sends the FD 1 to the IFE node.

[0208] Step S127, the IFE node sends the FD 1 to the camera service through the interface of the camera HAL.

[0209] Step S128, the camera service sends the FD 1 to the face recognition CA.

[0210] Step S129, the face recognition CA sends the FD 1 to the face recognition TA.

[0211] At step S130, the frame management module of the face recognition TA reads the RAW Data 1 from the secure memory according to the FD 1.

[0212] It should be noted that, since the TOF camera can obtain RAW Data 1 corresponding to each frame of image data in each acquisition cycle, after the TOF camera obtains RAW Data 1 corresponding to each frame of image data, steps S123-S130 can be performed once, so that the face recognition TA can read RAW Data 1 corresponding to each frame of image data from the secure memory. In addition, RAW Data 1 corresponding to different frames of image data can be stored in different positions of the secure memory, so as to avoid that RAW Data 1 stored later will overwrite RAW Data 1 stored earlier.

[0213] At step S131, the frame management module sends the RAW Data 1 to the TOF algorithm module.

[0214] At step S132, the TOF algorithm module obtains a grayscale image and a depth image corresponding to the RAW Data 1.

[0215] At step S133, the TOF algorithm module sends the grayscale image and the depth image corresponding to the RAW Data 1 to the face ID algorithm module.

[0216] At step S134, the face ID algorithm module obtains a face recognition result according to the grayscale image and the depth image corresponding to the RAW Data 1.

[0217] In a specific implementation, the face ID algorithm module can perform face recognition based on the grayscale image and perform anti-fake detection based on the depth image, so as to obtain the face recognition result. It should be noted that the face ID algorithm module can store grayscale images and depth images converted from face information previously input by a user. If the grayscale image corresponding to the face information currently acquired (RAW Data 1 acquired by the TOF camera based on the face ID mode) matches the grayscale image corresponding to the face information previously input by the user (RAW Data 1 acquired by the electronic device when the user performs a face input operation), it can be considered that the user is the same (i.e., the user performing the face input operation and the user performing the unlocking operation are the same).

[0218] If the current collected face information includes depth information, it can be considered that the current user is real and reliable (not disguised by photos, videos, etc.), at this time it can be considered that the face of the current user is safe, that is, the face recognition result is successful. If the current collected face information (TOF camera based on face ID mode to collect RAW Data 1) does not match the previously recorded face information (i.e. the gray scale image corresponding to the RAW Data 1 collected by the electronic device when the user performs the face recording operation), or if the current collected face information does not include depth information, it is considered that the face of the current user is not safe, that is, the face recognition result is failed.

[0219] If the face recognition fails, the face recognition TA can continue to obtain the RAW Data 1 collected by the TOF camera and continue to perform face recognition based on the RAW Data 1, and can continue to perform steps S138-S157 until the unlocking is successful or the unlocking is timed out and the unlocking is exited. If the face recognition is successful, steps S138-S157 do not need to be performed again.

[0220] Step S135, the face ID algorithm module sends the face recognition result to the frame management module.

[0221] Step S136, the frame management module sends the face recognition result to the face recognition CA.

[0222] Next, the electronic device can perform steps S180-S183, if the face recognition result is safe / normal (or the face recognition is successful), the electronic device can successfully unlock, if the face recognition result is unsafe / abnormal (or the face recognition is failed), the electronic device fails to unlock.

[0223] Step S137, the TOF algorithm module performs skin safety detection based on the brightness of the gray scale image corresponding to the RAW Data 1, and obtains a skin safety detection result.

[0224] In a specific implementation, the brightness of the gray scale image can be represented by a gray level (Gray Level). The gray level is a unitless value representing the brightness of a pixel in the gray scale image. That is, in the gray scale image, the brightness of each pixel can be represented by a numerical value (i.e. gray level).

[0225] For example, an 8-bit gray scale image has 256 gray levels, that is, the brightness of any pixel in the gray scale image can range from 0 to 255. The smaller the numerical value, the lower the brightness of the pixel, and the larger the numerical value, the higher the brightness of the pixel. 0 represents black, 255 represents white, and intermediate values represent different degrees of brightness.

[0226] For example, the brightness of the grayscale image is equal to the average value of the brightness of all pixels in the grayscale image, that is, the brightness of the grayscale image is equal to the average brightness of all pixels in the grayscale image.

[0227] For example, the skin safety detection result can include safe (normal) or unsafe (abnormal). In order to obtain the skin safety detection result, a threshold corresponding to a grayscale level, that is, a brightness threshold, can be preset, and the brightness of the grayscale image is compared with the brightness threshold. If the brightness (that is, the first brightness) of the grayscale image corresponding to the continuous n frames of RAW Data 1 (for example, the continuous RAW Data 11-RAW Data 1 n corresponding grayscale image) is greater than the brightness threshold, the skin safety detection result is unsafe (that is, the first detection result), and if the brightness of at least one of the grayscale images in the continuous n frames of RAW Data 1 is less than or equal to the brightness threshold (that is, the brightness is not all greater than the brightness threshold), the skin safety detection result is safe (that is, the second detection result).

[0228] For example, taking an 8-bit grayscale image as an example, the brightness threshold can be 240, and n can be equal to 15, that is, if the brightness of the continuous 15 frames of grayscale images is all greater than the brightness threshold, the skin safety detection result is unsafe, and if the brightness of at least one of the continuous 15 frames of grayscale images is less than or equal to 240, the skin safety detection result is safe.

[0229] It should be noted that when the brightness of the continuous n frames of grayscale images is all greater than the brightness threshold, it indicates that the TOF camera can be continuously directed towards the user, resulting in sufficient light signals being reflected back. In this case, based on the face ID mode configuration parameter, if the light signal continuously irradiates on the skin, the energy of the light signal will be continuously absorbed by the skin, thereby causing the total energy of the light signal absorbed by the skin in a period of time to be greater than the upper limit of the safe energy, and therefore, if the brightness of the continuous n frames of grayscale images is all greater than the brightness threshold, the skin safety detection result is unsafe.

[0230] When the brightness of the grayscale image is greater than the brightness threshold and lasts for a certain number of frames, it indicates that the TOF camera can be continuously directed towards the user, and therefore there is a skin safety risk. In this case, the electronic device further reduces the exposure time of the TOF camera, thereby facilitating improvement of the accuracy of entering the skin safety mode.

[0231] As can be seen, the face recognition method provided by the embodiments of the present application can perform skin safety detection based on the brightness of the grayscale image corresponding to the image frame, and no longer relies on the PD value of the light receiver in the Tx, and therefore, when the electronic device is equipped with a TOF camera with a baffle as shown in the drawings, the electronic device can also perform skin safety detection, thereby improving the adaptability of the face recognition method. Figure 6 As can be seen, the face recognition method provided by the embodiments of the present application can perform skin safety detection based on the brightness of the grayscale image corresponding to the image frame, and no longer relies on the PD value of the light receiver in the Tx, and therefore, when the electronic device is equipped with a TOF camera with a baffle as shown in the drawings, the electronic device can also perform skin safety detection, thereby improving the adaptability of the face recognition method.

[0232] At step S138, the TOF algorithm module sends the skin safety detection result to the frame management module.

[0233] At step S139, the frame management module sends the skin safety detection result to the face recognition CA.

[0234] At step S140, the face recognition CA sends the skin safety detection result to the camera HAL.

[0235] For example, the face recognition CA can directly deliver the skin safety detection result to the camera HAL through the HIDL interface. Alternatively, the face recognition CA can send the skin safety detection result to the camera service through the VNDK interface, so that the camera service sends the skin safety detection result to the camera HAL.

[0236] At step S141, the sensor node of the camera HAL determines the working mode of the TOF camera based on the skin safety detection result.

[0237] For example, if the skin safety detection result is safe, the sensor node determines the working mode of the TOF camera as the face ID mode, that is, the working mode of the TOF camera is not changed; if the skin safety detection result is unsafe, the sensor node determines the working mode of the TOF camera as the skin safety mode.

[0238] At step S142, the sensor node sends the configuration parameters of the skin safety mode to the camera driver module.

[0239] At step S143, the camera driver module writes the configuration parameters of the skin safety mode into the register of the TOF camera.

[0240] That is, the camera driver module can send the configuration parameters of the skin safety mode to the TOF camera.

[0241] For example, the camera driver module can write the configuration parameters of the skin safety mode into the register of the TOF camera through I2C. That is, the camera driver module can send the configuration parameters of the skin safety mode to the TOF camera through I2C.

[0242] At step S144, the TOF camera acquires RAW Data 2 based on the configuration parameters of the skin safety mode.

[0243] In a specific implementation, the Tx of the TOF camera can work at a current value I2 (i.e., a second working current) based on the configuration parameters of the skin safety mode, and emit a light signal based on a depth exposure time length (i.e., a second depth exposure time length) in the configuration parameters of the skin safety mode, and the Rx of the TOF camera can receive reflected light of the light signal and image based on an IR grayscale exposure time length (i.e., a second grayscale exposure time length) in the configuration parameters of the skin safety mode, so as to obtain image data.

[0244] For example, the configuration parameters of the skin safety mode include a current value 2800 mA, a depth exposure time length 5 μs, and an IR grayscale exposure time length 450 μs. Then, the Tx of the TOF camera can work at a current value 2800 mA, and the depth exposure time length (i.e., the time length of the emitted light signal) is 5 μs, and the IR grayscale exposure time length (i.e., the time length of the received reflected light) of the Rx of the TOF camera is 450 μs.

[0245] It should be noted that, compared with the configuration parameters of the face ID mode, the current value of the skin safety mode can be equal to the current value of the face ID mode, but the depth exposure time length and the IR grayscale exposure time length of the skin safety mode are less than the depth exposure time length and the IR grayscale exposure time length of the face ID mode. That is, under the skin safety mode, the intensity of the signal light emitted by the TOF camera can remain unchanged compared with the face ID mode, but the time length of the emitted light signal is shortened. In this way, the time length of the light signal irradiated on the skin is shortened, and the energy of the light signal absorbed by the skin is reduced, thereby avoiding the skin safety problem.

[0246] For example, different light signal wavelengths and different irradiation time lengths can correspond to different upper limit values of safety energy, and taking a light signal of 940 nanometers (nm) as an example, as shown in Table 3:

[0247] Table 3

[0248]

[0249] In the wavelength range of 700-1050 nm, C4=10 0.002(λ-700) W·m -2 Table formula: watt per square meter, J·m -2 Table formula: joule per square meter. For example, in the wavelength range of 700-1400 nm, if the light signal irradiation time length is 10 -7 -10 - 3 s, then the upper limit of the safety energy value of the light signal absorbed by the skin is 1.1×10 4 C4t 0.25 J·m -2 .

[0250] Exemplarily, taking the light signal of 940nm as an example, the depth exposure time length and the IR grayscale exposure time length in the configuration parameter of the skin safety mode can be determined based on Table 3, as long as the energy of the light signal absorbed by the skin is not higher than the corresponding safety upper limit value within the irradiation time length shown in Table 3.

[0251] That is, the current value 2800mA, the depth exposure time length 5μs, and the IR grayscale exposure time length 450μs in the configuration parameter of the skin safety mode are only examples, and different configuration parameters can also be determined based on Table 3 in the specific implementation, which is not limited by the embodiments of the application.

[0252] As can be seen, the depth exposure time length and the IR grayscale exposure time length in the configuration parameter of the skin safety mode can be determined based on the safety upper limit value in Table 3, so that the energy of the light signal absorbed by the skin is not higher than the corresponding safety upper limit value within the irradiation time length of Tx, so as to ensure that the light signal will not cause harm to the skin of a person, and the skin safety is protected.

[0253] In step S145, the TOF camera sends RAW Data 2 to the IFE module.

[0254] Exemplarily, the TOF camera can transmit RAW Data 2 to the IFE module through the MIPI interface.

[0255] In step S146, the IFE module stores RAW Data 2 in the secure memory.

[0256] The storage location of RAW Data 2 in the secure memory can be represented by FD 2.

[0257] In step S147, the IFE module sends FD 2 to the camera driving module.

[0258] In step S148, the camera driving module sends FD 2 to the IFE node.

[0259] In step S149, the IFE node sends FD 2 to the camera service through the interface of the camera HAL.

[0260] In step S150, the camera service sends FD 2 to the face recognition CA.

[0261] In step S151, the face recognition CA sends FD 2 to the face recognition TA.

[0262] In step S152, the frame management module of the face recognition TA reads RAW Data 2 from the secure memory according to FD 2.

[0263] In step S153 , the TOF algorithm module obtains the grayscale image and depth image corresponding to RAW Data 2 .

[0264] Step S154: The TOF algorithm module sends the grayscale image and depth image corresponding to RAW Data 2 to the face ID algorithm module.

[0265] In step S155 , the face ID algorithm module obtains a face recognition result based on the grayscale image and depth image corresponding to RAW Data 2 .

[0266] If face recognition fails, the face recognition TA can continue to obtain RAW Data 2 collected by the TOF camera and continue face recognition based on RAW Data 2, and can continue to execute steps S158-S173 until unlocking is successful or the unlocking timeout is exceeded. If face recognition is successful, steps S158-S173 do not need to be executed again.

[0267] Step S156: The face ID algorithm module sends the face recognition result to the frame management module.

[0268] Step S157: The frame management module sends the face recognition result to the face recognition CA.

[0269] Next, the electronic device can execute steps S180-S183. If the face recognition result is safe / normal (or the face recognition is successful), the electronic device can be successfully unlocked. If the face recognition result is unsafe / abnormal (or the face recognition fails), the electronic device fails to unlock.

[0270] It can be seen from this that in the face recognition method provided by the embodiment of the present application, the current value in the skin safety mode can be equal to the current value in the face ID mode, and the depth exposure time and IR grayscale exposure time in the skin safety mode are shorter than the depth exposure time and IR grayscale exposure time in the face ID mode. In other words, in the skin safety mode, the intensity of the signal light emitted by the TOF camera can remain unchanged compared to the face ID mode, but the duration of transmitting and receiving the light signal is shortened. In this way, the energy of the light signal absorbed by the skin is reduced, protecting the skin's safety, and face recognition can be performed normally, and the power consumption during face recognition can be reduced, achieving three goals at one stroke.

[0271] Figure 9 This is another flow chart of a face recognition method provided in an embodiment of the present application.

[0272] like Figure 9 As shown, in one implementation, after step S155, the method further includes:

[0273] Step S158, the TOF algorithm module obtains the comparison result of the brightness of the gray scale image corresponding to the RAW Data 2 and the brightness threshold.

[0274] The comparison result can include that the brightness (i.e., the second brightness) of the gray scale image corresponding to the RAW Data 2 is greater than or equal to the brightness threshold (i.e., the second comparison result), or the brightness of the gray scale image corresponding to the RAW Data 2 is less than the brightness threshold (i.e., the first comparison result). If the brightness of the gray scale image corresponding to the RAW Data 2 is greater than or equal to the brightness threshold, the TOF camera can still be continuously directed toward the user, resulting in still enough light signals being reflected back. If the brightness of the gray scale image corresponding to the RAW Data 2 is less than the brightness threshold, it indicates that the TOF camera can no longer be directed toward the user or is far away from the user, resulting in a decrease in the reflected light signals.

[0275] Step S159, the TOF algorithm module sends the comparison result to the frame management module.

[0276] Step S160, the frame management module sends the comparison result to the face recognition CA.

[0277] Step S161, the face recognition CA sends the comparison result to the camera HAL.

[0278] Step S162, the sensor node of the camera HAL determines the working mode of the TOF camera based on the comparison result.

[0279] For example, if the comparison result is that the brightness of the gray scale image corresponding to the RAW Data 2 is greater than or equal to the brightness threshold, the sensor node determines that the working mode of the TOF camera is the skin safety mode, that is, the working mode of the TOF camera is not changed; if the comparison result is that the brightness of the gray scale image corresponding to the RAW Data 2 is less than the brightness threshold, the sensor node determines that the working mode of the TOF camera is the face ID mode.

[0280] When the brightness of the gray scale image corresponding to the RAW Data 2 is less than the brightness threshold, it indicates that the skin safety risk is eliminated, and therefore, the electronic device can control the TOF camera to return to capturing image frames based on the face ID mode, so as to obtain image frames containing more information, which is conducive to improving the speed of face recognition.

[0281] Step S163, the sensor node sends the configuration parameters of the face ID mode to the camera driving module.

[0282] Step S164, the camera driving module writes the configuration parameters of the face ID mode into the register of the TOF camera.

[0283] Step S165, the TOF camera captures RAW Data 3 based on the configuration parameters of the face ID mode.

[0284] Step S166, the TOF camera sends RAW Data 3 to the IFE module.

[0285] For example, the TOF camera can transmit RAW Data 3 to the IFE module through the MIPI interface.

[0286] Step S167, the IFE module stores RAW Data in the secure memory.

[0287] The storage location of RAW Data in the secure memory can be represented by FD 3.

[0288] Step S168, the IFE module sends FD 3 to the camera driver module.

[0289] Step S169, the camera driver module sends FD 3 to the IFE node.

[0290] Step S170, the IFE node sends FD 3 to the camera service through the interface of the camera HAL.

[0291] Step S171, the camera service sends FD 3 to the face recognition CA.

[0292] Step S172, the face recognition CA sends FD 3 to the face recognition TA.

[0293] Step S173, the frame management module of the face recognition TA reads RAW Data 3 from the secure memory according to FD 3.

[0294] Step S174, the frame management module sends RAW Data 3 to the TOF algorithm module.

[0295] Step S175, the TOF algorithm module obtains the grayscale image and depth image corresponding to RAW Data 3.

[0296] Step S176, the TOF algorithm module sends the grayscale image and depth image corresponding to RAW Data 3 to the face ID algorithm module.

[0297] Step S177, the face ID algorithm module obtains the face recognition result according to the grayscale image and depth image corresponding to RAW Data 3.

[0298] Step S178, the face ID algorithm module sends the face recognition result to the frame management module.

[0299] In step S179, the frame management module sends the face recognition result to the face recognition CA.

[0300] Step S180: The face recognition CA sends the face recognition result to the face recognition service.

[0301] The face recognition CA can send the face recognition results to the face recognition service based on the callback previously registered with the face recognition service.

[0302] Step S181: The face recognition service sends the face recognition result to the face recognition SDK.

[0303] The face recognition service sends the face recognition results to the face recognition SDK based on the callback previously registered in the face recognition SDK.

[0304] Step S182: The face recognition SDK sends the face recognition result to the lock screen application.

[0305] The face recognition SDK sends the face recognition results to the lock screen application based on the callback previously registered in the lock screen application.

[0306] Step S183: The lock screen application decides whether to unlock based on the face recognition result.

[0307] If the facial recognition result is successful, the lock screen application can be successfully unlocked, and the electronic device can display the home screen or the interface of the application (system application or third-party application). If the facial recognition result is unsuccessful, the lock screen application will not be unlocked, that is, the face unlocking has failed. After the face unlocking fails, the lock screen application can disable the face recognition function for a period of time (for example, 5 minutes) when the face recognition failed.

[0308] For example, if the user sets up face unlock, such as Figure 10 As shown in (a) in FIG, when the user picks up the phone for face recognition, in response to the user picking up the phone, as shown in FIG. Figure 10 As shown in (b) in FIG. 7 , the mobile phone may display a lock screen interface 701 . During the face recognition process, the mobile phone may display an unlock icon 702 and a prompt text “Recognizing face” 703 on the lock screen interface 701 .

[0309] If face recognition is successful, Figure 11 As shown in (a) of FIG, the mobile phone may display an interface 704, which may include an unlock icon 705 (the unlock icon 705 may be an open lock, which may visually prompt the user that the face unlock is successful) and a prompt text "Swipe up to enter" 706. In response to the user's swiping operation, the mobile phone may display the home screen or the interface of an application (system application or third-party application). Or, as Figure 11As shown in (b), if the face recognition is successful, the mobile phone can be directly unlocked without additional user operation, that is, the home screen 707 can be displayed immediately, or the application interface can be directly displayed.

[0310] If face recognition fails, such as Figure 12 As shown in (a) in the figure, the mobile phone can display interface 708, which may include an unlock icon 709 (the unlock icon 709 may be a closed lock, which can vividly prompt the user that the face unlock is not successful) and a prompt text "Unrecognized, double-click the screen to try again" 710. In response to the user's double-click operation, the mobile phone can perform face recognition again (i.e., collect the user's face information again for comparison and anti-counterfeiting judgment). Or, in response to the user's upward sliding operation in interface 708, such as Figure 12 As shown in (b) of FIG, the mobile phone may display an interface 711. After entering the interface 711, the mobile phone may perform face recognition again. The interface 711 may include a face recognition icon 712 and a prompt text "Face recognition in progress" 713. If the recognition is still unsuccessful, as shown in FIG. Figure 12 As shown in (c), the mobile phone can display an interface 714, which can include a prompt text 715 "Unsuccessful recognition, click here to try again". The user can click the corresponding position to re-trigger face recognition, or can also enter a password through the soft keyboard 716 to unlock, avoiding the problem of poor user experience caused by unsuccessful recognition.

[0311] It should be noted that the above embodiment uses the lock screen application as an example to illustrate the face recognition method provided by the embodiment of the present application. The face recognition method provided by the embodiment of the present application can also be used for face recognition when the user pays or transfers money in the application, for example, when the user uses the payment application / financial application / chat application / shopping application to make a payment or transfer. The face recognition method provided by the embodiment of the present application can also be used for face recognition when the user registers an account or logs in to an account in the application, for example, when the user Face recognition is performed when registering an account or logging in to an account in the application. Of course, the face recognition method provided in the embodiment of the present application can also be used in other scenarios, which are not specifically limited here.

[0312] The following description is made by taking the working mode of the TOF camera determined by the sensor node as the Tx off mode as an example.

[0313] Figure 13 This is another flow chart of a face recognition method provided in an embodiment of the present application.

[0314] like Figure 13 As shown, after step S118, steps S184-S195 are also included:

[0315] In step S184, the sensor node of the camera HAL determines, based on the human eye safety detection result, that the working mode of the TOF camera is the Tx-off mode.

[0316] In step S185, the sensor node sends the configuration parameters of the Tx-off mode to the camera driving module.

[0317] In step S186, the camera driving module writes the configuration parameters of the Tx-off mode into the register of the TOF camera to drive the TOF camera to collect data.

[0318] That is, the camera driving module can send the configuration parameters of the Tx-off mode to the TOF camera.

[0319] For example, the camera driving module can write the configuration parameters of the Tx-off mode into the register of the TOF camera through I2C, that is, send the configuration parameters of the Tx-off mode to the TOF camera through I2C.

[0320] In step S187, the TOF camera collects RAW Data 4 based on the Tx-off mode.

[0321] The RAW Data 4 can be image data obtained by the Rx of the TOF camera receiving ambient emitted light and imaging when the Tx of the TOF camera is not powered on and does not emit light, which is usually a “black picture” without a clear face image.

[0322] In addition, the RAW Data 4 can also include Metadata. For example, the Metadata can include the current working mode of the TOF camera (for example, the face Tx-off mode), the configuration parameters of the current working mode (for example, the configuration parameters of the Tx-off mode), the working state of the TOF camera device, etc.

[0323] In step S188, the TOF camera transmits the RAW Data 4 to the IFE module.

[0324] For example, the TOF camera can transmit the RAW Data 4 collected by the TOF camera to the IFE module through MIPI.

[0325] In step S189, the IFE module sends the RAW Data 4 to the secure memory for storage.

[0326] The storage location of the RAW Data 4 collected by the TOF camera based on the Tx-off mode in the secure memory can be represented by FD 4.

[0327] In step S190, the IFE module sends the FD 4 to the camera driving module.

[0328] Step S191, the camera driving module sends the FD 4 to the IFE node.

[0329] Step S192, the IFE node sends the FD 4 to the camera service through the interface of the camera HAL.

[0330] Step S193, the camera service sends the FD 4 to the face recognition CA.

[0331] Step S194, the face recognition CA sends the FD 4 to the face recognition TA.

[0332] Step S195, the face recognition TA reads the RAW Data 4 from the secure memory according to the FD 4.

[0333] Step S196, the face recognition TA obtains the face recognition result according to the RAW Data 4.

[0334] Specifically, the face recognition TA can determine that the TOF camera is currently working in the Tx-off mode based on the Metadata data in the RAW Data 4. When the TOF camera works in the Tx-off mode, the face recognition result is failure. This is because the TOF camera cannot emit light in the Tx-off mode, so the TOF camera cannot collect a clear face image, and therefore the face recognition result is failure.

[0335] Step S197, the face recognition TA sends the face recognition result to the face recognition CA.

[0336] That is, the face recognition TA can notify the face recognition CA that the face recognition result is failure.

[0337] Step S198, the face recognition CA sends the face recognition result to the face recognition service.

[0338] The face recognition CA sends the face recognition result to the face recognition service based on the callback registered by the face recognition service previously. That is, the face recognition CA notifies the face recognition service that the face recognition result is failure.

[0339] Step S199, the face recognition service sends the face recognition result to the face recognition SDK.

[0340] The face recognition service sends the face recognition result to the face recognition SDK based on the callback registered by the face recognition SDK previously. That is, the face recognition service can notify the face recognition SDK that the face recognition result is failure.

[0341] Step S200, the face recognition SDK sends the face recognition result to the lock screen application.

[0342] The face recognition SDK sends the face recognition result to the lock screen application based on the callback registered by the lock screen application before. That is, the face recognition SDK can notify the lock screen application that the face recognition result is a failure.

[0343] In step S201, the lock screen application determines not to unlock according to the face recognition result.

[0344] Since the face recognition result is a failure, the lock screen application does not unlock.

[0345] For example, as shown in (a) of FIG. 7, the mobile phone can display an interface 708, which can include an unlock icon 709 (the style of the unlock icon 709 can be a closed lock head, which can visually prompt the user that the face is not successfully unlocked) and a prompt text "unrecognized successfully, double-click the screen to retry" 710. In response to a double-click operation of the user, the mobile phone can perform face recognition again (that is, collect the face information of the user again for comparison and anti-fraud judgment). Figure 12 Figure 12 For example, as shown in (b) of FIG. 7, the mobile phone can display an interface 711. After entering the interface 711, the mobile phone can perform face recognition again. The interface 711 can include a face recognition icon 712 and a prompt text "performing face recognition" 713. If the face recognition is still not successful, as shown in (c) of FIG. 7, the mobile phone can display an interface 714, which can include a prompt text "unrecognized successfully, click here to retry" 715. The user can click the corresponding position to trigger face recognition again, or can also input a password through a soft keyboard 716 to unlock, so as to avoid the problem that the user experience is low due to the fact that the face recognition is always not successful. Figure 12

[0346] Figure 14 FIG. 8 is another flowchart of a face recognition method provided by an embodiment of the present application.

[0347] As shown in FIG. 8, in an embodiment, the method can include the following steps S801-S803. Figure 14 In step S801, the electronic device controls the TOF camera to collect a first image frame based on a first exposure time.

[0348] In step S802, in a case where the brightness of the first image frame is greater than a preset brightness threshold, the electronic device determines a second exposure time, and the second exposure time is less than the first exposure time.

[0349] In step S803, the electronic device controls the TOF camera to collect a second image frame based on the second exposure time.

[0350]

[0351] ​​​The method provided by the embodiments of the present application reduces the exposure time of the TOF camera when the brightness of the first image frame collected by the TOF is greater than the preset brightness threshold, that is, when the face ID mode enters the skin safety mode, thereby reducing the time for which the light signal irradiates the human skin, reducing the energy of the light signal absorbed by the skin, avoiding damage to the human skin caused by the light signal, and protecting the skin safety.

[0352] In an implementation manner, the electronic device determines the second exposure time when the brightness of the first image frame is greater than the preset brightness threshold, including: the electronic device determines the first brightness of the first image frame, the first brightness being the average value of the brightness of all pixels in the gray-scale image corresponding to the first image frame; and the electronic device determines the second exposure time when the first brightness is greater than the preset brightness threshold. In this way, the electronic device can perform skin safety detection based on the brightness of the gray-scale image corresponding to the first image frame, and no longer rely on the PD value of the light receiver in the Tx, so that when the electronic device is equipped with the TOF camera with a baffle as shown in the drawings, the electronic device can also implement skin safety detection, and the adaptability of the face recognition method is improved. Figure 6

[0353] In an implementation manner, the electronic device determines the second exposure time when the first brightness is greater than the preset brightness threshold, including: the electronic device determines the second exposure time when the first brightness of n first image frames continuously collected by the TOF camera is greater than the brightness threshold, n being a positive integer greater than 1. When the first brightness of the first image frame is greater than the brightness threshold and lasts for a certain number of frames, it indicates that the TOF camera may be continuously directed towards the user, and thus there is a skin safety risk. In this case, the electronic device further reduces the exposure time of the TOF camera, thereby facilitating improvement of the accuracy of entering the skin safety mode.

[0354] In an implementation manner, after the electronic device controls the TOF camera to collect the second image frame based on the second exposure time, the method further includes: when the brightness of the second image frame is less than the brightness threshold, the electronic device controls the TOF camera to collect the image frame based on the first exposure time. When the brightness of the second image frame is less than the brightness threshold, it indicates that the skin safety risk is eliminated, and thus the electronic device can control the TOF camera to return to collecting the image frame based on the first exposure time, thereby obtaining an image frame containing more information, and facilitating improvement of the speed of face recognition.

[0355] ​In an implementation, the TOF camera includes a transmitter and a receiver; the first exposure time includes a first depth exposure time of the transmitter and a first grayscale exposure time of the receiver, and the second exposure time includes a second depth exposure time of the transmitter and a second grayscale exposure time of the receiver; the second depth exposure time is shorter than the first depth exposure time, and the second grayscale exposure time is shorter than the first grayscale exposure time. In this way, in the skin safety mode, the intensity of the signal light emitted by the TOF camera can remain unchanged compared with the face ID mode, but the time length of the emitted and received light signals is shortened. In this way, the energy of the light signal absorbed by the skin is reduced, the skin safety is protected, the face recognition can be normally performed, and the power consumption during the face recognition is reduced, achieving three goals at one time.

[0356] Steps S801-S803 and their respective implementations can refer to the above examples for implementation, and will not be described here.

[0357] Some embodiments of the present application provide an electronic device, which can include a touch screen, a memory and one or more processors. The touch screen, the memory and the processor are coupled. The memory is configured to store computer program code, which includes computer instructions. When the processor executes the computer instructions, the electronic device can perform various functions or steps performed by the electronic device in the above method embodiments. The structure of the electronic device can refer to the structure of the electronic device 100 shown in Figure 1

[0358] The embodiments of the present application also provide a chip system, for example, a system on a chip (SoC), as shown in Figure 15 The chip system includes at least one processor 1101 and at least one interface circuit 1102. The processor 1101 and the interface circuit 1102 can be interconnected by a line. For example, the interface circuit 1102 can be used to receive signals from other devices (for example, the memory of the electronic device). For another example, the interface circuit 1102 can be used to send signals to other devices (for example, the processor 1101 or the touch screen of the electronic device). For example, the interface circuit 1102 can read the instructions stored in the memory and send the instructions to the processor 1101. When the instructions are executed by the processor 1101, the electronic device can perform various steps in the above embodiments. Of course, the chip system can also include other discrete devices, which are not limited in the embodiments of the present application.

[0359] The embodiments of the present application also provide a TOF camera, which can be used to implement the eye safety mode, the face ID mode, the skin safety mode and the Tx-off mode in the above embodiments. The electronic device installed with the TOF camera can perform various functions or steps performed by the electronic device in the above method embodiments. ​

[0360] The embodiment of the present application further provides a computer readable storage medium, which comprises computer instructions, and when the computer instructions are run on the electronic device, the electronic device is caused to perform each function or step performed by the electronic device in the method embodiment.

[0361] The embodiment of the present application further provides a computer program product, and when the computer program product is run on the electronic device, the electronic device is caused to perform each function or step performed by the electronic device in the method embodiment.

[0362] Through the description of the above implementation mode, those skilled in the art can clearly understand that, for the convenience and brevity of description, only the above-mentioned division of each functional module is taken as an example for illustration, and in actual application, the above-mentioned functions can be completed by different functional modules according to needs, that is, the internal structure of the device is divided into different functional modules to complete all or part of the functions described above.

[0363] In several embodiments provided in the present application, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiment described above is only schematic, for example, the division of the modules or units is only a logical function division, and actual implementation can have another division manner, for example, a plurality of units or components can be combined or integrated into another device, or some features can be ignored or not executed. In addition, the coupling or direct coupling or communication connection between the displayed or discussed each other can be through some interface, indirect coupling or communication connection between the devices or units, which can be electrical, mechanical or other forms.

[0364] The units described as separate components can or can not be physically separated, and the components displayed as units can be one physical unit or multiple physical units, that is, can be located in one place, or can be distributed to multiple different places. Part or all of the units can be selected according to actual needs to achieve the purpose of the embodiment scheme.

[0365] In addition, each functional unit in each embodiment of the present application can be integrated in one processing unit, or each unit can exist physically independently, or two or more units can be integrated in one unit. The integrated unit can be realized in the form of hardware or in the form of software functional unit.

[0366] The integrated unit, if implemented in the form of a software function unit and sold or used as an independent product, can be stored in a readable storage medium. Based on such understanding, the technical solutions of the embodiments of the present application essentially or say the part that contributes to the prior art or the whole or part of the technical solutions can be embodied in the form of a software product. The software product is stored in a storage medium, including a plurality of instructions to make a device (which can be a single-chip microcomputer, a chip, etc.) or a processor execute all or part of the steps of the method described in various embodiments of the present application. The aforementioned storage medium includes: a U disk, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, and various media that can store program codes.

[0367] The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any change or replacement within the technical scope disclosed in the present application should be covered within the protection scope of the present application. Therefore, the protection scope of the present application should be subject to the protection scope of the claims.

Claims

1. A face recognition method, applied to an electronic device, wherein the electronic device includes a time-of-flight (TOF) camera, characterized in that: The method comprises: controlling the TOF camera to capture a first image frame based on a first exposure time; in a case where the brightness of the first image frame is greater than a preset brightness threshold, determining a second exposure time based on a corresponding relationship between the irradiation time of the light signal on the skin and the upper limit value of the safe energy of the light signal absorbed by the skin, so that when the TOF camera emits the light signal of the second exposure time to irradiate the skin, the energy value of the light signal absorbed by the skin is lower than or equal to the upper limit value of the safe energy corresponding to the second exposure time; controlling the TOF camera to capture a second image frame based on the second exposure time.

2. The face recognition method of claim 1, wherein, In a case where the brightness of the first image frame is greater than a preset brightness threshold, determining a second exposure time, comprising: determining a first brightness of the first image frame, the first brightness being the average brightness of all pixels in the grayscale image corresponding to the first image frame; in a case where the first brightness is greater than the brightness threshold, determining the second exposure time.

3. The face recognition method of claim 2, wherein, In a case where the first brightness is greater than the brightness threshold, determining the second exposure time, comprising: in a case where the first brightness of n frames of the first image frame continuously captured by the TOF camera is greater than the brightness threshold, determining the second exposure time, n being a positive integer greater than 1.

4. The face recognition method of claim 1, wherein, After the control of the TOF camera to capture a second image frame based on the second exposure time, further comprising: in a case where the brightness of the second image frame is less than the brightness threshold, controlling the TOF camera to capture an image frame based on the first exposure time.

5. The face recognition method of claim 1, wherein, The TOF camera comprises a transmitter and a receiver; the first exposure time comprises a first depth exposure time of the transmitter and a first grayscale exposure time of the receiver, and the second exposure time comprises a second depth exposure time of the transmitter and a second grayscale exposure time of the receiver; the second depth exposure time is less than the first depth exposure time, and the second grayscale exposure time is less than the first grayscale exposure time.

6. The face recognition method of claim 2, wherein, The electronic device comprises a camera hardware abstraction layer, and the control of the TOF camera to capture a first image frame based on a first exposure time comprises: The camera hardware abstraction layer configures the TOF camera with first configuration parameters corresponding to a first working mode, the first configuration parameters comprising the first exposure time and a first working current of the TOF camera; The TOF camera captures the first image frame based on the first configuration parameters.

7. The face recognition method according to claim 6, wherein In a case where the brightness of the first image frame is greater than a preset brightness threshold, determining a second exposure time, comprising: in a case where the brightness of the first image frame is greater than the brightness threshold, the camera hardware abstraction layer configures the TOF camera with second configuration parameters corresponding to a second working mode, the second configuration parameters comprising the second exposure time and the first working current; The control of the TOF camera to capture a second image frame based on the second exposure time comprises: The TOF camera collects the second image frame based on the second configuration parameter.

8. The face recognition method of claim 7, wherein, The electronic device comprises a face trust application, and after the TOF camera collects the first image frame based on the first configuration parameter, the electronic device further comprises: The face trust application acquires a gray image corresponding to the first image frame; The face trust application determines the first brightness based on the gray image corresponding to the first image frame; The face trust application obtains a detection result based on the first brightness and the brightness threshold value, wherein, in a case where the first brightness of n frames of the first image frame continuously collected by the TOF camera is greater than the brightness threshold value, the face trust application obtains a first detection result, and in a case where the first brightness of n frames of the first image frame continuously collected by the TOF camera is not greater than the brightness threshold value, the face trust application obtains a second detection result.

9. The face recognition method of claim 8, wherein, The electronic device comprises a face recognition control module, and after the face trust application obtains a detection result based on the first brightness and the brightness threshold value, the electronic device further comprises: The face trust application sends the detection result to the face recognition control module; The face recognition control module sends the detection result to the camera hardware abstraction layer.

10. The face recognition method of claim 9, wherein, In a case where the camera hardware abstraction layer receives the first detection result, the camera hardware abstraction layer configures the second configuration parameter corresponding to the second working mode for the TOF camera. After the TOF camera collects the second image frame based on the second configuration parameter, the electronic device further comprises:

11. The face recognition method of claim 9, wherein, The face trust application acquires a gray image corresponding to the second image frame; The face trust application determines a second brightness of the second image frame based on the gray image corresponding to the second image frame, the second brightness being an average value of brightness of all pixels in the gray image corresponding to the second image frame; The face trust application obtains a comparison result based on the second brightness and the brightness threshold value, wherein, in a case where the second brightness is less than the brightness threshold value, the face trust application obtains a first comparison result, and in a case where the second brightness is greater than or equal to the brightness threshold value, the face trust application obtains a second comparison result. After the face trust application obtains a comparison result based on the second brightness and the brightness threshold value, the electronic device further comprises:

12. The face recognition method of claim 11, wherein, The face trust application sends the comparison result to the face recognition control module; The face recognition control module sends the comparison result to the camera hardware abstraction layer; In a case where the camera hardware abstraction layer receives the first comparison result, the camera hardware abstraction layer configures the first configuration parameter corresponding to the first working mode for the TOF camera.

13. The face recognition method according to any one of claims 1-12, characterized in that, The first image frame and the second image frame are both used for face recognition. including:

14. An electronic device, comprising: ​ The electronic device comprises a memory, a TOF camera and one or more processors; the memory, the TOF camera and the processor are coupled; wherein the memory stores computer program code, the computer program code comprises computer instructions, and when the computer instructions are executed by the processor, the electronic device performs the face recognition method as claimed in any one of claims 1-13.

Images