Face recognition method and electronic equipment

Abstract

The embodiment of the invention provides a face recognition method and electronic equipment. The face recognition method can be applied to an electronic device comprising a TOF camera, and the method comprises the following steps: controlling the TOF camera to collect a first image frame based on a first exposure duration; under the condition that the brightness of the first image frame is greater than a preset brightness threshold value, determining a second exposure duration which is smaller than the first exposure duration; and controlling the TOF camera to collect a second image frame based on the second exposure duration. According to the method provided by the embodiment of the invention, under the condition that the brightness of the first image frame acquired by the TOF is greater than the preset brightness threshold value, the exposure duration of the TOF camera is reduced, so that the duration that the light signal irradiates the human skin is reduced, the energy of the light signal absorbed by the skin is reduced, the damage of the light signal to the human skin is avoided, and the skin safety is protected.

Description

Technical Field

[0001] The present application relates to the field of terminal technology, and in particular to a face recognition method and electronic device. Background Art

[0002] The time of flight (TOF) camera module may include a transmitter (Tx) and a receiver (Rx), wherein the transmitter is used to emit infrared light or laser pulses, and the receiver is used to receive reflected light and form an image. In the face recognition scenario, the TOF camera can collect the depth information of the face, so as to better identify the three-dimensional structure of the face. Therefore, compared with traditional 2D images, the images collected by the TOF camera module can enhance the uniqueness of the face, thereby improving the accuracy of recognition.

[0003] When the TOF camera collects images, the transmitter needs to project a light signal toward the human face. That is to say, the light signal projected by the transmitter will shine on the skin of the human face and other parts. Therefore, it is necessary to avoid the light signal projected by the transmitter from causing damage to the human skin. Summary of the invention

[0004] The embodiments of the present application provide a face recognition method and an electronic device, which can ensure that the light signal emitted by TOF does not cause damage to human skin, protect skin safety, and thus improve the user experience.

[0005] In a first aspect, an embodiment of the present application provides a face recognition method, which can be applied to an electronic device including a TOF camera, and the method includes the following steps: controlling the TOF camera to capture a first image frame based on a first exposure time; when the brightness of the first image frame is greater than a preset brightness threshold, determining a second exposure time, and the second exposure time is less than the first exposure time; controlling the TOF camera to capture a second image frame based on the second exposure time.

[0006] The method provided in the embodiment of the present application reduces the exposure time of the TOF camera when the brightness of the first image frame captured by TOF is greater than a preset brightness threshold, that is, enters the skin safety mode from the face ID mode, thereby reducing the time that the light signal is irradiated on the human skin, reducing the energy of the light signal absorbed by the skin, avoiding damage to the human skin by the light signal, and protecting the skin safety.

[0007] In one implementation, when the brightness of the first image frame is greater than a preset brightness threshold, determining the second exposure time includes: determining a first brightness of the first image frame, the first brightness being the average brightness of all pixels in the grayscale image corresponding to the first image frame; and determining the second exposure time when the first brightness is greater than the brightness threshold. In this way, the electronic device can perform skin safety detection based on the brightness of the grayscale image corresponding to the first image frame, and no longer rely on the PD value of the receiver in Tx. Therefore, when the electronic device is equipped with Figure 6 When using a TOF camera with a baffle as shown, the electronic device can also achieve skin safety detection, improving the adaptability of the face recognition method.

[0008] In one implementation, when the first brightness is greater than the brightness threshold, determining the second exposure time includes: when the first brightness of n first image frames continuously captured by the TOF camera is greater than the brightness threshold, determining the second exposure time, where n is a positive integer greater than 1. When the first brightness of the first image frame is greater than the brightness threshold and lasts for a certain number of frames, it means that the TOF camera may be continuously facing the user, so there is a skin safety risk. In this case, the electronic device further reduces the exposure time of the TOF camera, which is conducive to improving the accuracy of entering the skin safety mode.

[0009] In one implementation, after controlling the TOF camera to capture the second image frame based on the second exposure duration, the method further includes: when the brightness of the second image frame is less than the brightness threshold, controlling the TOF camera to capture the image frame based on the first exposure duration. When the brightness of the second image frame is less than the brightness threshold, it indicates that the skin safety risk is eliminated. Therefore, the electronic device can control the TOF camera to resume capturing the image frame based on the first exposure duration, thereby obtaining an image frame containing more information, which is conducive to improving the speed of face recognition.

[0010] In one implementation, the TOF camera includes a transmitter and a receiver; the first exposure time includes a first depth exposure time of the transmitter and a first grayscale exposure time of the receiver, and the second exposure time includes a second depth exposure time of the transmitter and a second grayscale exposure time of the receiver; the second depth exposure time is less than the first depth exposure time, and the second grayscale exposure time is less than the first grayscale exposure time. In this way, in the skin safety mode, the intensity of the signal light emitted by the TOF camera can remain unchanged compared to the face ID mode, but the duration of emitting and receiving the light signal is shortened. In this way, the energy of the light signal absorbed by the skin is reduced to protect the skin safety, and face recognition can be performed normally, and the power consumption during face recognition can be reduced, achieving three goals at one stroke.

[0011] In one implementation, the electronic device includes a camera hardware abstraction layer, and controls the TOF camera to collect a first image frame based on a first exposure duration, including: the camera hardware abstraction layer configures a first configuration parameter corresponding to a first working mode (i.e., a face ID mode) for the TOF camera, the first configuration parameter including a first exposure duration and a first working current of the TOF camera; and the TOF camera collects the first image frame based on the first configuration parameter. In this way, the TOF camera can work in the face ID mode based on the first configuration parameter to collect images.

[0012] In one implementation, when the brightness of the first image frame is greater than a preset brightness threshold, determining the second exposure duration includes: when the brightness of the first image frame is greater than the brightness threshold, the camera hardware abstraction layer configures the second configuration parameters corresponding to the second working mode for the TOF camera, the second configuration parameters including the second exposure duration and the first working current; controlling the TOF camera to collect the second image frame based on the second exposure duration includes: the TOF camera collects the second image frame based on the second configuration parameters. In this way, the TOF camera can work in the skin safety mode based on the second configuration parameters to collect images.

[0013] In one implementation, the electronic device includes a face trust application, and after the camera module captures the first image frame based on the first configuration parameter, it also includes: the face trust application obtains the grayscale image corresponding to the first image frame; the face trust application determines the first brightness based on the grayscale image corresponding to the first image frame; the face trust application obtains a detection result based on the first brightness and the brightness threshold, wherein, when the first brightness of the n frames of the first image frame continuously captured by the TOF camera is greater than the brightness threshold, the face trust application obtains the first detection result, and when the first brightness of the n frames of the first image frame continuously captured by the TOF camera is not greater than the brightness threshold, the face trust application obtains the second detection result. In this way, the face trust application can obtain different skin safety detection results based on the different numerical relationships between the first brightness and the brightness threshold, thereby providing a reference basis for whether the TOF camera switches to the skin safety mode.

[0014] In one implementation, the electronic device includes a face recognition control module. After the face trusted application obtains the detection result based on the first brightness and the brightness threshold, it also includes: the face trusted application sends the detection result to the face recognition control module; the face recognition control module sends the detection result to the camera hardware abstraction layer.

[0015] In one implementation, when the brightness of the first image frame is greater than the brightness threshold, the camera hardware abstraction layer configures the second configuration parameters corresponding to the second working mode for the TOF camera, including: when the camera hardware abstraction layer receives the first detection result, the camera hardware abstraction layer configures the second configuration parameters corresponding to the second working mode for the TOF camera. In this way, the camera hardware abstraction layer can switch the working mode of the TOF camera to the skin safety mode based on the received skin safety detection result.

[0016] In one implementation, after the TOF camera captures the second image frame based on the second configuration parameter, it also includes: the face trusted application obtains the grayscale image corresponding to the second image frame; the face trusted application determines the second brightness of the second image frame based on the grayscale image corresponding to the second image frame, and the second brightness is the average brightness of all pixels in the grayscale image corresponding to the second image frame; the face trusted application obtains a comparison result based on the second brightness and the brightness threshold, wherein when the second brightness is less than the brightness threshold, the face trusted application obtains the first comparison result, and when the second brightness is greater than or equal to the brightness threshold, the face trusted application obtains the second comparison result. In this way, the face trusted application can obtain different comparison results based on the different numerical relationships between the second brightness and the brightness threshold, thereby providing a reference basis for whether the TOF camera switches to the face ID mode.

[0017] In one implementation, after the face trusted application obtains the comparison result based on the second brightness and the brightness threshold, it also includes: the face trusted application sends the comparison result to the face recognition control module; the face recognition control module sends the comparison result to the camera hardware abstraction layer; when the camera hardware abstraction layer receives the first comparison result, the camera hardware abstraction layer configures the first configuration parameter corresponding to the first working mode for the TOF camera. In this way, the camera hardware abstraction layer can switch the working mode of the TOF camera to the face ID mode based on the received comparison result.

[0018] In one implementation, both the first image frame and the second image frame are used for face recognition. In the embodiment of the present application, the second image frame collected in the face ID mode and the skin safety mode can be used for face recognition. Therefore, even if the TOF camera works in the skin safety mode, it will not affect the normal face recognition process, thereby improving the user experience.

[0019] In a second aspect, an embodiment of the present application provides an electronic device, which includes a memory, a TOF camera and one or more processors; the memory, the TOF camera and the processor are coupled; wherein the memory stores computer program code, the computer program code includes computer instructions, and when the computer instructions are executed by the processor, the electronic device executes the face recognition method provided in the first aspect and any possible design method thereof.

[0020] In a third aspect, an embodiment of the present application provides a computer-readable storage medium, which includes computer instructions. When the computer instructions are executed on an electronic device, the electronic device executes a face recognition method as provided in the first aspect and any possible design thereof.

[0021] In a fourth aspect, an embodiment of the present application provides a computer program product. When the computer program product is run on an electronic device, the electronic device executes the face recognition method provided in the first aspect and any possible design thereof.

[0022] It can be understood that the beneficial effects that can be achieved by the technical solutions provided in the second to fifth aspects provided above can refer to the beneficial effects in the first aspect and any possible design method thereof, and will not be repeated here. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] Figure 1 A schematic diagram of the structure of an electronic device 100 provided in an embodiment of the present application;

[0024] Figure 2 It is a schematic diagram of the principle of a TOF imaging technology provided in an embodiment of the present application;

[0025] Figure 3 is a software architecture diagram of an electronic device provided in an embodiment of the present application;

[0026] Figure 4 is a schematic diagram of software modules and interactions between modules involved in the face recognition method provided in an embodiment of the present application;

[0027] Figure 5 1 is a schematic diagram of the structure of a Tx of a TOF camera provided in an embodiment of the present application;

[0028] Figure 6 is a schematic diagram of the structure of another TOF camera Tx provided in an embodiment of the present application;

[0029] Figure 7 is another schematic diagram of the interaction between software modules and modules involved in the face recognition method provided in an embodiment of the present application;

[0030] Figure 8 is a flow chart of a face recognition method provided by an embodiment of the present application;

[0031] Fig. 9 is another flow chart of a face recognition method provided by an embodiment of the present application;

[0032] Fig.10 This is a schematic diagram of an interaction of face unlocking provided in an embodiment of the present application;

[0033] Fig.11 This is another interactive schematic diagram of a face unlocking method provided in an embodiment of the present application;

[0034] Fig.12 This is another interactive schematic diagram of face unlocking provided by an embodiment of the present application;

[0035] Fig.13 is another flow chart of a face recognition method provided in an embodiment of the present application;

[0036] Fig.14 is another flow chart of a face recognition method provided in an embodiment of the present application;

[0037] Fig.15 It is a schematic diagram of a chip system provided in an embodiment of the present application. DETAILED DESCRIPTION

[0038] The technical scheme in the embodiment of the present application will be described below in conjunction with the drawings in the embodiment of the present application. Wherein, in the description of the present application, unless otherwise specified, "at least one" refers to one or more, and "a plurality of" refers to two or more than two. In addition, in order to facilitate the clear description of the technical scheme of the embodiment of the present application, in the embodiment of the present application, the words "first", "second" and the like are used to distinguish the same items or similar items with substantially the same functions and effects. Those skilled in the art will understand that the words "first", "second" and the like do not limit the quantity and execution order, and the words "first", "second" and the like do not limit necessarily different.

[0039] In order to make the description of the following embodiments clear and concise, a brief introduction to the relevant concepts or technologies is first given:

[0040] Rich execution environment (REE), also known as rich execution environment, common execution environment or untrusted execution environment, refers to the system operating environment of the mobile terminal, which can run operating systems such as Android, IOS and Linux. REE is open and extensible but not very secure.

[0041] Trusted execution environment (TEE), also known as the security side or security zone, is an area that requires authorization to access. TEE is an operating environment in which REE coexists with REE in electronic devices. It is isolated from REE through hardware support, has security capabilities, and can resist software attacks that conventional REE sides are susceptible to. TEE has its own operating space and defines strict protection measures. Therefore, it has a higher security level than REE and can protect assets in TEE, such as data and software, from software attacks and resist specific types of security threats.

[0042] REE+TEE architecture refers to an architecture that provides services for applications through the combination of TEE and REE. In other words, TEE and REE exist together in electronic devices. For example, TEE can implement an operating mechanism isolated from REE through hardware support. TEE has its own operating space, with a higher security level than REE, and can protect assets in TEE (such as data, software), etc. from software attacks. Only authorized security software can be executed in TEE, and it also protects the confidentiality of the resources and data of the security software. Compared with REE, TEE can better protect the security of data and resources due to its protection mechanisms such as isolation and permission control.

[0043] TA, or trusted application, is an application running in TEE that can provide security services to CA running outside TEE, such as password input, transaction signature generation, face recognition, etc.

[0044] CA, or client application, usually refers to an application running in REE. CA can call TA through the client application programming interface (API) and instruct TA to perform corresponding security operations.

[0045] Software development kit (SDK): In a broad sense, it refers to a collection of related documents, examples and tools that assist in the development of a certain type of software.

[0046] RAW Data, or raw data, can be understood as "unprocessed and uncompressed data". In the embodiment of the present application, RAW Data may refer to the raw image data that the TOF camera converts the captured light source signal into a digital signal. RAW Data also records some metadata generated by the camera shooting.

[0047] Metadata, also known as intermediate data or relay data, is data about data, mainly information describing data properties. In the embodiment of the present application, Metadata can indicate the camera's working mode, photocurrent value, TOF camera device working status, exposure value and other information.

[0048] Time of flight (TOF) imaging technology refers to an imaging technology in which a group of infrared light (or laser pulses) invisible to the human eye is emitted outward, reflected after encountering an object, and reflected to the camera. The time difference or phase difference from emission to reflection back to the camera is calculated, and the data is collected to form a set of distance depth data, thereby obtaining a three-dimensional 3D model. In other words, TOF imaging technology is based on traditional 2D XY axis imaging, adding depth information from the Z axis direction, and finally generating 3D image information. In the embodiment of the present application, TOF technology may refer to indirect light time of flight (indirect TOF, iToF) technology.

[0049] A TOF camera may include a transmitter (Tx) and a receiver (Rx), wherein the transmitter is used to emit infrared light or laser pulses, and the receiver is used to receive reflected light and form an image. In face recognition scenarios, the TOF camera can capture the depth information of the face, thereby better identifying the three-dimensional structure of the face. Therefore, compared with traditional 2D images, the images captured by the TOF camera module contain depth information, which can enhance the uniqueness of the face and thus improve the accuracy of recognition.

[0050] When the TOF camera collects images, the transmitter needs to project light signals onto the human face. That is to say, the light signals projected by the transmitter will shine on the skin of the human face and other parts. Therefore, it is necessary to avoid the light signals projected by the transmitter from causing damage to the human skin.

[0051] The embodiment of the present application provides a face recognition method, which can prevent the light signal projected by Tx from causing damage to human skin and improve user experience.

[0052] Figure 1 A schematic diagram of the structure of an electronic device 100 provided in an embodiment of the present application.

[0053] like Figure 1As shown, the electronic device 100 may include a processor 110, an external memory interface 120, an internal memory 121, a universal serial bus (USB) interface 130, a charging management module 140, a power management module 141, a battery 142, an antenna 1, an antenna 2, a mobile communication module 150, a wireless communication module 160, an audio module 170, a speaker 170A, a receiver 170B, a microphone 170C, an earphone interface 170D, a sensor module 180, a button 190, a motor 191, an indicator 192, a camera 193, a display screen 194, and a subscriber identification module (SIM) card interface 195, etc.

[0054] The processor 110 may include one or more processing units, for example, the processor 110 may include an application processor (AP), a modem processor, a graphics processor (GPU), an image signal processor (ISP), a controller, a memory, a video codec, a digital signal processor (DSP), a baseband processor, and / or a neural-network processing unit (NPU), etc. Different processing units may be independent devices or integrated into one or more processors.

[0055] The controller may be the nerve center and command center of the electronic device 100. The controller may generate an operation control signal according to the instruction operation code and the timing signal to complete the control of fetching and executing instructions.

[0056] NPU is a neural network (NN) computing processor. By drawing on the structure of biological neural networks, such as the transmission mode between neurons in the human brain, it can quickly process input information and can also continuously self-learn. Through NPU, applications such as intelligent cognition of electronic device 100 can be realized, such as image recognition, face recognition, voice recognition, text understanding, etc.

[0057] The processor 110 may also be provided with a memory for storing instructions and data. In some embodiments, the memory in the processor 110 is a cache memory. The memory may store instructions or data that the processor 110 has just used or cyclically used. If the processor 110 needs to use the instruction or data again, it may be directly called from the memory. This avoids repeated access, reduces the waiting time of the processor 110, and thus improves the efficiency of the system.

[0058] In some embodiments, the processor 110 may include one or more interfaces. The interface may include an inter-integrated circuit (I2C) interface, an inter-integrated circuits sound (I2S) interface, a pulse code modulation (PCM) interface, a universal asynchronous receiver / transmitter (UART) interface, a mobile industry processor interface (MIPI), a general-purpose input / output (GPIO) interface, a subscriber identity module (SIM) interface, and / or a universal serial bus (USB) interface, etc.

[0059] It is understandable that the interface connection relationship between the modules shown in this embodiment is only a schematic illustration and does not constitute a structural limitation on the electronic device 100. In other embodiments, the electronic device 100 may also adopt different interface connection methods in the above embodiments, or a combination of multiple interface connection methods.

[0060] The external memory interface 120 can be used to connect an external memory card, such as a Micro SD card, to expand the storage capacity of the electronic device 100. The external memory card communicates with the processor 110 through the external memory interface 120 to implement a data storage function. For example, files such as music and videos are stored in an external memory card. The internal memory 121 can be used to store computer executable program codes, and the executable program codes include instructions. The processor 110 can execute various functional applications and data processing of the electronic device 100 by running the instructions stored in the internal memory 121. For example, in an embodiment of the present application, the processor 110 can execute instructions stored in the internal memory 121, and the internal memory 121 may include a program storage area and a data storage area. Among them, the program storage area can store an operating system, an application required for at least one function (such as a sound playback function, an image playback function, etc.), etc. The data storage area can store data created during the use of the electronic device 100 (such as audio data, a phone book, etc.), etc. In addition, the internal memory 121 may include a high-speed random access memory and may also include a non-volatile memory, such as at least one disk storage device, a flash memory device, a universal flash storage (UFS), etc.

[0061] The charging management module 140 is used to receive charging input from a charger. While the charging management module 140 is charging the battery 142 , it can also power the electronic device through the power management module 141 .

[0062] The power management module 141 is used to connect the battery 142, the charging management module 140 and the processor 110. The power management module 141 receives input from the battery 142 and / or the charging management module 140, and supplies power to the processor 110, the internal memory 121, the external memory, the display screen 194, the camera 193, and the wireless communication module 160. In some other embodiments, the power management module 141 may also be provided in the processor 110. In other embodiments, the power management module 141 and the charging management module 140 may also be provided in the same device.

[0063] The wireless communication function of the electronic device 100 can be implemented through the antenna 1, the antenna 2, the mobile communication module 150, the wireless communication module 160, the modem processor and the baseband processor.

[0064] Antenna 1 and antenna 2 are used to transmit and receive electromagnetic wave signals. Each antenna in electronic device 100 can be used to cover a single or multiple communication frequency bands. Different antennas can also be reused to improve the utilization of antennas. For example, antenna 1 can be reused as a diversity antenna for a wireless local area network.

[0065] The mobile communication module 150 can provide solutions for wireless communications including 2G / 3G / 4G / 5G, etc., applied to the electronic device 100. The mobile communication module 150 may include at least one filter, a switch, a power amplifier, a low noise amplifier (LNA), etc. The mobile communication module 150 can receive electromagnetic waves from the antenna 1, and filter, amplify, etc. the received electromagnetic waves, and transmit them to the modulation and demodulation processor for demodulation. The mobile communication module 150 can also amplify the signal modulated by the modulation and demodulation processor, and convert it into electromagnetic waves for radiation through the antenna 1.

[0066] The modem processor may include a modulator and a demodulator. The modulator is used to modulate the low-frequency baseband signal to be sent into a medium-high frequency signal. The demodulator is used to demodulate the received electromagnetic wave signal into a low-frequency baseband signal. The demodulator then transmits the demodulated low-frequency baseband signal to the baseband processor for processing. After the low-frequency baseband signal is processed by the baseband processor, it is passed to the application processor. The application processor outputs a sound signal through an audio device (not limited to a speaker 170A, a receiver 170B, etc.), or displays an image or video through a display screen 194.

[0067] The wireless communication module 160 can provide wireless communication solutions including wireless local area networks (WLAN) (such as wireless fidelity (Wi-Fi) networks), bluetooth (BT), global navigation satellite system (GNSS), frequency modulation (FM), near field communication (NFC), infrared (IR), etc., which are applied to the electronic device 100. The wireless communication module 160 can be one or more devices integrating at least one communication processing module. The wireless communication module 160 receives electromagnetic waves via the antenna 2, modulates the frequency of the electromagnetic wave signal and performs filtering, and sends the processed signal to the processor 110. The wireless communication module 160 can also receive the signal to be sent from the processor 110, modulate the frequency of it, amplify it, and convert it into electromagnetic waves for radiation through the antenna 2.

[0068] In some embodiments, the antenna 1 of the electronic device 100 is coupled to the mobile communication module 150, and the antenna 2 is coupled to the wireless communication module 160, so that the electronic device 100 can communicate with the network and other devices through wireless communication technology. The wireless communication technology may include global system for mobile communications (GSM), general packet radio service (GPRS), code division multiple access (CDMA), wideband code division multiple access (WCDMA), time-division code division multiple access (TD-SCDMA), long term evolution (LTE), BT, GNSS, WLAN, NFC, FM, and / or IR technology. The GNSS may include a global positioning system (GPS), a global navigation satellite system (GLONASS), a Beidou navigation satellite system (BDS), a quasi-zenith satellite system (QZSS) and / or a satellite based augmentation system (SBAS).

[0069] The electronic device 100 can implement audio functions such as music playing and recording through the audio module 170, the speaker 170A, the receiver 170B, the microphone 170C, the headphone jack 170D, and the application processor.

[0070] The audio module 170 is used to convert digital audio information into analog audio signal output, and is also used to convert analog audio input into digital audio signals. The audio module 170 can also be used to encode and decode audio signals. The speaker 170A, also known as the "speaker", is used to convert audio electrical signals into sound signals. The receiver 170B, also known as the "earpiece", is used to convert audio electrical signals into sound signals. The microphone 170C, also known as the "microphone" or "microphone", is used to convert sound signals into electrical signals. The headphone jack 170D is used to connect wired headphones.

[0071] Among them, the sensor module 180 may include a pressure sensor 180A, a gyroscope sensor 180B, an air pressure sensor 180C, a magnetic sensor 180D, an acceleration sensor 180E, a distance sensor 180F, a proximity light sensor 180G, a fingerprint sensor 180H, a temperature sensor 180J, a touch sensor 180K, an ambient light sensor 180L, a bone conduction sensor 180M, etc.

[0072] The button 190 includes a power button, a volume button, etc. The button 190 can be a mechanical button. It can also be a touch button. The electronic device 100 can receive the button input and generate a key signal input related to the user settings and function control of the electronic device 100. The motor 191 can generate a vibration prompt. The motor 191 can be used for incoming call vibration prompts, and can also be used for touch vibration feedback. The indicator 192 can be an indicator light, which can be used to indicate the charging status, power changes, messages, missed calls, notifications, etc. The SIM card interface 195 is used to connect the SIM card. The SIM card can be inserted into the SIM card interface 195, or pulled out from the SIM card interface 195 to achieve contact and separation with the electronic device 100. The electronic device 100 can support 1 or N SIM card interfaces, where N is a positive integer greater than 1. The SIM card interface 195 can support Nano SIM cards, Micro SIM cards, SIM cards, etc.

[0073] The electronic device 100 implements the display function through a GPU, a display screen 194, and an application processor. The GPU is a microprocessor for image processing, which connects the display screen 194 and the application processor. The GPU is used to perform mathematical and geometric calculations for graphics rendering. The processor 110 may include one or more GPUs that execute program instructions to generate or change display information.

[0074] The display screen 194 is used to display images, videos, etc. The display screen 194 includes a display panel. The display panel can be a liquid crystal display (LCD), a light-emitting diode (LED), an organic light-emitting diode (OLED), an active-matrix organic light emitting diode or an active-matrix organic light emitting diode (AMOLED), a flexible light-emitting diode (FLED), Miniled, MicroLed, Micro-oLed, a quantum dot light emitting diode (QLED), etc.

[0075] The electronic device 100 can realize the shooting function through an ISP, a camera 193, a video codec, a GPU, a display screen 194, and an application processor. The ISP is used to process the data fed back by the camera 193. The camera 193 is used to capture static images or videos. The digital signal processor is used to process digital signals. In addition to processing digital image signals, it can also process other digital signals. The video codec is used to compress or decompress digital videos. The electronic device 100 can support one or more video codecs. In this way, the electronic device 100 can play or record videos in multiple encoding formats, such as: moving picture experts group (MPEG) 1, MPEG2, MPEG3, MPEG4, etc.

[0076] The camera 193 may include 1 to N cameras. For example, the electronic device may include 2 front cameras and 4 rear cameras. Among them, the front camera may include a TOF camera.

[0077] It is to be understood that the structure illustrated in this embodiment does not constitute a specific limitation on the electronic device 100. In other embodiments, the electronic device 100 may include more or fewer components than shown in the figure, or combine some components, or separate some components, or arrange the components differently. The components shown in the figure may be implemented in hardware, software, or a combination of software and hardware.

[0078] The methods in the following embodiments can all be implemented in the electronic device 100 having the above hardware structure.

[0079] Figure 2 It is a schematic diagram of the principle of a TOF imaging technology provided in an embodiment of the present application;

[0080] The TOF camera includes Tx and Rx. Tx can be used to transmit light signals (infrared light or laser pulses), and Rx can be used to receive images. Tx can be, for example, a transmitter, such as a laser diode (LD). Rx can be, for example, a complementary metal oxide semiconductor (CMOS) or a charge coupled device (CCD) image sensor.

[0081] For example, Figure 2 As shown in (a) in FIG. 1 , the transmitter Tx of the TOF camera can continuously send light signals (infrared light or laser pulses) to the target to be measured (for example, the user), and the sensor end Rx of the TOF camera receives the light signal returned by the target to be measured, as shown in FIG. Figure 2As shown in (b) in FIG. 1 , the depth information of the target can be obtained based on the phase difference (delay) between the transmitted and received light signals.

[0082] Among them, Tx and Rx can exchange information through the bus. For example, Rx can send configuration parameters to Tx through the bus (for example, a serial peripheral interface (SPI) bus), and the configuration parameters are used to indicate the address of the register corresponding to Tx and the value of the register. For example, the address of the register corresponding to Tx can be 0x11, and the storage space corresponding to 0x11 can store the current value. Tx can work at the corresponding current value based on the corresponding configuration parameters to emit a light signal of the corresponding light intensity. Rx can receive the reflected light of the light signal emitted by the transmitter and obtain the corresponding image data based on the reflected light. It should be noted that Tx can emit light signals of different light intensities when working at different current values. For example, Tx can emit a light signal of light intensity Q1 when working at the current value I1. Tx can emit a light signal of light intensity Q2 when working at the current value I2. The current value I2 is greater than the current value I1. The light intensity Q2 is greater than the light intensity Q1. The image data obtained by Rx based on the reflected light of the light signal of different intensities is also different. For example, when Tx operates at a current value of I1 and emits a light signal with a light intensity of Q1, Rx obtains first image data within the corresponding exposure time; when Tx operates at a current value of I2 and emits a light signal with a light intensity of Q2, Rx obtains second image data within the corresponding exposure time; the second image data is different from the first image data.

[0083] The software system of the electronic device 100 may adopt a layered architecture, an event-driven architecture, a micro-core architecture, a micro-service architecture, or a cloud architecture. The embodiment of the present invention takes the Android system of the layered architecture as an example to exemplify the software structure of the electronic device 100.

[0084] The layered architecture divides the software into several layers, each with clear roles and division of labor. The layers communicate with each other through interfaces. In some embodiments, the Android system may include an application layer, an application framework layer, an Android runtime (Android runtime) and a system library, a hardware abstraction layer (HAL) and a kernel layer. It should be noted that the embodiments of the present application are illustrated by taking the Android system as an example. In other operating systems (such as IOS systems, etc.), as long as the functions implemented by each functional module are similar to those of the embodiments of the present application, the solutions of the present application can also be implemented.

[0085] Among them, the application layer can include a series of application packages.

[0086] like Figure 3 As shown, the application package may include applications such as camera, gallery, calendar, call, map, navigation, WLAN, Bluetooth, music, video, short message, lock screen application, setting application, etc. Of course, the application layer may also include other application packages, such as payment application, shopping application, banking application, chat application or financial application, etc., which are not limited in this application.

[0087] Among them, the setting application has the function of recording a face, and the recorded face is used for face unlocking. The lock screen application has the function of unlocking in response to the user's unlocking operation (for example, pressing the power button). The lock screen application can perform unlocking processes such as face unlocking, fingerprint unlocking, and password unlocking. The embodiment of this application mainly takes face unlocking as an example for explanation.

[0088] The application framework layer provides an application programming interface (API) and a programming framework for the application programs in the application layer. The application framework layer includes some predefined functions. For example, it may include an activity manager, a window manager, a content provider, a view system, a resource manager, a notification manager, a camera service, and a face recognition service, etc., and the embodiments of the present application do not impose any restrictions on this.

[0089] The system library can include multiple functional modules, such as surface manager, media libraries, OpenGL ES, SGL, etc.

[0090] The surface manager is used to manage the display subsystem and provide the fusion of 2D and 3D layers for multiple applications.

[0091] The media library supports playback and recording of a variety of commonly used audio and video formats, as well as static image files, etc. The media library can support a variety of audio and video encoding formats, such as: MPEG4, H.264, MP3, AAC, AMR, JPG, PNG, etc.

[0092] OpenGL ES is used to implement 3D graphics drawing, image rendering, compositing, and layer processing.

[0093] SGL is a graphics engine for 2D graphics.

[0094] Android Runtime includes core libraries and virtual machines. Android runtime is responsible for scheduling and management of the Android system. The core library consists of two parts: one is the function that the Java language needs to call, and the other is the Android core library. The application layer and the application framework layer run in the virtual machine. The virtual machine executes the Java files of the application layer and the application framework layer as binary files. The virtual machine is used to perform object life cycle management, stack management, thread management, security and exception management, and garbage collection.

[0095] The HAL layer is an encapsulation of the Linux kernel driver, providing an interface to the upper layer and shielding the implementation details of the low-level hardware.

[0096] The HAL layer may include Wi-Fi HAL, audio HAL, camera HAL, face recognition control module, etc.

[0097] Among them, the camera HAL is the core software framework of the Camera, which can include sensor nodes and image front end (IFE) nodes. The sensor node and IFE node are components (nodes) in the image data and control instruction transmission path (also called transmission pipeline) created by the camera HAL.

[0098] The face recognition control module, also known as the face client application (Face CA), or face recognition CA, is the core software framework / application of face recognition. The face recognition control module can communicate with the face trusted application (Face TA).

[0099] A trusted face application, also known as a face recognition TA, is an application for face recognition that runs in a TEE environment.

[0100] Among them, Face TA may include a frame management module (FrameManager), a TOF algorithm (algo) module and a face (Face) ID algorithm module. The frame management module can send the RAW Data corresponding to each frame image to the TOF algorithm module. The TOF algorithm module can convert the RAW Data into a grayscale image (IR image) and a depth image (depth image). The TOF algorithm module can send the grayscale image and the depth image to the face ID algorithm module. The face ID algorithm module can perform face recognition based on the grayscale image (i.e., determine whether the current user is the owner of the machine), and perform liveness detection (anti-counterfeiting detection) based on the depth image, thereby obtaining face recognition results.

[0101] The kernel layer is the layer between hardware and software. The kernel layer contains at least display driver, camera driver, audio driver, and sensor driver.

[0102] Among them, the camera driver is the driver layer of the Camera device, which is mainly responsible for the interaction with the hardware.

[0103] The hardware layer includes display, TOF camera, IFE module and secure memory (Secure Buffer), etc.

[0104] Among them, secure memory refers to memory with security protection function, which can be used to store raw data collected by TOF camera.

[0105] A TOF camera, also called a TOF sensor, may include a transmitter Tx and a receiver Rx, wherein Tx is used to transmit infrared light or laser pulses, and Rx is used to receive reflected light and form an image.

[0106] IFE module (IFE-Lite): can be called an image pre-processing module, which can be used to forward image data without processing the image data during the forwarding process.

[0107] The following describes the software modules and interactions between modules involved in the face recognition method provided in the embodiment of the present application.

[0108] like Figure 4As shown, the lock screen application in the application layer can interact with the face recognition SDK, and the face recognition SDK can interact with the face recognition service in the framework layer by calling a preset application programming interface (API) interface, and the face recognition service can interact with the face recognition CA in the HAL layer, and the face recognition CA can interact with the camera HAL in the HAL layer through the camera service in the framework layer, or the face recognition CA can directly interact with the camera HAL in the HAL layer. The camera HAL may include sensor nodes and IFE nodes. The sensor node can interact with the camera driver module in the kernel layer, and the camera driver module can be used to drive the TOF camera in the hardware layer to collect image data in a default working mode (e.g., human eye safety mode). The IFE module can store the image data collected by the TOF camera in a secure memory. The storage location of the image data collected by the TOF camera in the secure memory can be represented by a file descriptor (filedescriptor, FD). The IFE module can send the FD of the image data to the camera driver module, the camera driver module can send the FD to the IFE node of the camera HAL, the IFE node can send the FD to the camera service, the camera service can send the FD to the face recognition CA, the face recognition CA can send the FD to the face recognition TA, the face recognition TA can read the image data from the secure memory according to the FD, process the image data, and can send the processing result to the face recognition CA, the face recognition CA can continue to interact with the camera HAL through the camera service to switch the working mode of the TOF camera. The camera HAL can continue to interact with the camera driver module so that the camera driver module can drive the TOF camera to collect image data with the switched working mode (for example, face ID mode), and the FD corresponding to the image data can continue to be sent to the face recognition TA through the IFE module, the camera driver module, the IFE node, the camera service and the face control module, and the face recognition TA can read the image data again and process it, and feedback the processing result (face recognition success or face recognition failure) to the face recognition CA. The face recognition CA can feed back the processing results to the lock screen application through the face recognition service and face recognition SDK, so that the lock screen application can determine whether to unlock (if the face recognition is successful, unlock; if the face recognition fails, do not unlock, that is, unlocking fails). Figure 4 The solid arrows in the figure can be used to represent control flow, and the dashed arrows can be used to represent data flow.

[0109] In an embodiment of the present application, the operating modes of the TOF camera may include eye safety mode, face ID mode (first operating mode), skin safety mode (second operating mode) and Tx off mode, etc.

[0110] EyeSafe Mode refers to a mode in which the Tx of the TOF camera operates at a low current (a current less than a preset threshold, such as the current value I1). The eyesafe mode can be used to perform eye safety detection on the TOF camera, that is, to detect whether the TOF camera is damaged. When the TOF camera is damaged, the light intensity of the optical signal emitted from the TOF camera may exceed the eye safety range, thereby causing damage to the human eye.

[0111] Face ID Mode refers to a mode in which the Tx of the TOF camera operates at a normal current (i.e., a first operating current, which is within a preset threshold range, such as current value I2). The Face ID Mode is used for scenarios such as secure face unlocking and secure payment. In the embodiment of the present application, in the Face ID Mode, the electronic device can also perform skin safety detection to determine whether the intensity of the optical signal emitted by the Tx is within the skin safety range.

[0112] Skin safety mode (FaceSafe Mode) refers to a mode in which the Tx of the TOF camera operates at a normal current (i.e., a first operating current, which is within a preset threshold range, such as current value I2), but compared to the Face ID mode, the maximum exposure time of the transmitter of the TOF camera and the maximum exposure time of the light receiver are shorter to reduce the energy value of the light signal irradiated on the skin, thereby protecting the safety of the skin. In the embodiment of the present application, the skin safety mode can also be used for scenarios such as secure face unlocking and secure payment.

[0113] Tx OFF Mode refers to the mode in which the Tx of the TOF camera is not powered (and therefore does not emit light). The Tx OFF Mode is used when the Tx device of the TOF camera is detected to be damaged or malfunctioning. This is because if the Tx device is still powered on when it is damaged, it may have an adverse effect on the human eye. Therefore, when the Tx device of the TOF camera is detected to be damaged, the Tx OFF Mode is used to make the Tx device not powered, thereby avoiding damage to the human eye.

[0114] Figure 5 It is a structural diagram of Tx of a TOF camera provided in an embodiment of the present application.

[0115] like Figure 5As shown in a in , in one implementation, the Tx of the TOF camera may include a light transmitter, a light receiver and a light diffuser, wherein the light transmitter may be, for example, a laser diode LD, and the light receiver may be, for example, a photodiode (PD). LD and PD may be arranged in parallel below the display screen cover (e.g., a glass cover) of the electronic device, LD projects a light signal, and PD is used to sense the reflected light of the light signal to generate a corresponding electrical signal, such as a voltage signal or a current signal, and the intensity value of the electrical signal is related to the light intensity of the light signal. The light diffuser is arranged between the display screen cover and the LD and PD, and the light diffuser is located in the positive projection area of ​​the LD and PD, that is, it covers the top of the LD and PD. When the light signal emitted by the LD passes through the light diffuser, the light diffuser can homogenize and scatter the light signal, so that the light signal can pass through the display screen cover evenly and illuminate targets such as human faces in a larger range, and reflect back.

[0116] When the light signal projected by the LD passes through the light diffuser and the display cover, a portion of the light signal will be reflected back by the light diffuser and the display cover. When the PD senses the reflected light, it will generate an electrical signal related to the light intensity of the reflected light, such as a voltage signal and a current signal. In the embodiment of the present application, the voltage value of the voltage signal or the current value of the current signal generated by the PD is referred to as the PD value. It can be understood that the PD value is positively correlated with the light intensity of the reflected light sensed by the PD. Generally speaking, Figure 5 As shown in a, if the light diffuser is not damaged, the light intensity of the reflected light is relatively stable in the EyeSafe Mode, so the PD value is equal to a standard PD value or within a standard PD value range. When the light diffuser is damaged, the light intensity of the reflected light will change, causing the PD value to change.

[0117] Exemplary, after actual testing: Figure 5 As shown in b, when the light diffuser is partially intact and partially damaged in the orthogonal projection area of ​​LD, and the light diffuser is completely damaged in the orthogonal projection area of ​​PD, the PD value will increase. Figure 5 As shown in c, when the light diffuser is partially damaged in the orthogonal projection area of ​​LD and partially intact, and the light diffuser is not damaged in the orthogonal projection area of ​​PD, the PD value will increase. Figure 5 As shown in d, when the light diffuser is completely damaged in the positive projection area of ​​LD and is not damaged in the positive projection area of ​​PD, the change of PD value is uncertain, that is, PD value may increase, decrease or remain unchanged. Figure 5 As shown in Figure e, when the light diffuser is completely damaged, the PD value increases.

[0118] Based on the characteristic that the PD value changes when the light diffuser is damaged, the eye safety detection can be performed based on the PD value in the eye safety mode, that is, to detect whether the TOF camera is damaged. For example, if the PD value is greater than the standard PD value, the light diffuser is damaged, that is, the TOF camera is damaged. However, when the damage of the TOF camera is as follows Figure 5 As shown in d in the figure, the change of PD value is uncertain. When the PD value decreases or remains unchanged, it will lead to the failure to detect damage to the camera. In other words, there is a problem of false detection of human eye safety.

[0119] Figure 6 It is a structural diagram of Tx of another TOF camera provided in an embodiment of the present application.

[0120] like Figure 6 As shown in a in FIG, in order to solve the problem of false detection of human eye safety, the TOF camera also includes a baffle, which is arranged in the positive projection area of ​​the PD and is located between the PD and the light diffuser. Figure 6 In the structure shown in a, if the light diffuser is not damaged, the light intensity of the reflected light is relatively stable in the EyeSafe Mode, so the PD value is equal to a standard PD value or within a standard PD value range. When the light diffuser is damaged, the light intensity of the reflected light will change, causing the PD value to change.

[0121] Exemplary, after actual testing: Figure 6 As shown in b, when the light diffuser is partially intact and partially damaged in the orthogonal projection area of ​​LD, and the light diffuser is completely damaged in the orthogonal projection area of ​​PD, the PD value will decrease. Figure 6 As shown in c, when the light diffuser is partially damaged in the orthogonal projection area of ​​LD and partially intact, and the light diffuser is not damaged in the orthogonal projection area of ​​PD, the PD value will increase. Figure 5 As shown in d, when the light diffuser is completely damaged in the positive projection area of ​​LD and the light diffuser is not damaged in the positive projection area of ​​PD, the PD value will decrease. Figure 5 As shown in Figure e, when the light diffuser is completely damaged, the PD value decreases. Figure 6 In the Tx structure shown, no matter how the light diffuser is damaged, the change of the PD value is certain. Therefore, when performing eye safety detection based on the PD value in the eye safety mode, as long as the PD value is greater than or less than the standard PD value, it can be determined that the light diffuser is damaged, that is, the TOF camera is damaged, and there will be no false eye safety detection.

[0122] In addition, based on Figure 5The structure shown can also perform skin safety detection based on the PD value. For example, in the face ID mode, when a face or other part of the skin blocks the TOF camera, the light signal emitted by the LD will be irradiated on the skin and reflected, and the light receiver will receive the reflected light and generate a corresponding electrical signal. In this way, it can be determined whether there is a skin safety problem based on the size and change of the PD value. However, the method of skin safety detection based on the PD value is Figure 6 However, it cannot be used in the Tx structure shown. This is because after a baffle is set between the PD and the light diffuser, the external reflected light will be blocked by the baffle and cannot be irradiated onto the PD. Therefore, it is impossible to determine whether there is a skin safety issue based on the size and change of the PD value.

[0123] For example, Figure 7 As shown, the sensor node in the camera HAL can be used to select the working mode of the TOF camera.

[0124] The default initial working mode of the TOF camera can be an eye safety mode. In the eye safety mode, the electronic device can perform an eye safety test, and after obtaining the eye safety test result, the sensor node in the camera HAL can switch the working mode of the TOF camera. Specifically, if the eye safety test result is successful (i.e., there is no eye safety problem), the TOF camera can be switched to the face ID mode. If the eye safety test result fails (i.e., there is an eye safety problem), the TOF camera can be switched to the Tx off mode. In this way, eye safety can be guaranteed. In the face ID mode, the electronic device can perform a skin safety test, and after obtaining the skin safety test result, the sensor node in the camera HAL can switch the working mode of the TOF camera. Specifically, if the skin safety test result is successful (i.e., there is no skin safety problem), the TOF camera can be kept in the face ID mode. If the skin safety test result fails (i.e., there is a skin safety problem), the TOF camera can be switched to the skin safety mode. In this way, skin safety can be guaranteed.

[0125] For ease of understanding, the method provided in the embodiments of the present application is described in detail below with reference to the accompanying drawings.

[0126] Figure 8 It is a flow chart of a face recognition method provided in an embodiment of the present application.

[0127] like Figure 8 As shown, the face recognition method provided in the embodiment of the present application may include the following steps:

[0128] Step S101: The lock screen application calls the face recognition SDK to perform face recognition.

[0129] When the electronic device detects the user's unlocking operation (first operation), the lock screen application can call the face recognition SDK to perform face recognition. The user's unlocking operation includes the user picking up the electronic device, or pressing the power button, or operating on the screen (clicking, sliding, etc.), or unplugging the charging cable.

[0130] In addition, the lock screen application can register a callback with the face recognition SDK. The purpose of registering the callback is that when the face recognition SDK obtains the face recognition result, the face recognition SDK can return the face recognition result to the lock screen application.

[0131] Step S102: The face recognition SDK sends a face recognition request to the face recognition service.

[0132] The face recognition request may carry the face recognition type identifier, image resolution size, and data stream format. The face recognition type includes a 2D face recognition type (eg, corresponding to identifier 0) and a 3D face recognition type (eg, corresponding to identifier 1).

[0133] Exemplarily, the face recognition type identifier carried in the face recognition request can be 1 (i.e., 3D face recognition type), the image resolution size can be 1280x2898 pixels (pixel), and the data stream format can be the raw image format (RAW) 16.

[0134] In addition, the face recognition SDK can register a callback with the face recognition service. The purpose of registering the callback is that when the face recognition service obtains the face recognition result, the face recognition service can return the face recognition result to the face recognition SDK.

[0135] Step S103: The face recognition service sends a face recognition request to the face recognition CA.

[0136] For the face recognition request, please refer to the description of step S102, which will not be repeated here.

[0137] That is, the face recognition SDK can notify the face recognition CA to perform face recognition through the face recognition service. The face recognition service can send the face recognition request received from the face recognition SDK to the face recognition CA.

[0138] In addition, the face recognition service can register a callback with the face recognition CA. The purpose of registering the callback is that when the face recognition CA obtains the face recognition result, the face recognition CA can return the face comparison result to the face recognition service.

[0139] Step S104, in response to receiving the face recognition request, the face recognition CA matches the camera according to the face recognition request.

[0140] Specifically, the face recognition CA can obtain the face recognition type identifier, image resolution size, and data stream format from the face recognition request, and determine the matching camera by querying the camera capability from the camera service.

[0141] It should be understood that during the startup of an electronic device, the camera service can send a camera capability query request to the camera HAL, and the camera capability query request is used to request the camera capabilities supported by the electronic device. After receiving the camera capability query request, the camera HAL can send the capabilities of the cameras supported by the electronic device to the camera service, and the camera service can store the received capabilities of the cameras supported by the electronic device. Among them, the camera capabilities supported by the electronic device include the camera identification (identity, ID) of each camera, the maximum resolution size supported, the format of the data stream, and whether the camera supports the acquisition of depth information, etc.

[0142] For example, assuming that three cameras are installed on a mobile phone, the capability information of the three cameras may be as shown in Table 1:

[0143] Table 1

[0144] Camera ID Installation location Maximum supported resolution Data stream format Depth Information 1 Post 4096×3072 pixels YUV no 2 Pre 3264×2448 pixels YUV no 3 Pre 1280×2898 pixels RAW16 yes

[0145] Among them, the camera with camera ID 3 can be a TOF camera that supports collecting depth information. The cameras with camera IDs 1 and 2 can be ordinary cameras that do not support collecting depth information. Of course, more front or rear cameras can be installed on the mobile phone, for example, the mobile phone can be installed with 2 front cameras and 4 rear cameras.

[0146] The face recognition CA can send a camera capability query request to the camera service, and the camera service can send the capabilities of the cameras supported by the electronic device to the face recognition CA. The face recognition CA can determine the camera that matches the face recognition request based on the capabilities of the cameras supported by the electronic device. For example, it can be determined that the matching camera is the camera with ID 3 (i.e., the TOF camera).

[0147] It should be noted that Table 1 is only an example, and each camera may correspond to multiple data stream formats. For example, the camera with camera ID 1 may correspond not only to the YUV data stream format, but also to the RAW16 data stream format, which is not limited in this application.

[0148] S105, the face recognition CA sends a request for starting the camera to the camera service.

[0149] Exemplarily, the face recognition CA can send a request to start the camera to the camera service through the vendor's native development kit (VNDK) interface. The request to start the camera carries information such as the security identifier, camera ID, resolution size, and data stream format. The security identifier is used to indicate that the data is stored in a secure buffer. In other words, the security identifier can be used to apply for a secure memory, which is subsequently used to store data collected by the camera. For example, the security identifier can be 1 or 0, 1 means that the data is stored in a secure buffer, and 0 means that the data is stored in a non-secure buffer.

[0150] Exemplarily, the security identifier carried in the request for starting the camera may be 1 (ie, the data is stored in a secure buffer), the image resolution may be 1280x2898 pixels, the data stream format may be RAW16, and the camera ID may be 3.

[0151] In addition, the face recognition CA can register a callback with the camera service. The purpose of registering the callback is to notify the face recognition CA that the camera startup is complete after the camera service completes the camera startup.

[0152] Step S106, in response to receiving the request for starting the camera, the camera service sends a request for starting the camera to the camera HAL, and the request for starting the camera carries information such as a security identifier, a camera ID, a resolution size, and a data stream format.

[0153] When the camera HAL receives a request to start the camera, it can cache information such as the security identifier, camera ID, image resolution, and data stream format for a preset period of time.

[0154] In addition, the camera service can register a callback with the camera HAL, which is used by the camera HAL to notify the camera service of the result of creating the channel.

[0155] Step S107: The camera HAL creates a corresponding channel according to the camera ID, image resolution and data stream format.

[0156] The camera HAL can select available nodes according to the camera ID, resolution and data stream format, and then create corresponding paths according to the available nodes. For example, if the resolution is 1280x2898 pixel, the data stream format is RAW16, and the camera ID is 3, it can be determined to select the sensor node and the IFE node. This is because the sensor node and the IFE node can support the transmission of data with a resolution of 1280x2898 pixel and a data stream format of RAW 16 collected by the camera with camera ID 3.

[0157] Among them, the path corresponding to the sensor node can be: a path consisting of sensor node-camera driver-TOF camera-IFE module-secure memory. The path corresponding to the IFE node can be: a path consisting of IFE module (carrying FD)-camera driver-IFE node. The camera HAL can connect the output port of the sensor node and the input port of the IFE node at the HAL layer. Thus, the path corresponding to the sensor node and the path corresponding to the IFE node can form a closed-loop path. After the path is created, the hardware in the path is powered on (that is, the hardware circuit is powered on) and waits for data requests.

[0158] Step S108: The camera HAL sends the result of creating the channel to the camera service.

[0159] The result of creating the path may be success or failure. If the result of creating the path is failure, the camera HAL notifies the camera service that the path creation failed. If the result of creating the path is success, the camera HAL notifies the camera service that the path creation was successful, and step S109 and subsequent steps may continue to be executed.

[0160] Step S109, in response to receiving the notification of successful channel creation, the camera service sends a message of camera startup completion to the face recognition CA.

[0161] It can be understood that the completion of camera startup means that the preparation work before the camera takes pictures or videos (for example, camera parameter configuration, power-on and other preparation work) has been completed.

[0162] Step S110: In response to receiving a message indicating that the camera startup is complete, the face recognition CA sends a data request to the camera service.

[0163] The data request is used to request the camera's data stream.

[0164] Step S111, in response to receiving the data request sent by the face recognition CA, the camera service calls the camera HAL to obtain the data stream.

[0165] Step S112: The sensor node of the camera HAL selects the working mode of the camera.

[0166] Specifically, the sensor node may select the working mode of the camera according to the camera resolution and data stream format cached in S106. Exemplarily, the sensor node may select the working mode of the camera by looking up a table (eg, Table 2).

[0167] Table 2

[0168]

[0169]

[0170] Of course, the camera's operating modes may include more, which are not limited in this application.

[0171] According to Table 2, when the maximum image resolution is 1280x2898 pixels and the data stream format is Raw, the camera's working modes may include eye safety mode, face ID mode, skin safety mode, Tx off mode, etc.

[0172] In the embodiment of the present application, the sensor node may use the eye-safe mode as the initial working mode of the camera. Therefore, in step S112, the sensor node may select the eye-safe mode.

[0173] Step S113: The sensor node sends the configuration parameters of the eye safety mode to the camera driving module.

[0174] The sensor node may store configuration parameters corresponding to different operating modes of the TOF sensor. Exemplarily, the configuration parameters corresponding to the eye safety mode may be: current value of 700mA, depth exposure time of 10μs, and IR grayscale exposure time of 10μs. The operating parameters corresponding to the face ID mode may be: current value of 2800mA, depth exposure time of 1.8ms, and IR grayscale exposure time of 3ms. The configuration parameters of the skin safety mode may be: current value of 2800mA, depth exposure time of 5μs, and IR grayscale exposure time of 450μs. The operating parameters corresponding to the Tx off mode may be: current value of 0mA, depth exposure time of 0μs, and IR grayscale exposure time of 0μs. Among them, the depth exposure time and IR grayscale exposure time in each configuration parameter may refer to the maximum exposure time.

[0175] Among them: Depth exposure time refers to the time when the TOF camera is capturing depth information, when the laser or infrared light transmitter is turned on, that is, the time when Tx emits light signals. This time determines the time window for the sensor to measure depth. Generally, the longer the depth exposure time, the more reflected laser or infrared light signals Rx can receive, thereby improving the accuracy of depth measurement. IR grayscale exposure time refers to the time when the TOF camera's Rx is in the receiving state.

[0176] Step S114: The camera driving module writes the configuration parameters of the eye safety mode into the register of the TOF camera.

[0177] That is to say, the camera driver module can send the configuration parameters of the eye safety mode to the TOF camera.

[0178] Exemplarily, the camera driver module can write the configuration parameters of the eye safety mode into the register of Rx of the TOF camera through the inter-integrated circuit (I2C). Among them, the register can be set inside the TOF sensor or outside the TOF sensor. The address corresponding to the register of Rx can be 0×01. The register corresponding to Rx can include multiple registers, which are not limited in this application. In other words, the camera driver module can send the configuration parameters of the eye safety mode to Rx of the TOF camera through I2C. Among them, the configuration parameters of the eye safety mode include configuration parameters for Rx and Tx. For example, the configuration parameters for Tx can be the current value I1 and the depth exposure time. The configuration parameters for Rx can be the IR grayscale exposure time. Rx can write the configuration parameters corresponding to Tx into the register corresponding to Tx through the SPI bus. The address of the register corresponding to Tx can be 0×11. The register corresponding to Tx can include multiple registers, which are not limited in this application.

[0179] Step S115: the camera driving module sends a start (stream on) command / instruction to the TOF camera.

[0180] The start command is used to drive the TOF camera to collect data.

[0181] Step S116, in response to receiving the start command, the TOF camera operates based on the configuration parameters of the eye-safe mode.

[0182] In a specific implementation, the Tx in the TOF sensor can emit a light signal from the light transmitter of Tx based on the depth exposure time and current value I1 in the configuration parameters of the eye safety mode. Of course, this embodiment does not limit the specific value of the above-mentioned current value I1. As long as the current value can be guaranteed not to harm the human eye and support the TOF sensor to complete the eye safety detection, it can be used. In other words, the light signal emitted by the TOF sensor based on the current value I1 is harmless to the human eye. At the same time, the above-mentioned depth exposure time should not be too long, as long as it can support the TOF sensor to complete the eye safety detection.

[0183] The optical transmitter of Tx transmits the optical signal device, and the optical receiver of Tx can sense the reflected light of the optical signal and generate an electrical signal related to the light intensity of the reflected light. The TOF sensor can obtain the human eye safety detection result based on the numerical relationship between the PD value of the electrical signal generated by the optical receiver of Tx and the standard PD value.

[0184] Exemplarily, the eye safety detection result may include safe (normal) or unsafe (abnormal), wherein if the PD value in the eye safety detection result is equal to the standard PD value or within the standard PD value range, the eye safety detection result is safe; if the PD value is greater than or less than the standard PD value or outside the standard PD value range, the eye safety detection result is unsafe.

[0185] Next, the TOF sensor may also write the eye safety detection result into a register, wherein the register may be set in the TOF sensor. In addition, although the TOF sensor does not output image data to the outside during operation based on the configuration parameters of the eye safety mode, this part of the image data also involves user privacy data. Therefore, this embodiment may store these image data and the eye safety detection result into the register together to ensure data security.

[0186] Step S117: The camera driving module reads the eye safety detection result from the register of the TOF camera.

[0187] Exemplarily, the camera driving module can read the eye safety detection result from the register of the TOF camera through an inter-integrated circuit (I2C).

[0188] Step S118: The camera driver module sends the eye safety detection result to the camera HAL.

[0189] Step S119: The sensor node of the camera HAL determines the working mode of the TOF camera based on the eye safety detection result.

[0190] Exemplarily, if the eye safety detection result is safe, the sensor node determines that the working mode of the TOF camera is the face ID mode; if the eye safety detection result is unsafe, the sensor node determines that the working mode of the TOF camera is the Tx off mode.

[0191] It should be noted that the embodiments of the present application do not limit the execution order of steps S101-S119. In some embodiments, after steps S101-S107 are executed, steps S112-S118 can be executed directly, and after step S118, steps S108-S111 can be executed, and after S111, S119 can be executed. Of course, S101-S119 can also have other combination orders to ensure that the sensor node of the camera HAL can obtain the human eye safety detection results, so as to determine the working mode of the TOF camera. This application will not go into details here.

[0192] The following description is made by taking the working mode of the TOF camera determined by the sensor node as the face ID mode as an example. After S119, steps S120-S157 are also included:

[0193] Step S120: The sensor node sends configuration parameters of the face ID mode to the camera driving module.

[0194] Step S121, the camera driving module writes the configuration parameters of the face ID mode into the register of the TOF camera.

[0195] That is to say, the camera driver module can send the configuration parameters of the face ID mode to the TOF camera.

[0196] Exemplarily, the camera driver module can write the configuration parameters of the face ID mode into the TOF camera register via I2C. In other words, the camera driver module can send the configuration parameters of the face ID mode to the TOF camera via I2C.

[0197] Step S122: The TOF camera collects RAW Data 1 based on the face ID mode.

[0198] In a specific implementation, the Tx of the TOF camera can operate at a current value of I2 (i.e., the first operating current) based on the configuration parameters of the face ID mode, and emit a light signal based on the depth exposure time (i.e., the first depth exposure time) in the configuration parameters of the face ID mode. The Rx of the TOF camera can receive the reflected light of the light signal and form an image based on the IR grayscale exposure time (i.e., the first grayscale exposure time) in the configuration parameters of the face ID mode, thereby obtaining image data.

[0199] Exemplarily, the configuration parameters of the face ID mode include: current value 2800mA, depth exposure time 1.8ms, IR grayscale exposure time 3ms. Then, the Tx of the TOF camera can operate at a current value of 2800mA, a depth exposure time (i.e., the time for emitting light signals) of 1.8ms, and an IR grayscale exposure time (i.e., the time for receiving reflected light) of the Rx of the TOF camera of 3ms.

[0200] It should be noted here that in the face ID mode, the Tx and Rx in the TOF camera are exposed periodically. In each acquisition cycle, the Tx in the TOF camera emits a light signal once within the depth exposure time, and the Rx in the TOF camera receives the reflected light once within the IR grayscale exposure time and forms an image, thereby obtaining a frame of image data, that is, obtaining the RAW Data 1 corresponding to this frame of image data. In other words, the TOF camera can obtain RAW Data 1 corresponding to a frame of image data in each acquisition cycle, and can obtain multiple RAW Data1s corresponding to multiple frames of image data in multiple acquisition cycles, such as RAW Data 11 to RAW Data 1. n , n is an integer greater than 1.

[0201] In addition, RAW Data 1 may also include Metadata. Exemplarily, Metadata may include the current working mode of the TOF camera (e.g., face ID mode), configuration parameters of the current working mode (e.g., configuration parameters of the face ID mode), working status of the TOF camera device, etc.

[0202] Step S123, the TOF camera sends RAW Data 1 to the IFE module.

[0203] Exemplarily, the TOF camera can transmit RAW Data 1 to the IFE module via the MIPI interface.

[0204] Step S124, the IFE module sends RAW Data 1 to the secure buffer for storage.

[0205] The storage location of RAW Data 1 in the secure memory can be represented by FD 1.

[0206] Step S125: The IFE module sends FD 1 to the camera driver module.

[0207] Step S126: The camera driving module sends FD 1 to the IFE node.

[0208] Step S127, the IFE node sends FD 1 to the camera service through the interface of the camera HAL.

[0209] Step S128: The camera service sends FD 1 to the face recognition CA.

[0210] Step S129: the face recognition CA sends FD 1 to the face recognition TA.

[0211] Step S130 , the frame management module of the face recognition TA reads RAW Data 1 from the secure memory according to FD 1 .

[0212] It should be noted that since the TOF camera can obtain RAW Data 1 corresponding to a frame of image data in each acquisition cycle, steps S123-S130 can be performed once after the TOF camera obtains RAW Data 1 corresponding to a frame of image data, so that the face recognition TA can read the RAW Data 1 corresponding to this frame of image data from the secure memory. In addition, the RAW Data 1 corresponding to image data of different frames can be stored in different locations of the secure memory to avoid the RAW Data 1 stored later from overwriting the RAW Data 1 stored earlier.

[0213] Step S131: The frame management module sends RAW Data 1 to the TOF algorithm module.

[0214] Step S132: The TOF algorithm module obtains the grayscale image and depth image corresponding to RAW Data 1.

[0215] Step S133: The TOF algorithm module sends the grayscale image and depth image corresponding to RAW Data 1 to the face ID algorithm module.

[0216] Step S134, the face ID algorithm module obtains the face recognition result according to the grayscale image and depth image corresponding to RAW Data 1.

[0217] In the specific implementation, the face ID algorithm module can perform face recognition based on the grayscale image and perform anti-counterfeiting detection based on the depth map to obtain face recognition results. It should be noted that the face ID algorithm module can store the grayscale image and depth map converted from the face information previously entered by the user. If the currently collected face information (RAW Data 1 collected by the TOF camera based on the face ID mode) matches the grayscale image corresponding to the previously entered face information (that is, RAW Data 1 collected by the electronic device when the user performs the face entry operation), it can be considered to be the same user (that is, the same user performs the face entry operation and the unlocking operation).

[0218] If the currently collected face information includes depth information, the current user can be considered to be authentic (not disguised by photos, videos, etc.), and the face of the current user can be considered safe, that is, the face recognition result is successful. If the currently collected face information (RAW Data 1 collected by the TOF camera based on the face ID mode) does not match the previously recorded face information (that is, the grayscale image corresponding to RAW Data 1 collected by the electronic device when the user performs the face entry operation), or if the currently collected face information does not include depth information, the face of the current user is considered to be unsafe, that is, the face recognition result is failed.

[0219] If face recognition fails, the face recognition TA can continue to obtain RAW Data 1 collected by the TOF camera, and continue face recognition based on RAW Data 1, and can continue to execute steps S138-S157 until the unlocking is successful or the unlocking times out. If face recognition is successful, there is no need to execute steps S138-S157.

[0220] Step S135, the face ID algorithm module sends the face recognition result to the frame management module.

[0221] Step S136: The frame management module sends the face recognition result to the face recognition CA.

[0222] Next, the electronic device can execute steps S180-S183. If the face recognition result is safe / normal (or the face recognition is successful), the electronic device can be successfully unlocked. If the face recognition result is unsafe / abnormal (or the face recognition fails), the electronic device fails to unlock.

[0223] Step S137 , the TOF algorithm module performs skin safety detection based on the brightness of the grayscale image corresponding to RAW Data 1 to obtain a skin safety detection result.

[0224] In a specific implementation, the brightness of a grayscale image can be represented by a gray level, which is a unitless value that represents the brightness of a pixel in the grayscale image. In other words, in a grayscale image, the brightness of each pixel can be represented by a numerical value (i.e., gray level).

[0225] Exemplarily, an 8-bit grayscale image has a total of 256 gray levels, that is, the brightness range of any pixel in the grayscale image can be 0 to 255, where the smaller the value, the lower the brightness of the pixel, and the larger the value, the higher the brightness of the pixel. 0 represents black, 255 represents white, and the intermediate values ​​represent different degrees of brightness.

[0226] Exemplarily, the brightness of the grayscale image is equal to the average brightness of all pixels in the grayscale image, that is, the brightness of the grayscale image is equal to the average brightness of all pixels in the grayscale image.

[0227] Exemplarily, the skin safety detection result may include safe (normal) or unsafe (abnormal). In order to obtain the skin safety detection result, a threshold corresponding to the grayscale level, i.e., the brightness threshold, may be preset, and the brightness of the grayscale image may be compared with the brightness threshold. n If the brightness of at least one grayscale image in the grayscale images corresponding to n consecutive frames of RAW Data 1 (i.e., the first brightness) is greater than the brightness threshold, the skin safety detection result is unsafe (i.e., the first detection result); if the brightness of at least one grayscale image in the grayscale images corresponding to n consecutive frames of RAW Data 1 is less than or equal to the brightness threshold (i.e., the brightness is not greater than the brightness threshold), the skin safety detection result is safe (i.e., the second detection result).

[0228] Exemplarily, taking an 8-bit grayscale image as an example, the brightness threshold can be 240, and n can be equal to 15. That is to say, if the brightness of 15 consecutive frames of grayscale images is greater than the brightness threshold, the skin safety detection result is unsafe; if the brightness of at least one frame of the 15 consecutive frames of grayscale images is less than or equal to 240, the skin safety detection result is safe.

[0229] It should be noted here that when the brightness of n consecutive grayscale frames is greater than the brightness threshold, it means that the TOF camera may be continuously facing the user, resulting in enough light signals being reflected back. In this case, based on the face ID mode configuration parameters, if the light signal continues to shine on the skin, the energy of the light signal will continue to be absorbed by the skin, causing the total energy of the light signal absorbed by the skin over a period of time to be greater than the safe energy upper limit. Therefore, if the brightness of n consecutive grayscale frames is greater than the brightness threshold, the skin safety detection result is unsafe.

[0230] When the brightness of the grayscale image is greater than the brightness threshold and lasts for a certain number of frames, it means that the TOF camera may be continuously facing the user, so there is a skin safety risk. In this case, the electronic device reduces the exposure time of the TOF camera, which is helpful to improve the accuracy of entering the skin safety mode.

[0231] It can be seen that the face recognition method provided by the embodiment of the present application can perform skin safety detection based on the brightness of the grayscale image corresponding to the image frame, and no longer depends on the PD value of the light receiver in Tx. Figure 6 When using a TOF camera with a baffle as shown, the electronic device can also achieve skin safety detection, improving the adaptability of the face recognition method.

[0232] Step S138: The TOF algorithm module sends the skin safety detection result to the frame management module.

[0233] Step S139: The frame management module sends the skin safety detection result to the face recognition CA.

[0234] Step S140: the face recognition CA sends the skin safety detection result to the camera HAL.

[0235] Exemplarily, the face recognition CA can directly pass the skin safety detection result to the camera HAL through the HIDL interface. Alternatively, the face recognition CA can send the skin safety detection result to the camera service through the VNDK interface, so that the camera service sends the skin safety detection result to the camera HAL.

[0236] Step S141: The sensor node of the camera HAL determines the working mode of the TOF camera based on the skin safety detection result.

[0237] Exemplarily, if the skin safety detection result is safe, the sensor node determines that the working mode of the TOF camera is face ID mode, that is, the working mode of the TOF camera is not changed; if the skin safety detection result is unsafe, the sensor node determines that the working mode of the TOF camera is skin safety mode.

[0238] Step S142: The sensor node sends the configuration parameters of the skin safety mode to the camera driving module.

[0239] Step S143: The camera driving module writes the configuration parameters of the skin safety mode into the register of the TOF camera.

[0240] That is to say, the camera driver module can send the configuration parameters of the skin safety mode to the TOF camera.

[0241] Exemplarily, the camera driver module can write the configuration parameters of the skin safety mode into the TOF camera register via I2C. In other words, the camera driver module can send the configuration parameters of the skin safety mode to the TOF camera via I2C.

[0242] Step S144: The TOF camera collects RAW Data 2 based on the configuration parameters of the skin safety mode.

[0243] In a specific implementation, the Tx of the TOF camera can operate at a current value of I2 (i.e., the second operating current) based on the configuration parameters of the skin safety mode, and emit a light signal based on the depth exposure time (i.e., the second depth exposure time) in the configuration parameters of the skin safety mode. The Rx of the TOF camera can receive the reflected light of the light signal and form an image based on the IR grayscale exposure time (i.e., the second grayscale exposure time) in the configuration parameters of the skin safety mode, thereby obtaining image data.

[0244] Exemplarily, the configuration parameters of the skin safety mode include: current value 2800mA, depth exposure time 5μs, IR grayscale exposure time 450μs. Then, the Tx of the TOF camera can operate at a current value of 2800mA, a depth exposure time (i.e., the time for emitting light signals) of 5μs, and an IR grayscale exposure time (i.e., the time for receiving reflected light) of the Rx of the TOF camera of 450μs.

[0245] It should be noted that the configuration parameters of the skin safety mode are compared with those of the face ID mode. The current value of the skin safety mode can be equal to the current value of the face ID mode, but the depth exposure time and IR grayscale exposure time of the skin safety mode are shorter than those of the face ID mode. In other words, in the skin safety mode, the intensity of the signal light emitted by the TOF camera can remain unchanged compared to the face ID mode, but the duration of the light signal emission is shortened. In this way, the time the light signal is irradiated on the skin will be shortened, and the energy of the light signal absorbed by the skin will be reduced, thereby avoiding skin safety issues.

[0246] Exemplarily, different optical signal wavelengths and different irradiation durations may correspond to different safety energy upper limits, taking an optical signal of 940 nanometers (nm) as an example, as shown in Table 3:

[0247] Table 3

[0248]

[0249] Among them, when the wavelength range is 700-1050nm, C4=10 0.002(λ-700) .W·m -2 Surface watt per square meter, J·m -2 For example, in the wavelength range of 700-1400nm, if the light signal is irradiated for 10 -7 -10 - 3 s, then the upper limit of the safe energy value of the light signal absorbed by the skin is 1.1×10 4 C4t 0.25 J.m -2 .

[0250] Exemplarily, taking the 940nm light signal as an example, the depth exposure time and IR grayscale exposure time in the configuration parameters of the skin safety mode can be determined based on Table 3, as long as it is ensured that within the exposure time shown in Table 3, the energy of the light signal absorbed by the skin is not higher than the corresponding safety energy upper limit.

[0251] That is to say, the current value of 2800mA, the depth exposure time of 5μs, and the IR grayscale exposure time of 450μs in the configuration parameters of the skin safety mode are only used as examples. In specific implementation, different configuration parameters can also be determined based on Table 3, and the embodiments of the present application are not limited to this.

[0252] It can be seen from this that the depth exposure time and IR grayscale exposure time in the configuration parameters of the skin safety mode can be determined based on the safety energy upper limit value in Table 3. In this way, it can be ensured that within the irradiation time of Tx, the energy of the light signal absorbed by the skin is not higher than the corresponding safety energy upper limit value, thereby ensuring that the light signal will not cause damage to human skin and protect skin safety.

[0253] Step S145, the TOF camera sends RAW Data 2 to the IFE module.

[0254] Exemplarily, the TOF camera can transmit RAW Data 2 to the IFE module via the MIPI interface.

[0255] Step S146: The IFE module sends RAW Data 2 to the secure memory for storage.

[0256] The storage location of RAW Data 2 in the secure memory can be represented by FD 2.

[0257] Step S147: The IFE module sends FD 2 to the camera driver module.

[0258] Step S148: The camera driving module sends FD 2 to the IFE node.

[0259] Step S149, the IFE node sends FD 2 to the camera service through the interface of the camera HAL.

[0260] Step S150: The camera service sends FD 2 to the face recognition CA.

[0261] Step S151, the face recognition CA sends FD 2 to the face recognition TA.

[0262] Step S152, the frame management module of the face recognition TA reads RAW Data 2 from the secure memory according to FD 2.

[0263] Step S153: The TOF algorithm module obtains the grayscale image and depth image corresponding to RAW Data 2.

[0264] Step S154: The TOF algorithm module sends the grayscale image and depth image corresponding to RAW Data 2 to the face ID algorithm module.

[0265] Step S155, the face ID algorithm module obtains the face recognition result according to the grayscale image and depth image corresponding to RAW Data 2.

[0266] If face recognition fails, the face recognition TA can continue to obtain RAW Data 2 collected by the TOF camera, and continue face recognition based on RAW Data 2, and can continue to execute steps S158-S173 until the unlocking is successful or the unlocking times out. If face recognition is successful, there is no need to execute steps S158-S173.

[0267] Step S156, the face ID algorithm module sends the face recognition result to the frame management module.

[0268] Step S157: The frame management module sends the face recognition result to the face recognition CA.

[0269] Next, the electronic device can execute steps S180-S183. If the face recognition result is safe / normal (or the face recognition is successful), the electronic device can be successfully unlocked. If the face recognition result is unsafe / abnormal (or the face recognition fails), the electronic device fails to unlock.

[0270] It can be seen that in the face recognition method provided by the embodiment of the present application, the current value of the skin safety mode can be equal to the current value of the face ID mode, and the depth exposure time and IR grayscale exposure time of the skin safety mode are shorter than the depth exposure time and IR grayscale exposure time of the face ID mode. In other words, in the skin safety mode, the intensity of the signal light emitted by the TOF camera can remain unchanged compared to the face ID mode, but the duration of transmitting and receiving the light signal is shortened. In this way, the energy of the light signal absorbed by the skin is reduced to protect the skin safety, face recognition can be performed normally, and the power consumption during face recognition can be reduced, achieving three goals at one stroke.

[0271] Fig. 9 This is another flow chart of a face recognition method provided in an embodiment of the present application.

[0272] like Fig. 9 As shown, in one implementation, the method further includes, after step S155:

[0273] Step S158: The TOF algorithm module obtains a comparison result between the brightness of the grayscale image corresponding to RAW Data 2 and the brightness threshold.

[0274] The comparison result may include that the brightness of the grayscale image corresponding to RAW Data 2 (i.e., the second brightness) is greater than or equal to the brightness threshold (i.e., the second comparison result), or that the brightness of the grayscale image corresponding to RAW Data 2 is less than the brightness threshold (i.e., the first comparison result). If the brightness of the grayscale image corresponding to RAW Data 2 is greater than or equal to the brightness threshold, the TOF camera may still be continuously facing the user, resulting in sufficient light signals being reflected back. If the brightness of the grayscale image corresponding to RAW Data2 is less than the brightness threshold, it means that the TOF camera may no longer be facing the user, or is far away from the user, resulting in a decrease in the reflected light signal.

[0275] Step S159: The TOF algorithm module sends the comparison result to the frame management module.

[0276] Step S160: The frame management module sends the comparison result to the face recognition CA.

[0277] Step S161, the face recognition CA sends the comparison result to the camera HAL.

[0278] Step S162: The sensor node of the camera HAL determines the operating mode of the TOF camera based on the comparison result.

[0279] Exemplarily, if the comparison result is that the brightness of the grayscale image corresponding to RAW Data 2 is greater than or equal to the brightness threshold, the sensor node determines that the working mode of the TOF camera is the skin safety mode, that is, the working mode of the TOF camera is not changed; if the comparison result is that the brightness of the grayscale image corresponding to RAW Data 2 is less than the brightness threshold, the sensor node determines that the working mode of the TOF camera is the face ID mode.

[0280] When the brightness of the grayscale image corresponding to RAW Data 2 is less than the brightness threshold, it indicates that the skin safety risk is eliminated. Therefore, the electronic device can control the TOF camera to return to collecting image frames based on the face ID mode, thereby obtaining image frames containing more information, which is conducive to improving the speed of face recognition.

[0281] Step S163: The sensor node sends the configuration parameters of the face ID mode to the camera driver module.

[0282] Step S164: The camera driver module writes the configuration parameters of the face ID mode into the register of the TOF camera.

[0283] Step S165: The TOF camera collects RAW Data 3 based on the configuration parameters of the face ID mode.

[0284] Step S166: The TOF camera sends RAW Data 3 to the IFE module.

[0285] Exemplarily, the TOF camera can transmit RAW Data 3 to the IFE module via the MIPI interface.

[0286] Step S167, the IFE module sends the RAW Data to the secure memory for storage.

[0287] The storage location of RAW Data in secure memory can be represented by FD 3.

[0288] Step S168: The IFE module sends FD 3 to the camera driver module.

[0289] Step S169: The camera driving module sends FD 3 to the IFE node.

[0290] Step S170, the IFE node sends FD 3 to the camera service through the interface of the camera HAL.

[0291] Step S171: The camera service sends FD 3 to the face recognition CA.

[0292] Step S172, the face recognition CA sends FD 3 to the face recognition TA.

[0293] Step S173 , the frame management module of the face recognition TA reads RAW Data 3 from the secure memory according to FD 3 .

[0294] Step S174: The frame management module sends RAW Data 3 to the TOF algorithm module.

[0295] Step S175 , the TOF algorithm module obtains the grayscale image and depth image corresponding to RAW Data 3 .

[0296] Step S176: The TOF algorithm module sends the grayscale image and depth image corresponding to RAW Data 3 to the face ID algorithm module.

[0297] Step S177, the face ID algorithm module obtains the face recognition result according to the grayscale image and depth image corresponding to RAW Data 3.

[0298] Step S178, the face ID algorithm module sends the face recognition result to the frame management module.

[0299] Step S179: the frame management module sends the face recognition result to the face recognition CA.

[0300] Step S180: the face recognition CA sends the face recognition result to the face recognition service.

[0301] The face recognition CA can send the face recognition results to the face recognition service based on the callback previously registered with the face recognition service.

[0302] Step S181, the face recognition service sends the face recognition result to the face recognition SDK.

[0303] The face recognition service sends the face recognition results to the face recognition SDK based on the callback previously registered in the face recognition SDK.

[0304] Step S182: The face recognition SDK sends the face recognition result to the lock screen application.

[0305] The face recognition SDK sends the face recognition result to the lock screen application based on the callback previously registered in the lock screen application.

[0306] Step S183, the lock screen application decides whether to unlock based on the face recognition result.

[0307] If the face recognition result is successful, the lock screen application can be successfully unlocked, so that the electronic device can display the main screen or the interface of the application (system application or third-party application). If the face recognition result is failed, the lock screen application is not unlocked, that is, the face unlocking fails. After the face unlocking fails, the lock screen application can disable the face recognition function within a period of time (for example, 5 minutes) when the face recognition fails.

[0308] For example, if the user sets up face unlocking, Fig.10 As shown in (a) of FIG. 1 , when the user picks up the mobile phone for face recognition, in response to the user picking up the mobile phone, as shown in FIG. 1 , Fig.10 As shown in (b) in FIG. 7 , the mobile phone may display a lock screen interface 701 . During the face recognition process, the mobile phone may display an unlock icon 702 and a prompt text “face recognition in progress” 703 on the lock screen interface 701 .

[0309] If face recognition is successful, Fig.11 As shown in (a) of FIG. 1 , the mobile phone may display an interface 704, which may include an unlock icon 705 (the unlock icon 705 may be an open lock, which may vividly prompt the user that the face unlock is successful) and a prompt text "Swipe up to enter" 706. In response to the user's swiping operation, the mobile phone may display the home screen or the interface of an application (system application or third-party application). Or, as Fig.11As shown in (b), if the face recognition is successful, the mobile phone can be unlocked directly without additional user operation, that is, the main screen 707 can be displayed immediately, or the application interface can be directly displayed.

[0310] If face recognition fails, Fig.12 As shown in (a) of FIG. 7 , the mobile phone may display an interface 708, which may include an unlock icon 709 (the unlock icon 709 may be a closed lock, which may vividly prompt the user that the face unlocking is not successful) and a prompt text "Unrecognized, double-click the screen to try again" 710. In response to the user's double-click operation, the mobile phone may perform face recognition again (i.e., collect the user's face information again for comparison and anti-counterfeiting judgment). Or, in response to the user's upward sliding operation in the interface 708, such as Fig.12 As shown in (b) of FIG. 1 , the mobile phone may display an interface 711. After entering the interface 711, the mobile phone may perform face recognition again. The interface 711 may include a face recognition icon 712 and a prompt text "face recognition in progress" 713. If the recognition is still unsuccessful, such as Fig.12 As shown in (c), the mobile phone can display an interface 714, which may include a prompt text 715 "Unsuccessful recognition, click here to try again". The user can click the corresponding position to re-trigger face recognition, or can also enter a password through the soft keyboard 716 to unlock, avoiding the problem of poor user experience caused by unsuccessful recognition.

[0311] It should be noted that the above embodiment uses the lock screen application as an example to illustrate the face recognition method provided in the embodiment of the present application. The face recognition method provided in the embodiment of the present application can also be used for face recognition when the user pays or transfers money in the application, for example, when the user uses the payment application / financial application / chat application / shopping application to make a payment or transfer. The face recognition method provided in the embodiment of the present application can also be used for face recognition when the user registers or logs in to an account in the application, for example, when the user Face recognition is performed when registering an account or logging in to an account in the application. Of course, the face recognition method provided in the embodiment of the present application can also be used in other scenarios, which are not specifically limited here.

[0312] The following is an example in which the working mode of the TOF camera determined by the sensor node is the Tx off mode.

[0313] Fig.13 This is another flow chart of a face recognition method provided in an embodiment of the present application.

[0314] like Fig.13 As shown, after step S118, steps S184-S195 are also included:

[0315] Step S184: The sensor node of the camera HAL determines that the working mode of the TOF camera is the Tx off mode based on the eye safety detection result.

[0316] Step S185: The sensor node sends the configuration parameters of the Tx off mode to the camera driver module.

[0317] Step S186: The camera driving module writes the configuration parameters of the Tx off mode into the register of the TOF camera to drive the TOF camera to collect data.

[0318] That is, the camera driver module can send the configuration parameters of the Tx off mode to the TOF camera.

[0319] Exemplarily, the camera driver module can write the configuration parameters of the Tx off mode into the TOF camera register via I2C, that is, send the configuration parameters of the Tx off mode to the TOF camera via I2C.

[0320] Step S187, the TOF camera collects RAW Data 4 based on the Tx off mode.

[0321] Among them, RAW Data 4 can be the image data obtained by the Rx of the TOF camera receiving the ambient light and imaging when the Tx of the TOF camera is not powered on and does not emit light. It is usually a "black picture" without a clear face image.

[0322] In addition, Metadata may also be included in RAW Data 4. Exemplarily, Metadata may include the current working mode of the TOF camera (e.g., face Tx off mode), configuration parameters of the current working mode (e.g., configuration parameters of the Tx off mode), working status of the TOF camera device, etc.

[0323] Step S188, the TOF camera transmits RAW Data 4 to the IFE module.

[0324] Exemplarily, the TOF camera can transmit the RAW Data 4 collected by the TOF camera to the IFE module via MIPI.

[0325] Step S189, the IFE module sends the RAW Data 4 to the secure memory for storage.

[0326] The storage location of RAW Data 4 collected by the TOF camera based on the Tx off mode in the secure memory can be represented by FD 4.

[0327] Step S190: The IFE module sends FD 4 to the camera driving module.

[0328] Step S191: The camera driving module sends FD 4 to the IFE node.

[0329] Step S192, the IFE node sends FD 4 to the camera service through the interface of the camera HAL.

[0330] In step S193, the camera service sends FD 4 to the face recognition CA.

[0331] Step S194, the face recognition CA sends FD 4 to the face recognition TA.

[0332] Step S195, the face recognition TA reads RAW Data 4 from the secure memory according to FD 4.

[0333] Step S196, face recognition TA obtains face recognition results based on RAW Data 4.

[0334] Specifically, the face recognition TA can determine that the TOF camera is currently working in Tx off mode based on the Metadata data in RAW Data 4. When the TOF camera is working in Tx off mode, the face recognition result is failed. This is because the TOF camera cannot emit light in Tx off mode, so the TOF camera cannot collect a clear face image, so the face recognition result is failed.

[0335] Step S197, face recognition TA sends the face recognition result to face recognition CA.

[0336] That is, the face recognition TA can notify the face recognition CA that the face recognition result is a failure.

[0337] Step S198, the face recognition CA sends the face recognition result to the face recognition service.

[0338] The face recognition CA sends the face recognition result to the face recognition service based on the callback previously registered by the face recognition service. That is, the face recognition CA notifies the face recognition service that the face recognition result is a failure.

[0339] Step S199: The face recognition service sends the face recognition result to the face recognition SDK.

[0340] The face recognition service sends the face recognition result to the face recognition SDK based on the callback previously registered by the face recognition SDK. That is, the face recognition service can notify the face recognition SDK that the face recognition result is a failure.

[0341] Step S200: The face recognition SDK sends the face recognition result to the lock screen application.

[0342] The face recognition SDK sends the face recognition result to the lock screen application based on the callback previously registered by the lock screen application. That is, the face recognition SDK can notify the lock screen application that the face recognition result is a failure.

[0343] Step S201: The lock screen application determines not to unlock the device based on the face recognition result.

[0344] Since the face recognition result is a failure, the lock screen app is not unlocked.

[0345] For example, Fig.12 As shown in (a) of FIG. 7 , the mobile phone may display an interface 708, which may include an unlock icon 709 (the unlock icon 709 may be a closed lock, which may vividly prompt the user that the face unlocking is not successful) and a prompt text "Unrecognized, double-click the screen to try again" 710. In response to the user's double-click operation, the mobile phone may perform face recognition again (i.e., collect the user's face information again for comparison and anti-counterfeiting judgment). Or, in response to the user's upward sliding operation in the interface 708, such as Fig.12 As shown in (b) of FIG. 1 , the mobile phone may display an interface 711. After entering the interface 711, the mobile phone may perform face recognition again. The interface 711 may include a face recognition icon 712 and a prompt text "face recognition in progress" 713. If the recognition is still unsuccessful, such as Fig.12 As shown in (c), the mobile phone can display an interface 714, which may include a prompt text 715 "Unsuccessful recognition, click here to try again". The user can click the corresponding position to re-trigger face recognition, or can also enter a password through the soft keyboard 716 to unlock, avoiding the problem of poor user experience caused by unsuccessful recognition.

[0346] Fig.14 This is another flow chart of a face recognition method provided in an embodiment of the present application.

[0347] like Fig.14 As shown, in one embodiment, the method may include the following steps S801-S803.

[0348] Step S801: The electronic device controls the TOF camera to capture a first image frame based on a first exposure duration.

[0349] Step S802: When the brightness of the first image frame is greater than a preset brightness threshold, the electronic device determines a second exposure time, and the second exposure time is less than the first exposure time.

[0350] Step S803: The electronic device controls the TOF camera to capture a second image frame based on a second exposure duration.

[0351] The method provided in the embodiment of the present application reduces the exposure time of the TOF camera when the brightness of the first image frame captured by TOF is greater than a preset brightness threshold, that is, enters the skin safety mode from the face ID mode, thereby reducing the time that the light signal is irradiated on the human skin, reducing the energy of the light signal absorbed by the skin, avoiding damage to the human skin by the light signal, and protecting the skin safety.

[0352] In one implementation, when the brightness of the first image frame is greater than a preset brightness threshold, the electronic device determines the second exposure time, including: the electronic device determines the first brightness of the first image frame, the first brightness being the average brightness of all pixels in the grayscale image corresponding to the first image frame; when the first brightness is greater than the preset brightness threshold, the electronic device determines the second exposure time. In this way, the electronic device can perform skin safety detection based on the brightness of the grayscale image corresponding to the first image frame, and no longer relies on the PD value of the light receiver in Tx. Therefore, when the electronic device is equipped with Figure 6 When using a TOF camera with a baffle as shown, the electronic device can also achieve skin safety detection, improving the adaptability of the face recognition method.

[0353] In one implementation, when the first brightness is greater than a preset brightness threshold, the electronic device determines the second exposure duration, including: when the first brightness of n first image frames continuously captured by the TOF camera is greater than the brightness threshold, the electronic device determines the second exposure duration, where n is a positive integer greater than 1. When the first brightness of the first image frame is greater than the brightness threshold and lasts for a certain number of frames, it means that the TOF camera may be continuously facing the user, so there is a skin safety risk. In this case, the electronic device further reduces the exposure duration of the TOF camera, which is conducive to improving the accuracy of entering the skin safety mode.

[0354] In one implementation, after the electronic device controls the TOF camera to capture the second image frame based on the second exposure duration, the method further includes: when the brightness of the second image frame is less than the brightness threshold, the electronic device controls the TOF camera to capture the image frame based on the first exposure duration. When the brightness of the second image frame is less than the brightness threshold, it indicates that the skin safety risk is eliminated. Therefore, the electronic device can control the TOF camera to resume capturing the image frame based on the first exposure duration, thereby obtaining an image frame containing more information, which is conducive to improving the speed of face recognition.

[0355] In one implementation, the TOF camera includes a transmitter and a receiver; the first exposure time includes a first depth exposure time of the transmitter and a first grayscale exposure time of the receiver, and the second exposure time includes a second depth exposure time of the transmitter and a second grayscale exposure time of the receiver; the second depth exposure time is less than the first depth exposure time, and the second grayscale exposure time is less than the first grayscale exposure time. In this way, in the skin safety mode, the intensity of the signal light emitted by the TOF camera can remain unchanged compared to the face ID mode, but the duration of emitting and receiving the light signal is shortened. In this way, the energy of the light signal absorbed by the skin is reduced to protect the skin safety, and face recognition can be performed normally, and the power consumption during face recognition can be reduced, killing three birds with one stone.

[0356] Steps S801-S803 and their implementation methods may be specifically implemented with reference to the above examples, and will not be described in detail here.

[0357] Some embodiments of the present application provide an electronic device, which may include: a touch screen, a memory, and one or more processors. The touch screen, the memory, and the processor are coupled. The memory is used to store computer program code, and the computer program code includes computer instructions. When the processor executes the computer instructions, the electronic device can perform the various functions or steps performed by the electronic device in the above method embodiment. The structure of the electronic device can refer to Figure 1 The structure of the electronic device 100 is shown.

[0358] The present application also provides a chip system, for example, a system on a chip (SoC), such as Fig.15 As shown, the chip system includes at least one processor 1101 and at least one interface circuit 1102. The processor 1101 and the interface circuit 1102 can be interconnected via lines. For example, the interface circuit 1102 can be used to receive signals from other devices (such as a memory of an electronic device). For another example, the interface circuit 1102 can be used to send signals to other devices (such as a processor 1101 or a touch screen of an electronic device). Exemplarily, the interface circuit 1102 can read instructions stored in the memory and send the instructions to the processor 1101. When the instructions are executed by the processor 1101, the electronic device can execute the various steps in the above embodiments. Of course, the chip system may also include other discrete devices, which are not specifically limited in the embodiments of the present application.

[0359] An embodiment of the present application also provides a TOF camera, which can be used to implement the eye safety mode, face ID mode, skin safety mode and Tx off mode in the above embodiments. An electronic device equipped with the TOF camera can perform the various functions or steps performed by the electronic device in the above method embodiments.

[0360] An embodiment of the present application also provides a computer-readable storage medium, which includes computer instructions. When the computer instructions are executed on the above-mentioned electronic device, the electronic device executes each function or step executed by the electronic device in the above-mentioned method embodiment.

[0361] The embodiment of the present application also provides a computer program product. When the computer program product is run on an electronic device, the electronic device executes each function or step executed by the electronic device in the above method embodiment.

[0362] Through the description of the above implementation methods, technical personnel in the relevant field can clearly understand that for the convenience and simplicity of description, only the division of the above-mentioned functional modules is used as an example. In actual applications, the above-mentioned functions can be assigned to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above.

[0363] In the several embodiments provided in the present application, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of the modules or units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another device, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0364] The units described as separate components may or may not be physically separated, and the components shown as units may be one physical unit or multiple physical units, that is, they may be located in one place or distributed in multiple different places. Some or all of the units may be selected according to actual needs to achieve the purpose of the present embodiment.

[0365] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0366] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a readable storage medium. Based on this understanding, the technical solution of the embodiment of the present application is essentially or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product, which is stored in a storage medium, including several instructions to enable a device (which can be a single-chip microcomputer, chip, etc.) or a processor (processor) to perform all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read only memory (ROM), random access memory (RAM), disk or optical disk and other media that can store program code.

[0367] The above contents are only specific implementation methods of the present application, but the protection scope of the present application is not limited thereto. Any changes or substitutions within the technical scope disclosed in the present application shall be included in the protection scope of the present application. Therefore, the protection scope of the present application shall be subject to the protection scope of the claims.

Claims

1. A face recognition method, applied to an electronic device, wherein the electronic device includes a time-of-flight TOF camera, characterized in that: The method comprises: Controlling the TOF camera to capture a first image frame based on a first exposure duration; When the brightness of the first image frame is greater than a preset brightness threshold, determining a second exposure time, wherein the second exposure time is less than the first exposure time; Control the TOF camera to capture a second image frame based on the second exposure duration.

2. The face recognition method according to claim 1, characterized in that: When the brightness of the first image frame is greater than a preset brightness threshold, determining a second exposure time includes: Determine a first brightness of the first image frame, where the first brightness is an average brightness of all pixels in a grayscale image corresponding to the first image frame; When the first brightness is greater than the brightness threshold, the second exposure duration is determined.

3. The face recognition method according to claim 2, characterized in that: When the first brightness is greater than the brightness threshold, determining the second exposure time includes: When the first brightness of n frames of the first image frames continuously captured by the TOF camera is greater than the brightness threshold, the second exposure duration is determined, where n is a positive integer greater than 1.

4. The face recognition method according to claim 1, characterized in that: After controlling the TOF camera to collect the second image frame based on the second exposure duration, the method further includes: When the brightness of the second image frame is less than the brightness threshold, the TOF camera is controlled to capture image frames based on the first exposure duration.

5. The face recognition method according to claim 1, characterized in that: The TOF camera includes a transmitter and a receiver; the first exposure time includes a first depth exposure time of the transmitter and a first grayscale exposure time of the receiver, and the second exposure time includes a second depth exposure time of the transmitter and a second grayscale exposure time of the receiver; the second depth exposure time is shorter than the first depth exposure time, and the second grayscale exposure time is shorter than the first grayscale exposure time.

6. The face recognition method according to claim 2, characterized in that: The electronic device includes a camera hardware abstraction layer, and the controlling the TOF camera to collect a first image frame based on a first exposure duration includes: The camera hardware abstraction layer configures first configuration parameters corresponding to a first operating mode for the TOF camera, where the first configuration parameters include the first exposure time and a first operating current of the TOF camera; The TOF camera captures the first image frame based on the first configuration parameters.

7. The face recognition method according to claim 6, characterized in that: The determining of the second exposure time when the brightness of the first image frame is greater than a preset brightness threshold comprises: When the brightness of the first image frame is greater than the brightness threshold, the camera hardware abstraction layer configures second configuration parameters corresponding to the second operating mode for the TOF camera, where the second configuration parameters include the second exposure time and the first operating current; The controlling the TOF camera to capture a second image frame based on the second exposure duration includes: The TOF camera captures the second image frame based on the second configuration parameters.

8. The face recognition method according to claim 7, characterized in that: The electronic device includes a trusted face application, and after the TOF camera acquires the first image frame based on the first configuration parameter, the electronic device further includes: The trusted face application obtains a grayscale image corresponding to the first image frame; The trusted face application determines the first brightness based on a grayscale image corresponding to the first image frame; The trusted face application obtains a detection result based on the first brightness and the brightness threshold, wherein, when the first brightness of the n frames of the first image frames continuously collected by the TOF camera is greater than the brightness threshold, the trusted face application obtains a first detection result; when the first brightness of the n frames of the first image frames continuously collected by the TOF camera is not greater than the brightness threshold, the trusted face application obtains a second detection result.

9. The face recognition method according to claim 8, characterized in that: The electronic device includes a face recognition control module, and after the face trusted application obtains a detection result based on the first brightness and the brightness threshold, it also includes: The trusted face application sends the detection result to the face recognition control module; The face recognition control module sends the detection result to the camera hardware abstraction layer.

10. The face recognition method according to claim 9, characterized in that: When the brightness of the first image frame is greater than the brightness threshold, the camera hardware abstraction layer configures a second configuration parameter corresponding to the second working mode for the TOF camera, including: When the camera hardware abstraction layer receives the first detection result, the camera hardware abstraction layer configures the second configuration parameters corresponding to the second working mode for the TOF camera.

11. The face recognition method according to claim 9, characterized in that: After the TOF camera acquires the second image frame based on the second configuration parameter, the method further includes: The trusted face application obtains a grayscale image corresponding to the second image frame; The trusted face application determines a second brightness of the second image frame based on the grayscale image corresponding to the second image frame, where the second brightness is an average brightness of all pixels in the grayscale image corresponding to the second image frame; The trusted face application obtains a comparison result based on the second brightness and the brightness threshold, wherein when the second brightness is less than the brightness threshold, the trusted face application obtains a first comparison result, and when the second brightness is greater than or equal to the brightness threshold, the trusted face application obtains a second comparison result.

12. The face recognition method according to claim 11, characterized in that: After the face trusted application obtains a comparison result based on the second brightness and the brightness threshold, the method further includes: The trusted face application sends the comparison result to the face recognition control module; The face recognition control module sends the comparison result to the camera hardware abstraction layer; When the camera hardware abstraction layer receives the first comparison result, the camera hardware abstraction layer configures the first configuration parameter corresponding to the first working mode for the TOF camera.

13. The face recognition method according to any one of claims 1 to 12, characterized in that: The first image frame and the second image frame are both used for face recognition.

14. An electronic device, characterized in that: include: A memory, a TOF camera and one or more processors; the memory, the TOF camera and the processor are coupled; wherein the memory stores computer program code, the computer program code includes computer instructions, and when the computer instructions are executed by the processor, the electronic device executes the face recognition method as described in any one of claims 1 to 13.

Images