Face recognition method and electronic equipment

Abstract

The embodiment of the invention provides a face recognition method and electronic equipment, the electronic equipment comprises a camera, the method comprises the following steps: receiving a first operation of a user, the first operation being used for triggering face recognition; in response to the first operation, electrifying the camera; after the camera is powered on, a configuration command is loaded, and the configuration command is used for indicating the camera to load camera parameters; the time difference between the end time of electrifying the camera and the start time of loading the configuration command is smaller than a preset time threshold value; camera parameters are loaded according to the configuration command; acquiring images based on the camera parameters; and performing face recognition based on the image. According to the invention, the time for the camera to prepare to output the image can be shortened, so that the time consumption of the face recognition process is reduced, and the face recognition efficiency is improved.

Description

Technical Field

[0001] This application relates to the field of electronic technology, specifically to a face recognition method and an electronic device. Background Technology

[0002] With the continuous development of smart terminal devices, facial recognition technology has been widely used. Typically, after a user initiates facial unlocking in front of an electronic device (such as a mobile phone or tablet), the device's camera captures an image of the user's face. The device then extracts facial features from the image and matches these features with pre-stored facial features to authenticate the user. Based on the authentication result, the electronic device is unlocked.

[0003] However, in related technologies, the response time from initiating face unlock to the camera capturing an image is relatively long. This results in a lengthy face unlock process, affecting the face unlock performance of electronic devices and leading to a poor user experience. Summary of the Invention

[0004] This application provides a face recognition method and electronic device that can shorten the time for the camera to prepare the image, thereby reducing the time consumed in the face unlock process and improving the efficiency of face unlock.

[0005] In a first aspect, this application provides a face recognition method applied to an electronic device, the electronic device including a camera. The method may include: receiving a first operation from a user, the first operation being used to trigger face recognition; powering on the camera in response to the first operation; after powering on the camera, loading a configuration command, the configuration command (i.e., the loading instruction hereinafter) being used to instruct the camera to load camera parameters; the time difference between the end time of powering on the camera and the start time of loading the configuration command being less than a preset time threshold; loading camera parameters according to the configuration command; acquiring an image based on the camera parameters; and performing face recognition based on the image.

[0006] The first operation can be used to trigger facial recognition. For example, the first operation can be a facial unlock operation, a facial payment operation, a facial verification login operation, etc. Taking facial unlock as an example, the facial unlock operation can be a user's triggering operation (such as pressing) of the power button of the electronic device when the screen is off, or a user's triggering operation on the display screen of the electronic device (such as clicking, swiping, etc.), or a user's operation of picking up the electronic device, or an operation of unplugging the charging cable. The specific type of facial unlock operation is not limited in this embodiment.

[0007] Camera parameters can be used to create a data stream through which images can be acquired for face recognition. For example, camera parameters can be preset or set when the electronic device first performs face recognition, i.e., preset streaming parameters as described below. Optionally, camera parameters may include an identifier for the face recognition type, image resolution, frame rate, data stream format, etc.

[0008] The preset time threshold can be set according to requirements. For example, the preset time threshold is 73ms. In this embodiment, after receiving the first operation to trigger face recognition, the camera can be powered on in advance, and the configuration command can be loaded immediately after powering on the camera. In this way, the time difference between the end time of powering on the camera and the start time of loading the configuration command is less than the preset time threshold, which can shorten the time for the camera to prepare for image output, thereby reducing the time consumed by the entire face recognition process and improving the efficiency of face recognition.

[0009] In one possible implementation of the first aspect, the first operation is pressing the power button while the screen is off.

[0010] In this embodiment, the first operation can be a face unlock operation, such as pressing the power button while the screen is off. That is, when the user presses the power button while the electronic device is in a screen-off state, the camera is powered on, and configuration commands are loaded immediately after powering on. This shortens the time it takes for the camera to prepare for image output, thereby reducing the time required for face unlock and improving its efficiency.

[0011] In one possible implementation of the first aspect, the electronic device includes a first application located at the application layer and a camera hardware abstraction layer located at the hardware abstraction layer. The application layer includes an AIDL interface. The power-on of the camera in response to the first operation may include: the first application sending a first instruction to the camera hardware abstraction layer through the AIDL interface, the first instruction being used to instruct the camera to power on; and the camera hardware abstraction layer controlling the camera to power on in response to the first instruction.

[0012] The first application is an application that can trigger facial recognition, such as the lock screen application described below. For example, when the first application is a lock screen application, the first instruction can be the facial unlock request described below.

[0013] In this embodiment, by pre-defining an AIDL interface, applications at the application layer (such as the first application) can directly interact with the camera hardware abstraction layer (HAL) through the AIDL interface, thus shortening the interaction process from the upper-layer application to the HAL. In response to the first operation, the first application can send a first command to the HAL through the pre-defined AIDL interface. In response to the first command, the HAL controls the camera to power on, enabling early power-on of the camera and thereby shortening the camera's frame output time.

[0014] In one possible implementation of the first aspect, the above-mentioned control of the camera power-on may include: the camera hardware abstraction layer controlling the camera power-on by calling the PreAcquireDevice() function and the AcquireDevice() function.

[0015] In one possible implementation of the first aspect, the aforementioned electronic device further includes a face recognition control module located at the hardware abstraction layer and a camera service located at the application framework layer. The aforementioned loading configuration command may include: a first application sending a second instruction to the camera service through the face recognition control module, the second instruction being used to instruct face recognition; the camera service responding to the second instruction sending a third instruction to the camera hardware abstraction layer, the third instruction being used to instruct the camera to power on; and the camera hardware abstraction layer responding to the third instruction, loading the configuration command when it is determined that the camera is powered on.

[0016] The second instruction can be a face recognition request as described below. The third instruction can be a camera opening request as described below.

[0017] In this embodiment, since the camera can be powered on in advance in response to the first operation, when the first application performs face recognition, a second instruction can be sent to the camera service through the face recognition module. After receiving the second instruction, the camera service sends a third instruction to the camera hardware abstraction layer. After receiving the third instruction, the camera hardware abstraction layer can load the configuration command. In this way, the configuration command is loaded immediately after the camera is powered on, which can shorten the time for the camera to prepare for image output, thereby reducing the time consumption of face recognition and improving the efficiency of face recognition.

[0018] In one possible implementation of the first aspect, the method further includes: the camera hardware abstraction layer responding to the first instruction by setting the value of the first state parameter to a first value; wherein, loading the configuration command when it is determined that the camera is in a powered-on state includes: the camera hardware abstraction layer loading the configuration command when it is determined that the value of the first state parameter is the first value.

[0019] The first status parameter indicates the current status of the camera. The camera's status can be set by adjusting the value of the first status parameter. A value of 1 indicates that the camera has completed power-on. For example, a value of 1 indicates that the camera has not been acquired or is being acquired. A value of 2 indicates that the camera has been acquired and is waiting for a data stream request. A value of 3 indicates that the camera is undergoing initialization settings. A value of 4 indicates that the camera initialization settings have been completed and is waiting for a data stream request. The first value can be, for example, 3.

[0020] In this embodiment, when the camera hardware abstraction layer receives a first instruction sent by a first application, it can set the value of the first state parameter to a first value. This allows the camera hardware abstraction layer to obtain the value of the first state parameter when loading a configuration command, ensuring that the configuration command is loaded only when the first state parameter is the first value. This ensures that the configuration command is loaded immediately after powering on the camera, preventing camera malfunctions and improving user experience. In one possible implementation of the first aspect, the method further includes: responding to the first instruction, after the camera is powered on, the camera hardware abstraction layer performs initialization configuration on the camera; wherein, loading the configuration command when the camera hardware abstraction layer determines that the first state parameter is the first value includes: loading the configuration command when the camera hardware abstraction layer determines that the first state parameter is the first value and the initialization configuration is complete.

[0021] In this embodiment, to ensure the accuracy of face recognition, the electronic device needs to acquire clearer facial image data containing more facial details. Therefore, when performing face recognition, the electronic device typically acquires the user's facial image data according to preset parameters. Since the parameters of the image data depend on the camera's capabilities, and the capabilities of each camera in the electronic device are fixed, the flow parameters of the camera used for face recognition on the electronic device are usually fixed. Therefore, in response to the first instruction, after the camera is powered on, the camera hardware abstraction layer initializes the camera configuration, thus enabling early power-on and parameter configuration of the camera. Furthermore, when the camera hardware abstraction layer receives the third instruction from the camera service, it directly loads the configuration command. This allows for early power-on and parameter configuration of the camera, shortening the waiting time for image output, thereby reducing the face recognition process time and improving the efficiency of face recognition.

[0022] In one possible implementation of the first aspect, the method further includes: after the camera hardware abstraction layer has completed loading the configuration command, setting the value of the first state parameter to an initial value.

[0023] The first state parameter has an initial value, indicating that the camera is in its initial state, that is, the state before the camera is powered on.

[0024] In this embodiment, after the camera hardware abstraction layer completes the loading of the configuration command, it restores the camera's state to its initial state. This ensures that when the camera is called again, it can be powered on and its state set in advance, thereby avoiding the camera's normal operation being affected by the failure to restore its state.

[0025] In one possible implementation of the first aspect, the above-mentioned loading configuration command includes: the camera hardware abstraction layer calling LoadSensorInitCfgCmd() to load the configuration command.

[0026] In one possible implementation of the first aspect, the aforementioned first application is a lock screen application, a payment application, a shopping application, a chat application, or a banking application.

[0027] The face recognition method provided in this application can be applied to a variety of different scenarios. For example, taking a payment application as an example, when the payment application receives a user's face payment input, it interacts with the camera HAL (Hardware Abstraction Layer) by calling the camera interface to control the camera to power on and configure its parameters in advance. This reduces the waiting time for camera startup, improves the speed of face recognition, and thus improves the efficiency of face payment.

[0028] For example, in a banking application, when the application receives a user's facial recognition login input, it interacts with the camera's Hardware Abstraction Layer (HAL) via the camera interface. The HAL then controls the camera to power on and configure its parameters in advance. This reduces camera startup time, increases facial recognition speed, and ultimately improves facial login efficiency.

[0029] Secondly, this application provides an apparatus included in an electronic device, which has the function of implementing the behaviors of the electronic device in the first aspect and possible implementations thereof. The function can be implemented by hardware or by hardware executing corresponding software. The hardware or software includes one or more modules or units corresponding to the above-described functions. For example, a receiving module or unit, a processing module or unit, etc.

[0030] Thirdly, this application provides an electronic device, which includes a processor, a memory, and an interface; the processor, memory, and interface cooperate with each other to enable the electronic device to execute any one of the methods in the first aspect of the technical solution.

[0031] Fourthly, this application provides a chip including a processor. The processor is used to read and execute a computer program stored in a memory to perform the methods in the first aspect and any possible implementation thereof.

[0032] Optionally, the chip may also include memory, which is connected to the processor via circuitry or wires.

[0033] Alternatively, the chip may also include a communication interface.

[0034] Fifthly, this application provides a computer-readable storage medium storing a computer program, which, when executed by a processor, causes the processor to perform any one of the methods in the first aspect of the technical solution.

[0035] Sixthly, this application provides a computer program product, which includes computer program code that, when executed on an electronic device, causes the electronic device to perform any one of the methods in the first aspect of the technical solution. Attached Figure Description

[0036] Figure 1 This is a schematic diagram illustrating the interaction between software modules during a face unlock process, provided in an embodiment of this application.

[0037] Figure 2 This is a schematic diagram of the structure of the electronic device provided in the embodiments of this application;

[0038] Figure 3 This is a software structure block diagram of the electronic device provided in the embodiments of this application;

[0039] Figure 4 This is another example of the interaction between software modules during the face unlock process provided in this application embodiment;

[0040] Figure 5 This is a schematic diagram of a face recognition process provided in an embodiment of this application;

[0041] Figure 6 This is a schematic diagram of the interactive flow of a camera configuration image output process provided in an embodiment of this application;

[0042] Figure 7 This is a schematic diagram of an example of the interaction process of updating the camera status using the camera HAL, provided in an embodiment of this application.

[0043] Figure 8 This is a schematic diagram of an interactive process for controlling a camera to load preset distribution parameters, provided in an embodiment of this application.

[0044] Figure 9This is a schematic diagram comparing the face unlock time before and after optimization, provided in an embodiment of this application.

[0045] Figure 10 This is a schematic diagram illustrating the process of powering on and configuring parameters of a camera controlled by a HAL according to an embodiment of this application.

[0046] Figure 11 This is a flowchart illustrating an example of the original configuration drawing process provided in this application embodiment;

[0047] Figure 12 This is a flowchart illustrating an example of a face recognition process provided in an embodiment of this application;

[0048] Figure 13 This is a schematic diagram illustrating the state changes of a camera during the camera configuration output process provided in an embodiment of this application. Detailed Implementation

[0049] The technical solutions of the embodiments of this application will be described below with reference to the accompanying drawings. In the description of the embodiments of this application, unless otherwise stated, " / " means "or," for example, A / B can mean A or B; "and / or" in this text is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, A and B existing simultaneously, and B existing alone. Furthermore, in the description of the embodiments of this application, "multiple" refers to two or more than two.

[0050] Hereinafter, the terms "first," "second," and "third" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. Thus, a feature defined as "first," "second," or "third" may explicitly or implicitly include one or more of that feature.

[0051] References to "one embodiment" or "some embodiments" as described in this application specification mean that one or more embodiments of this application include a specific feature, structure, or characteristic described in connection with that embodiment. Therefore, the phrases "in one embodiment," "in some embodiments," "in other embodiments," "in still other embodiments," etc., appearing in different parts of this application specification do not necessarily refer to the same embodiment, but rather mean "one or more, but not all, embodiments," unless otherwise specifically emphasized. The terms "comprising," "including," "having," and variations thereof mean "including but not limited to," unless otherwise specifically emphasized.

[0052] To better understand the embodiments of this application, the terms or concepts that may be involved in the embodiments are explained below.

[0053] A rich execution environment (REE), also known as a general execution environment, a common execution environment, or an untrusted execution environment, refers to the runtime environment of an operating system (such as Android, iOS, and Linux) on an electronic device. REEs offer greater openness and scalability.

[0054] A trusted execution environment (TEE), also known as a security side or security zone, is an area that requires authorization to access.

[0055] The REE+TEE architecture refers to an architecture where TEE and REE work together to provide services to applications. In other words, the TEE and REE coexist in the operating environment of the electronic device. Through hardware support, the TEE can achieve an isolated operating mechanism from the REE. The TEE has its own operating space, and only authorized security software can run within it. This ensures that assets running in the TEE (such as data and software) are not subject to attacks or threats from other software within the REE, thereby improving the confidentiality of the security software's assets. Compared to the REE, the TEE has a higher level of security due to its isolation and access control mechanisms, providing better protection for data and resource security.

[0056] A trusted application (TA) is an application that runs within a TEE (Trusted Application Environment) and provides security services to client applications running before the TEE, such as password input, transaction signature generation, and facial recognition.

[0057] A client application (CA, or simply client application) is an application that runs within a REE. Client applications can communicate with trusted applications within a TEE via an application programming interface (API) to request the trusted application to perform appropriate security operations.

[0058] With the continuous development of smart terminal devices, facial recognition technology has been widely used. Typically, after a user initiates facial unlocking on an electronic device, the device's camera captures an image of the user's face. The device then extracts facial features from the image and matches these features with pre-stored facial features to authenticate the user. Based on the authentication result, the electronic device is unlocked.

[0059] However, the inventors discovered through analysis that the image acquisition process in the face unlocking process takes a long time, which affects the face unlocking performance of electronic devices.

[0060] To facilitate understanding, the following will be combined with... Figure 1 The technical problems encountered during image acquisition are explained.

[0061] Figure 1 This is a schematic diagram illustrating the interaction between software modules during a face unlock process, as provided in an embodiment of this application. Figure 1 As shown, when a user initiates face unlock on an electronic device, the lock screen application at the application layer sends a face recognition request (also known as a face unlock request) to the face recognition service at the application framework layer via the face recognition software development kit (SDK). Upon receiving the face recognition request, the face recognition service forwards it to the face recognition control module at the hardware abstraction layer. Upon receiving the face recognition request, the face recognition control module sends a camera request to the camera service at the application framework layer via the vendor native development kit (VNDK) interface. Upon receiving the camera request, the camera service interacts with the camera hardware abstraction layer (camera HAL) to send a configuration image output request. After receiving the configuration image output request, the camera hardware abstraction layer interacts with the camera to begin configuring the camera's image output.

[0062] In addition, configuring the camera's image output involves four main stages: the open stage, the configure stage, the request stage, and the result stage. First, in the open stage, the camera hardware abstraction layer (HAL) acquires access to the camera and controls it to power on. Next, in the configure stage, the HAL sends configure parameters to the camera and instructs the camera to load them. Then, in the request stage, the HAL initiates a data stream request (i.e., an image acquisition request) to instruct the camera to begin acquiring images. Finally, in the result stage, after completing image acquisition, the camera returns the image acquisition result to the HAL.

[0063] As described above, on the one hand, the process from initiating face unlock to sending the image configuration request to the camera hardware abstraction layer requires interaction between multiple modules, making the request sending process to the camera hardware abstraction layer lengthy. On the other hand, after the image configuration request is sent to the camera hardware abstraction layer, the camera needs to be started first, and then the camera stream needs to be configured before image acquisition can begin. In other words, after receiving the image configuration request, the camera hardware abstraction layer needs to wait for a certain period of time before executing the image acquisition command, further increasing the waiting time for image output. Therefore, the process from initiating face unlock to the camera acquiring images is time-consuming, resulting in a long overall face unlock process, impacting face unlock performance and leading to a poor user experience.

[0064] In view of this, this application provides a face recognition method. After initiating face unlock, the lock screen application can interact with the camera HAL to notify the camera HAL to control the camera to power on and configure its parameters, thereby enabling the camera to power on and configure its parameters in advance. Simultaneously, the lock screen application can interact with the face recognition control module of the hardware abstraction layer to notify the face recognition control module to execute the face recognition process. During face recognition, the face recognition control module can send a camera request to the camera HAL. Upon receiving the camera request, the camera HAL can control the camera to load parameters and acquire image data, and then perform face recognition based on the image data. This allows for early power-on and parameter configuration of the camera, shortening the waiting time for image output, thereby reducing the face unlock process time and improving the efficiency of face unlock.

[0065] Before introducing the face recognition method provided in the embodiments of this application, the electronic devices to which the face recognition method provided in this application is applicable will be described first.

[0066] The face recognition method provided in this application can be applied to electronic devices that can install applications (APPs), such as mobile phones, tablets, wearable devices, in-vehicle devices, augmented reality (AR) / virtual reality (VR) devices, laptops, ultra-mobile personal computers (UMPCs), netbooks, and personal digital assistants (PDAs). This application does not impose any restrictions on the specific type of electronic device.

[0067] For example, Figure 2This is a schematic diagram of the structure of an electronic device 100 provided in an embodiment of this application. The electronic device 100 may include a processor 110, an external memory interface 120, an internal memory 121, a universal serial bus (USB) interface 130, a charging management module 140, a power management module 141, a battery 142, an antenna 1, an antenna 2, a mobile communication module 150, a wireless communication module 1A, an audio module 1B, a speaker 1BA, a receiver 1BB, a microphone 1BC, a headphone jack 1BD, a sensor module 180, buttons 190, a motor 191, an indicator 192, a camera 193, a display screen 194, and a subscriber identification module (SIM) card interface 195, etc. The sensor module 180 may include a pressure sensor 180A, a gyroscope sensor 180B, a barometric pressure sensor 180C, a magnetic sensor 180D, an accelerometer sensor 180E, a distance sensor 180F, a proximity sensor 180G, a fingerprint sensor 180H, a temperature sensor 180J, a touch sensor 180K, an ambient light sensor 180L, a bone conduction sensor 180M, etc.

[0068] It is understood that the structures illustrated in the embodiments of this application do not constitute a specific limitation on the electronic device 100. In other embodiments of this application, the electronic device 100 may include more or fewer components than illustrated, or combine some components, or split some components, or have different component arrangements. The illustrated components may be implemented in hardware, software, or a combination of software and hardware.

[0069] Processor 110 may include one or more processing units, such as: application processor (AP), modem processor, graphics processing unit (GPU), image signal processor (ISP), controller, memory, video codec, digital signal processor (DSP), baseband processor, and / or neural network processing unit (NPU), etc. Different processing units may be independent devices or integrated into one or more processors.

[0070] The controller can be the nerve center and command center of the electronic device 100. The controller can generate operation control signals according to the instruction opcode and timing signals to complete the control of fetching and executing instructions.

[0071] The processor 110 may also include a memory for storing instructions and data. In some embodiments, the memory in the processor 110 is a cache memory. This memory can store instructions or data that the processor 110 has just used or that are used repeatedly. If the processor 110 needs to use the instruction or data again, it can retrieve it directly from the memory. This avoids repeated accesses, reduces the waiting time of the processor 110, and thus improves the efficiency of the system.

[0072] In some embodiments, the processor 110 may include one or more interfaces. Interfaces may include an inter-integrated circuit (I2C) interface, an inter-integrated circuit sound (I2S) interface, a pulse code modulation (PCM) interface, a universal asynchronous receiver / transmitter (UART) interface, a mobile industry processor interface (MIPI), a general-purpose input / output (GPIO) interface, a subscriber identity module (SIM) interface, and / or a universal serial bus (USB) interface, etc.

[0073] Electronic device 100 implements display functions through a GPU, a display screen 194, and an application processor. The GPU is a microprocessor for image processing, connected to the display screen 194 and the application processor. The GPU is used to perform mathematical and geometric calculations and for graphics rendering. Processor 110 may include one or more GPUs, which execute program instructions to generate or modify display information.

[0074] Electronic device 100 can perform shooting functions through ISP, camera 193, video codec, GPU, display 194 and application processor.

[0075] The ISP (Image Signal Processor) is used to process data fed back from the camera 193. For example, when taking a picture, the shutter is opened, and light is transmitted through the lens to the camera's photosensitive element. The light signal is converted into an electrical signal, and the camera's photosensitive element transmits the electrical signal to the ISP for processing, transforming it into an image visible to the naked eye. The ISP can also perform algorithmic optimization of image noise, brightness, and skin tone. The ISP can also optimize parameters such as exposure and color temperature of the shooting scene. In some embodiments, the ISP can be set in the camera 193.

[0076] Camera 193 is used to capture still images or videos. An object passes through the lens, generating an optical image that is projected onto a photosensitive element. This photosensitive element can be a charge-coupled device (CCD) or a complementary metal-oxide-semiconductor (CMOS) phototransistor. The photosensitive element converts the light signal into an electrical signal, which is then passed to an ISP (Image Signal Processor) for conversion into a digital image signal. The ISP outputs the digital image signal to a DSP (Digital Signal Processor) for further processing. The DSP converts the digital image signal into standard RGB, YUV, or other image formats.

[0077] In some embodiments, the electronic device 100 may include one or N cameras 193, where N is a positive integer greater than 1. For example, the electronic device may include two front-facing cameras and four rear-facing cameras. The front-facing cameras may include a TOF (time-of-flight) camera. The TOF camera may be a 2D TOF camera or a 3D TOF camera. The TOF camera includes a transmitter (TX) and a receiver (RX). The TX can be used to emit light signals (infrared light or laser pulses), and the RX can be used to receive images. The TX may be, for example, an infrared light emitter. The RX may be, for example, a complementary metal-oxide-semiconductor (CMOS) or charge-coupled device (CCD) image sensor.

[0078] Digital signal processors (DSPs) are used to process digital signals. Besides digital image signals, they can also process other digital signals. For example, when electronic device 100 selects a frequency, the DSP can perform Fourier transforms on the frequency energy.

[0079] Video codecs are used to compress or decompress digital video. Electronic device 100 may support one or more video codecs. Thus, electronic device 100 can play or record videos in various encoding formats, such as Moving Picture Experts Group (MPEG) 1, MPEG2, MPEG3, MPEG4, etc.

[0080] An NPU (Neural Processing Unit) is a computational processor for neural networks (NNs). By borrowing the structure of biological neural networks, such as the transmission patterns between neurons in the human brain, it can rapidly process input information and continuously learn on its own. NPUs enable intelligent cognitive applications in electronic devices, such as image recognition, facial recognition, speech recognition, and text understanding.

[0081] Internal memory 121 can be used to store computer executable program code, which includes instructions. Processor 110 executes various functional applications and data processing of electronic device 100 by running the instructions stored in internal memory 121. Internal memory 121 may include a program storage area and a data storage area. The program storage area may store the operating system, at least one application program required for a function (such as sound playback, image playback, etc.), etc. The data storage area may store data created during the use of electronic device 100 (such as audio data, phonebook, etc.). Furthermore, internal memory 121 may include high-speed random access memory and may also include non-volatile memory, such as at least one disk storage device, flash memory device, universal flash storage (UFS), etc.

[0082] The above describes the electronic devices to which the face recognition method provided in this application is applicable. Next, the software system of the aforementioned electronic device will be described. The software system of the electronic device 100 can adopt a layered architecture, event-driven architecture, microkernel architecture, microservice architecture, or cloud architecture. This application embodiment uses a layered Android system as an example to exemplify the software structure of the electronic device 100.

[0083] Figure 3 This is a software structure block diagram of an electronic device 100 according to an embodiment of this application. The layered architecture divides the software into several layers, such as... Figure 3 As shown, a layered software system is divided into several layers, each with a clear role and function. Layers communicate with each other through software interfaces. In some embodiments, the Android system is divided into four layers, from top to bottom: the application layer (referred to as the application layer), the application framework layer (referred to as the application framework layer), the Android runtime and system libraries, the hardware abstraction layer (HAL), the kernel layer, and the hardware layer. The application layer can include a series of application packages.

[0084] like Figure 3As shown, the application package may include applications such as camera, gallery, calendar, call, map, navigation, WLAN, Bluetooth, music, video, and SMS.

[0085] like Figure 3 As shown, the application package may include applications such as camera, gallery, calendar, call, map, navigation, WLAN, Bluetooth, music, video, SMS, lock screen application, and settings application. Of course, the application layer may also include other application packages, such as payment applications, shopping applications, banking applications, chat applications, or financial management applications; this application is not limited to these.

[0086] The settings application includes a face registration function, which is used for face unlock. The lock screen application has the function of unlocking in response to the user's unlocking operation (e.g., pressing the power button). The lock screen application can perform unlocking processes such as face unlock, fingerprint unlock, and password unlock. This application embodiment mainly uses face unlock as an example for description.

[0087] The application framework layer provides application programming interfaces and a programming framework for applications in the application layer. The application framework layer includes some predefined functions.

[0088] like Figure 3 As shown, the application framework layer can include a Camera Service and a Face Recognition Service. The Camera Service is responsible for communicating with the camera hardware on the device. For example, the Camera Service can interact with the camera through the Hardware Abstraction Layer (HAL) to enable functions such as turning the camera on and off, and capturing images. The Camera Service can also provide interfaces for accessing camera functions. By calling these interfaces, applications can control various camera functions, such as starting a preview, capturing photos, recording video, adjusting focus, and setting exposure.

[0089] In this embodiment, a predefined interface based on Android Interface Definition Language (AIDL) is used. The lock screen application can interact with the camera HAL (Hardware Abstraction Layer) by calling this predefined private AIDL interface (i.e., the camera interface). When the lock screen application receives a user's face unlock operation, it sends a face unlock request to the camera HAL by calling this camera interface. After receiving the face unlock request, the camera HAL controls the camera to power on and configure its parameters. In other words, before the face recognition control module configures the camera to output an image through the camera HAL, the camera HAL first controls the camera to perform preparatory work before image acquisition. Thus, when the face recognition request is sent to the camera HAL, the preparatory work for image acquisition has been completed, and image acquisition can proceed directly.

[0090] It should be noted that other applications in the application layer can interact with the camera HAL in the hardware abstraction layer by calling this camera interface, so as to control the camera to power on and configure parameters in advance through the camera HAL.

[0091] For example, taking a payment application as an example, when the payment application receives a user's facial recognition payment input, it interacts with the camera HAL (Hardware Abstraction Layer) by calling the camera interface. The camera HAL then controls the camera to power on and configure its parameters in advance. This reduces the camera startup time, increases the speed of facial recognition, and thus improves the efficiency of facial payment.

[0092] For example, in a banking application, when the application receives a user's facial recognition login input, it interacts with the camera's Hardware Abstraction Layer (HAL) via the camera interface. The HAL then controls the camera to power on and configure its parameters in advance. This reduces camera startup time, increases facial recognition speed, and ultimately improves facial login efficiency.

[0093] The application framework layer may also include a window manager, content provider, view system, phone manager, resource manager, notification manager, etc., and this application embodiment does not impose any limitations on this.

[0094] The Android Runtime comprises the core libraries and the virtual machine. The Android Runtime is responsible for scheduling and managing the Android system. The core libraries consist of two parts: one part contains the functionalities that Java calls, and the other part is the core Android library itself. The application layer and application framework layer run within the virtual machine. The virtual machine executes the Java files from the application layer and application framework layer as binary files. The virtual machine is used for managing object lifecycles, stack management, thread management, security and exception management, and garbage collection, among other functions.

[0095] The system library can include multiple functional modules, such as a surface manager, media libraries, 3D graphics processing libraries (e.g., OpenGL ES), and a 2D graphics engine (e.g., SGL). The surface manager manages the display subsystem and provides fusion of 2D and 3D layers for multiple applications. The media libraries support playback and recording of various common audio and video formats, as well as still image files. The media libraries support multiple audio and video encoding formats, such as MPEG4, H.264, MP3, AAC, AMR, JPG, and PNG. The 3D graphics processing libraries are used for 3D graphics drawing, image rendering, compositing, and layer processing. The 2D graphics engine is the drawing engine for 2D graphics.

[0096] A hardware abstraction layer (HAL) is used to abstract away hardware. For example, an HAL can encapsulate Linux kernel drivers, providing an interface to the upper layers and shielding them from the implementation details of the lower-level hardware.

[0097] The hardware abstraction layer may include Wi-Fi HAL, audio HAL, camera HAL, and face recognition control module, etc.

[0098] The camera HAL is the core software framework of the camera. The camera HAL can include sensor nodes and image front end (IFE) nodes. Sensor nodes and IFE nodes are components (nodes) in the image data and control command transmission path (also called transmission pipeline) created by the camera HAL. The face recognition control module is the core software framework / application of face recognition.

[0099] The kernel layer is the layer between hardware and software. It contains at least the display driver, camera driver, audio driver, and sensor driver. The camera driver, in particular, is the driver layer for the camera device and is primarily responsible for interacting with the hardware.

[0100] The hardware layer can include a display, a Time-of-Flight (TOF) camera (such as a 2D TOF camera or a 3D TOF camera), and secure memory (secure buffer). The TOF camera can also be called a TOF sensor. Secure memory refers to memory with security protection features, which can be used to store the raw image data (RAW data) captured by the TOF camera.

[0101] Figure 4 This is another example of the interaction between software modules during the face unlock process provided in this application embodiment.

[0102] The following is combined with Figure 4 Taking a lock screen application as an example, this paper illustrates the interaction process between software modules involved in the face recognition method provided in this application embodiment.

[0103] like Figure 4As shown, the lock screen application in the application layer can interact with the camera HAL in the hardware abstraction layer by calling the camera interface (i.e., the private AIDL interface). After receiving a user's face unlock operation, the lock screen application sends a face unlock request to the camera HAL by calling the camera interface. After receiving the face unlock request, the camera HAL interacts with the camera driver through the camera kernel layer to control the camera (such as a TOF camera) to power on and configure its parameters. Additionally, the camera HAL updates the camera's status to "power-on complete".

[0104] Furthermore, the lock screen application can interact with the face recognition SDK to perform face recognition. The face recognition SDK can interact with the face recognition service in the application framework layer by calling preset application programming interfaces (APIs) and send face recognition requests to the service. After receiving the face recognition request, the face recognition service can interact with the face recognition control module in the hardware abstraction layer and send a face recognition request to it. Upon receiving the face recognition request, the face recognition control module interacts with the camera service in the application framework layer by calling existing VNDK interfaces and sends a camera request to the camera service. This camera request requests the camera service to configure the camera for image output.

[0105] The process of controlling the camera's image output configuration is as follows: The camera service can interact with the camera HAL in the hardware abstraction layer. The camera HAL can interact with the camera driver through the camera kernel layer, and vice versa. Upon receiving a camera request, the camera service sends a power-on request to the camera HAL. Upon receiving the power-on request, the camera HAL determines if the camera's current state is power-on complete. If so, the camera HAL calls the camera driver through the camera kernel layer and sends a loading request to the camera to request the camera to load the pre-configured streaming parameters. After loading the pre-configured streaming parameters, the camera sends a loading completion message to the camera HAL through the camera driver and camera kernel layer. Upon receiving the loading completion message, the camera HAL returns a loading completion message to the camera service. Upon receiving the loading completion message, the camera service sends an image acquisition request to the camera HAL. Upon receiving the image acquisition request, the camera HAL calls the camera driver through the camera kernel layer to drive the camera to acquire the user's facial image. After capturing the user's facial image, the camera stores the captured image in secure memory and returns the image capture result to the camera HAL, indicating that the camera has completed the image capture process. Upon receiving the image capture result, the camera HAL returns it to the camera service. The camera service, upon receiving the image capture result, interacts with the face recognition control module and the trusted face application (TA) running in the TEE environment to execute the face recognition process.

[0106] like Figure 4 As shown, the face recognition control module, also known as the face recognition daemon module, can include a face HAL processing unit, a face client application (referred to as face CA), a camera control processing unit, and a media secure processor.

[0107] After receiving the image acquisition result, the face recognition module interacts with the face TA running in the TEE environment via the face CA, and sends the image acquisition result to the secure camera image acquisition unit in the face TA. The secure camera image acquisition unit interacts with secure memory through the trusted camera service and retrieves the facial image captured by the camera from the secure memory. The secure camera image acquisition unit sends the facial image to the TOF algorithm module through the algorithm adaptation layer. The TOF algorithm module can transform the facial image (e.g., convert it into a grayscale image and a depth map), and then sends the transformed facial image to the face recognition algorithm module through the algorithm adaptation layer. The face recognition algorithm module uses a face detection algorithm to locate the face region in the transformed facial image, and extracts key facial features (such as the position of the eyes, nose, and mouth, and facial contours) from the face region, and then sends the extracted facial features to the face feature storage and processing unit in the face TA. The Internet Finance Identity Authentication Alliance Trusted Application (IFAA TA) compares facial features with pre-stored facial features to obtain a face recognition result, which is then returned to the face recognition control module via the face TA. Upon receiving the face recognition result, the face recognition control module returns it to the face recognition service, which in turn returns it to the lock screen application via the face recognition SDK. After receiving the face recognition result, the lock screen application determines whether to unlock the device based on the result (if face recognition is successful, it unlocks; if face recognition fails, it does not unlock, i.e., unlocking fails).

[0108] The following embodiments of this application will be used to illustrate having Figure 2 and Figure 3 Taking the electronic device with the structure shown as an example, and in conjunction with the accompanying drawings and application scenarios, the face recognition method provided in this application embodiment will be specifically described.

[0109] Before introducing the face recognition method provided in the embodiments of this application, the application scenarios and face recognition process will be explained first.

[0110] Facial recognition technology is widely used in electronic devices, bringing users a convenient and secure experience. The above are some application scenarios of facial recognition technology in smart terminals:

[0111] Scenario 1: When using a mobile phone, a user can unlock it using a lock screen app. During face unlock, the lock screen app can obtain the user's facial image and perform facial recognition to determine whether to unlock the phone based on the recognition results.

[0112] Scenario 2: Users can make facial recognition payments through a payment app on their mobile phone. When making a facial recognition payment, the app captures the user's facial image and uses facial recognition technology to verify their identity. Based on the verification result, it determines whether to continue the payment process.

[0113] Scenario 3: When purchasing goods in a mobile shopping app, users can choose to complete the transaction using facial recognition payment. During the facial recognition payment process, the shopping app can capture the user's facial image and use facial recognition technology to verify their identity. Based on the verification result, it determines whether to continue the payment operation.

[0114] Scenario 4: When logging into a banking application, users can choose to verify their identity through facial recognition. The banking application can then determine whether the user has the necessary access rights based on the facial recognition results, thereby improving the security of the banking application.

[0115] Figure 5 This is a schematic diagram of a face recognition process provided in an embodiment of this application. Figure 5 As shown, the face recognition process can include: First, acquiring the user's face image (i.e., facial image). Next, preprocessing the face image (e.g., converting it to grayscale and depth maps). After preprocessing, a feature model is used to extract key facial features from the face image. This feature model is trained using a large number of face sample images. During model training, Gabor filters and local binary patterns (LBP) can be used to extract facial features from the face sample images, or deep learning models (such as convolutional neural networks) can be used to extract facial features and train the model using these features. Finally, the extracted facial features are matched with pre-stored facial features in a database to output the face recognition result.

[0116] For example, when acquiring facial images, the user's facial image can be acquired in real time via a camera, or a previously acquired and stored image can be used directly, or continuous facial images can be acquired through video recording. This application does not limit the method of acquiring facial images.

[0117] The following example illustrates the face recognition method provided in this application by using a lock screen application in an electronic device to acquire facial images through the camera in the electronic device.

[0118] In this embodiment, after receiving a user's face unlock operation, the lock screen application can interact with the camera HAL to notify the camera HAL to control the camera to power on and configure parameters, thereby enabling the camera to power on and configure parameters in advance. Simultaneously, the lock screen application can interact with the face recognition control module of the hardware abstraction layer to notify the face recognition control module to execute the face recognition process. During face recognition, the face recognition control module can send a camera open request to the camera HAL. Upon receiving the camera open request, the camera HAL can control the camera to load parameters and acquire image data, and then perform face recognition based on the image data.

[0119] Additionally, during the process of camera HAL controlling camera configuration and image output, the camera HAL can set the camera's status. Based on this, while the camera HAL controls the camera to power on and configure parameters, it can also update the camera's status to "power-on complete," indicating that the camera has completed power-on and parameter configuration. Therefore, upon receiving a camera request, the camera HAL can determine whether the camera's current status is "power-on complete," i.e., based on the camera's current status, it can determine whether the camera has completed power-on and parameter configuration. If the camera's current status is "power-on complete," it means the camera has completed power-on and parameter configuration, and at this point, the camera is controlled to load parameters and acquire image data. Otherwise, the camera is first controlled to power on and configure parameters; after the camera has completed power-on and parameter configuration, the camera is controlled to load parameters and acquire image data.

[0120] The above is a general description of the face recognition method provided in the embodiments of this application. According to the above description, the face recognition method may include a camera configuration and image output stage and a face recognition stage. The camera configuration and image output stage and the face recognition stage will be described in detail below with reference to the accompanying drawings.

[0121] 1. Camera configuration and image output stage

[0122] According to the description of the camera configuration and image output process in the foregoing embodiments, the camera configuration and image output process may include a startup phase (also known as a power-on phase), a data distribution phase, a request phase (i.e., an image acquisition phase), and a result return phase.

[0123] Specifically, the startup phase can include two stages: pre-acquisition and camera acquisition. In the pre-acquisition stage, the availability of camera resources can be checked and resources allocated to the camera. In the camera acquisition stage, camera resources are acquired to ensure the camera can be used by the application or service. The streaming phase can include two stages: initialization setup and loading initialization configuration. In the initialization setup stage, camera parameters (i.e., streaming parameters) are set, such as image resolution, frame rate, and data stream format. In the loading initialization configuration stage, the set streaming parameters are loaded into the camera so that the camera can acquire images according to the streaming parameters.

[0124] It is understandable that controlling the camera to power on, as mentioned earlier, refers to controlling the camera to perform the startup phase. Controlling the camera to configure parameters refers to controlling the camera to perform the initialization setup phase.

[0125] The following is combined with Figure 6 The process of HAL-controlled camera configuration and image output is illustrated.

[0126] Figure 6 This is a schematic diagram of the interactive flow of a camera configuration and image output process provided in an embodiment of this application. For example... Figure 6 As shown, the process of camera HAL controlling camera configuration and output can include the following steps:

[0127] S601, the lock screen application in the application layer receives face unlock operations.

[0128] Face unlocking can be used to trigger face recognition. For example, face unlocking can be a user's action on the power button of an electronic device (e.g., pressing), a user's action on the display screen of the electronic device (e.g., clicking, swiping), a user picking up the electronic device, or a user unplugging the charging cable. The specific type of face unlocking operation is not limited in this embodiment.

[0129] In response to a face unlock operation, the lock screen application sends a face unlock request to the camera HAL in the hardware abstraction layer by calling the AIDL interface.

[0130] Specifically, the lock screen application can interact with the camera HAL (Hardware Abstraction Layer) by calling the camera interface, and after receiving the user's face unlock input, it sends a face unlock request to the camera HAL by calling the AIDL interface. This face unlock request can be used to instruct the camera to power on. In this embodiment, by pre-defining the AIDL interface, the lock screen application can directly interact with the camera HAL through the AIDL interface, shortening the interaction process from the upper-layer application to the camera HAL.

[0131] Optionally, a face unlock request can also be used to instruct the camera to be set to a powered-on state.

[0132] The "Powered On" status indicates that the camera has completed power-on and parameter configuration. For example, the "Powered On" status can specifically be "DO_INIT_AFTER_ACQUIRED", meaning that the camera has been acquired and is in the process of initialization.

[0133] In S603, after receiving a face unlock request from the lock screen application, the camera HAL creates an asynchronous power-on thread to control the camera to power on and configure parameters.

[0134] Specifically, after receiving a face unlock request (i.e., Get Command), the camera HAL can determine the camera that matches the preset streaming parameters from among multiple cameras on the electronic device (for ease of description, the camera matching the preset streaming parameters is referred to as Camera 1). The preset streaming parameters can be used to create a data stream, through which image data for face recognition can be acquired. After determining Camera 1, the camera HAL creates an asynchronous power-on thread to control Camera 1 to power on and configure its parameters in advance. It is understood that in subsequent steps, "camera" refers to Camera 1.

[0135] It should be noted that the process of the camera's HAL controlling the camera to power on and configure parameters can be found in the following section. Figure 9 Detailed explanation.

[0136] S604, the camera HAL sets the status of the camera (i.e., camera 1) to powered on.

[0137] Specifically, taking the "DO_INIT_AFTER_ACQUIRED" state as an example, after the camera HAL receives a face unlock request sent by the lock screen application, it sets the camera state to "DO_INIT_AFTER_ACQUIRED".

[0138] It should be noted that the above S503 and S504 can be in accordance with Figure 6 The execution order shown in the illustrated embodiment can also be executed in other orders, or S503 and S504 can be executed simultaneously. This application embodiment does not impose any limitation on the execution order of S503 and S504 described above.

[0139] The above steps S602 to S604 are illustrative descriptions of the process of powering on the camera in advance and configuring its parameters. The following is an illustrative description of the process of loading parameters and acquiring images for the camera.

[0140] Figure 7 This is a schematic diagram illustrating a process for updating the camera's state using the HAL (Hybrid Allocation) method, as provided in an embodiment of this application. For example,... Figure 7 As shown, the process by which the camera HAL updates the camera's state to "powered on" (i.e., step S604) may include the following steps:

[0141] S701, the camera HAL sends a status setting request to the camera kernel layer, whereby the status setting request is used to indicate that the camera's status is set to the powered-on state.

[0142] Specifically, after receiving a face unlock request, the camera HAL can determine the camera (i.e., camera 1) that matches the preset streaming parameters, and obtain the camera identity (ID) of camera 1. Then, the camera HAL can send a state setting request to the camera kernel layer, which can carry the camera identity of camera 1, to set the state of camera 1 to power-on.

[0143] In S702, the camera kernel layer receives a status setting request from the camera HAL and then sends a status setting request to the camera driver.

[0144] S703: When the camera driver receives a status setting request from the camera kernel layer, it sends a status setting request to the camera.

[0145] S704, the camera sets its current status to powered on.

[0146] Specifically, after receiving the status setting request sent by the camera driver, the camera sets the status parameter in its register to the powered-on state (such as DO_INIT_AFTER_ACQUIRED).

[0147] In this embodiment, when the camera HAL receives a face unlock request from the lock screen application, it can set the camera status to "powered on". This allows the camera HAL to obtain the camera's status upon receiving a camera unlock request, and control the camera to load parameters while the camera is powered on. This ensures that parameters are loaded immediately after powering on the camera, preventing camera malfunctions and improving user experience.

[0148] In S605, the lock screen application performs face recognition by calling the application-layer face recognition SDK.

[0149] Specifically, after the lock screen application sends a face unlock request to the camera HAL in the hardware abstraction layer by calling the camera interface, the lock screen application performs face recognition by calling the face recognition SDK in the application layer.

[0150] Additionally, the lock screen app can register a callback with the face recognition SDK so that the face recognition SDK can return the face recognition result to the lock screen app after obtaining the face recognition result.

[0151] It should be noted that, according to the description of the foregoing embodiments, after receiving a face unlock request from the lock screen application, the camera HAL sets the camera's state and creates an asynchronous power-on thread to control the camera's power-on and parameter configuration. Simultaneously, the lock screen application can call the face recognition SDK to perform face recognition. That is, the execution order of S603, S604, and S605 is related to the actual execution process. In this embodiment, the execution order of S603, S604, and S605 is not limited.

[0152] S606, the face recognition SDK sends a face recognition request to the face recognition service in the application framework layer.

[0153] A face recognition request is used to instruct a face recognition service to perform face recognition. For example, a face recognition request may carry an identifier of the face recognition type, an image resolution of 1, and a data stream format of 1.

[0154] The face recognition type can include 2D face recognition and 3D face recognition. Each face recognition type has a unique identifier to distinguish it from others. For example, the identifier for 2D face recognition is 0, and the identifier for 3D face recognition is 1.

[0155] Image resolution indicates the size of the image to be captured. For example, an image resolution of 1920x1080 pixels means that the width of the image to be captured is 1920 pixels and the height is 1080 pixels.

[0156] The data stream format can be used to indicate the image format of the image to be acquired. For example, data stream formats may include YUV, raw image format (RAW), JPEG (joint photographic experts group), BMP (bitmap), and TIFF (tagged image file format), etc.

[0157] For example, if the face recognition type identifier carried in the face recognition request is 1, the image resolution is 1280x2898 pixels, and the data stream format is RAW16, the face recognition request can indicate that 3D face recognition is to be performed, the size of the image to be acquired is 1280x2898 pixels, and the format of the image to be acquired is RAW16.

[0158] In addition, the face recognition SDK can also register a callback with the face recognition service so that the face recognition service can return the face recognition result to the face recognition SDK after obtaining the face recognition result.

[0159] S607: After receiving a face recognition request from the face recognition SDK, the face recognition service sends a face recognition request to the face recognition control module of the hardware abstraction layer.

[0160] In other words, the face recognition SDK can notify the face recognition control module to perform face recognition through the face recognition service.

[0161] For example, a face recognition request may include an image storage address, which is used to store the facial image captured by the camera.

[0162] In addition, the face recognition service can also register a callback with the face recognition control module so that the face recognition control module can return the face recognition result to the face recognition service after obtaining the face recognition result.

[0163] S608, in response to a face recognition request, the face recognition control module matches the camera.

[0164] Specifically, upon receiving a face recognition request from the face recognition service, the face recognition control module can obtain the face recognition type identifier, image resolution 1, and data stream format 1 from the request. Furthermore, the face recognition control module can query the capability information of each camera in the electronic device from the camera service. Then, based on the face recognition type identifier, image resolution 1, data stream format 1, and the capability information of each camera, the face recognition control module determines the matching camera (i.e., camera 1) from the multiple cameras of the electronic device.

[0165] The camera's capability information may include the camera identifier, the maximum resolution supported by the camera, the data stream format supported by the camera, and whether the camera supports depth information acquisition. The camera's capability information may also include the camera's location information, such as front or rear camera. Of course, the camera's capability information may also include other performance information of the camera; however, this embodiment does not specifically limit the camera's capability information.

[0166] For example, taking an electronic device equipped with three cameras as an example, the capability information of these three cameras can be shown in Table 1:

[0167] Table 1 Capability information of each camera in electronic devices

[0168] Camera logo Maximum resolution Data stream format Depth Information Location information 1 4096x3072 pixels YUV No (depth information collection is not supported) Rear 2 3264x2448 pixels YUV no Front 3 1280x2898 pixels RAW16 Yes (supports collecting depth information) Front

[0169] Among them, the camera identified as 3 can be a TOF camera, which supports the acquisition of depth information. The cameras identified as 1 and 2 can be ordinary cameras and do not support the acquisition of depth information. Of course, electronic devices can also be equipped with more front or rear cameras, such as two front cameras and four rear cameras. This application embodiment does not limit this.

[0170] It should be noted that Table 1 is only an example, and the data stream format corresponding to each camera can include multiple formats. For example, the data stream format supported by the camera identified as 1 can include YUV format or RAW format, and this embodiment of the application does not limit this.

[0171] For example, continuing with the face recognition request carrying a face recognition type identifier of 1, an image resolution of 1280x2898 pixels, a data stream format of RAW16, and the electronic device equipped with the above three cameras, the face recognition control module determines the matching camera (i.e., camera 1) as the camera with camera identifier 3 (i.e., TOF camera) based on the face recognition type identifier (such as 1), image resolution 1 (1280x2898 pixels), and data stream format 1 (RAW16).

[0172] In addition, camera capability information is typically stored in the camera HAL (Host Access Array). The camera service can retrieve the capability information of each camera in the electronic device from the camera HAL. Specifically, during the power-on process of the electronic device, the camera service can send a capability information query request to the camera HAL. This request is used to query the camera capabilities supported by each camera in the electronic device. After receiving the capability information query request, the camera HAL can send the capability information of each camera in the electronic device to the camera service. After receiving the capability information of each camera, the camera service stores the capability information of each camera. Thus, when the camera service receives a capability information query request from the face recognition control module, it sends the capability information of each camera in the electronic device to the face recognition control module.

[0173] S609, the face recognition control module interacts with the camera service of the application framework layer through the VNDK interface and sends camera requests to the camera service.

[0174] A camera request is used to instruct the camera to configure image output. The camera request can carry the camera identifier and image storage address of the camera to be called (i.e., camera 1).

[0175] For example, the camera request may also carry image resolution 1, data stream format 1, and a security identifier. The security identifier can be used to indicate that the captured image data is stored in secure memory. That is, the security identifier can be used to request a block of secure memory to store the image data captured by the camera. For example, the security identifier can be 0 or 1; 1 indicates that the image data captured by the camera is stored in secure memory, and 0 indicates that the image data captured by the camera is stored in non-secure memory. S610, upon receiving the camera request from the face recognition control module, the camera service sends a camera open request to the camera HAL.

[0176] The camera-on request is used to instruct the camera HAL to control the camera to start, that is, the camera HAL controls the camera to perform a power-on operation. The camera-on request can carry the camera identifier of the camera to be called (i.e., camera 1).

[0177] Additionally, the camera service can register a callback with the camera HAL to return the camera-open result after the camera HAL executes the camera-open request.

[0178] S611, after receiving the camera opening request sent by the camera service, the camera HAL determines whether the current state of the camera is powered on. If the current state of the camera is powered on, then S612 is executed; otherwise, the camera is first controlled to power on and configure parameters. After the camera has completed power-on and parameter configuration, the camera is controlled to load parameters and acquire images (i.e., the original configuration and image output process).

[0179] Specifically, after receiving a camera open request from the camera service, the camera HAL interacts with the camera kernel layer and calls the camera driver to send a status acquisition request to the camera. This status acquisition request may carry the camera identifier for camera 1 to obtain the camera's current status. Upon receiving the status acquisition request, the camera reads its current status from its corresponding register and returns it to the camera driver. The camera driver then returns the current status to the camera HAL through the camera kernel layer. After receiving the current status, the camera HAL determines whether the camera is in a powered-on state (i.e., DO_INIT_AFTER_ACQUIRED). If the camera is powered-on, it means the camera has completed power-on and parameter configuration; at this point, the camera loads parameters and acquires image data. If the camera is not powered-on, it means the camera has not completed power-on and parameter configuration; therefore, the camera is first powered on and configured. After the camera completes power-on and parameter configuration, the camera loads parameters and acquires image data (i.e., the original configuration and image output process is executed). For details on executing the original configuration and drawing process, please refer to the following section. Figure 11Detailed explanation.

[0180] In this way, after receiving a request to turn on the camera, the camera's HAL can obtain the camera's status and, if the camera is powered on, control the camera to load parameters. This ensures that parameters are loaded immediately after powering on the camera, preventing camera malfunctions and improving the user experience.

[0181] In some examples, after determining that the camera's current state is powered on, the camera HAL updates the camera's state from restored to powered on to the initial state (i.e., m_status). The initial state refers to the state of the camera before it was powered on.

[0182] In this embodiment, after determining that the current state of the camera is powered on, the camera hardware abstraction layer restores the camera's state to its initial state. This ensures that when the camera is called again, it can be powered on and its state set in advance, thereby avoiding the camera's normal operation being affected by the failure to restore its state.

[0183] S612, after determining that the camera is currently powered on, the camera HAL controls the camera to load preset distribution parameters.

[0184] For example, such as Figure 8 As shown, the process of controlling the camera to load preset streaming parameters is as follows: After determining that the camera is currently powered on, the camera HAL calls the LoadSensorInitCfgCmd() function to execute the loading instruction. Then, the camera kernel layer calls the LoadSensorInitCfgCmd() function to execute the loading instruction. Next, the camera driver calls the LoadSensorInitCfgCmd() function to execute the loading instruction, thereby driving the camera to load the preset streaming parameters. After completing the loading of the preset streaming parameters, the camera returns a loading completion message to the camera driver. The camera driver then sends a loading completion message to the camera HAL through the camera kernel layer, and the camera HAL returns a loading completion message to the camera service. The loading completion message indicates that the camera's preparation work before image acquisition (such as power-on, parameter configuration, and parameter loading) has been completed.

[0185] S613, after receiving the loading complete message from the camera, the camera HAL sends a loading complete message to the camera service.

[0186] S614, upon receiving the loading completion message from the camera HAL, the camera service sends an image acquisition request to the camera HAL. The image acquisition request requests the camera to acquire the user's facial image based on preset streaming parameters. The image acquisition request may include the image storage address.

[0187] After receiving an image acquisition request, the S615 camera HAL controls the camera to capture the user's facial image.

[0188] Specifically, after receiving an image acquisition request from the camera service, the camera HAL sends an image acquisition request to the camera kernel layer. After receiving the image acquisition request from the camera HAL, the camera kernel layer sends an image acquisition request to the camera driver. After receiving the image acquisition request from the camera kernel layer, the camera driver sends an image acquisition request to the camera.

[0189] S616: After receiving the image acquisition results sent by the camera, the camera HAL sends the image acquisition results to the camera service.

[0190] Specifically, after receiving an image acquisition request from the camera HAL, the camera acquires the user's facial image (raw data) and stores it in secure memory according to the image storage address. After completing image acquisition, the camera sends the image acquisition result to the camera driver, which in turn sends the image acquisition result to the camera HAL through the camera kernel layer. The image acquisition result indicates that the camera has completed image acquisition.

[0191] In this embodiment, after initiating face unlock, the lock screen application can interact with the camera HAL to notify the camera HAL to power on and configure the camera parameters, thus enabling the camera to power on and configure its parameters in advance. Simultaneously, the lock screen application can interact with the face recognition control module of the hardware abstraction layer to notify the face recognition control module to execute the face recognition process. During face recognition, the face recognition control module can send a camera request to the camera HAL. Upon receiving the camera request, the camera HAL can control the camera to load parameters and acquire image data, and then perform face recognition based on the image data. This allows for early power-on and parameter configuration of the camera, shortening the waiting time for image output, thereby reducing the face unlock process time and improving the efficiency of face unlock.

[0192] Figure 10 This is a comparative diagram showing the face unlock time before and after optimization. (Example:) Figure 10As shown in (a) of the related technology, after initiating face unlock, the CLSAcquireDevice() function is executed to acquire camera resources, i.e., to control the camera's power-on. After the CLSAcquireDevice() function completes, the HAL3ConfigureStreams() function is called to perform the streaming phase. Only during the streaming phase is the LoadSensorInitCfgCmd() function executed to control the camera's loading parameters. It can be seen that there is a certain time interval (e.g., 73ms) between the end of the CLSAcquireDevice() function execution and the start of the LoadSensorInitCfgCmd() function execution. This means there is a relatively long interval between the camera's power-on completion and the start of the streaming parameter loading, which can cause blocking, affecting the camera's image output time and thus impacting the efficiency of face unlock.

[0193] like Figure 10 As shown in (b) of this embodiment, after initiating face unlock (e.g., pressing the power button), the CLSAcquireDevice() function is executed to acquire camera resources, i.e., to power on the camera. After the camera is powered on, the LoadSensorInitCfgCmd() function is executed, i.e., to load parameters for the camera. Figure 10 Compared to (a) in the previous example, there is almost no time interval between the end of the CLSAcquireDevice() function execution and the start of the LoadSensorInitCfgCmd() function execution. In other words, the face recognition method provided in this application loads the parameter instructions immediately after the camera is powered on, thus reducing the camera's image output time and shortening the face unlock process. Through this optimization, the total time of the face unlock process can be shortened to 150–170 milliseconds.

[0194] The above is a schematic illustration of the process of configuring and generating images using the camera control system. The following is a schematic illustration of the process of powering on and configuring the parameters of the camera using the HAL control system. Figure 9 This is a schematic diagram illustrating a process for a camera HAL to control a camera for power-on and parameter configuration, as provided in an embodiment of this application. For example, as shown... Figure 10 As shown, the process of the camera HAL controlling the camera to power on and configure parameters (i.e., step S603) may include the following steps:

[0195] S901, the camera HAL receives a face unlock request sent by the lock screen application via the AIDL interface.

[0196] S902, in response to the face unlock request, the camera HAL sends a power-on request to the camera kernel layer. This power-on request instructs the camera used for face recognition (i.e., camera 1) to perform a power-on operation. The power-on request may include the camera's identifier.

[0197] For example, the camera HAL calls the PreAcquiredDevice() and AcquiredDevice() functions to send a power-on request.

[0198] In some examples, the camera HAL can obtain preset streaming parameters before sending a power-on request to the camera kernel layer. These preset streaming parameters may include an identifier for the face recognition type, image resolution 1, and data stream format 1. Based on the preset streaming parameters and the capability information of each camera in the electronic device, the camera HAL determines the camera (i.e., camera 1) that matches the preset streaming parameters from among the multiple cameras in the electronic device, and obtains the camera identifier of camera 1.

[0199] For example, continuing with the preset streaming parameters including face recognition type identifier as 1, image resolution 1 as 1280x2898 pixels, and data stream format 1 as RAW16, the camera HAL determines the matching camera (i.e., camera 1) as the camera with camera identifier 3 (i.e., TOF camera) based on the face recognition type identifier (such as 1), image resolution 1 (1280x2898 pixels), and data stream format 1 (RAW16).

[0200] For details on the specific implementation of HAL matching between the camera and the camera, please refer to the aforementioned implementation of S508, which will not be repeated here.

[0201] S903: After receiving the power-on request from the camera HAL, the camera kernel layer sends a power-on request to the camera driver.

[0202] For example, the camera kernel layer calls the PreAcquiredDevice() and AcquiredDevice() functions to send a power-on request.

[0203] S904: After receiving the power-on request from the camera kernel layer, the camera driver sends a power-on request to the camera.

[0204] For example, the camera driver calls the PreAcquiredDevice() and AcquiredDevice() functions to send a power-on request.

[0205] S905: After receiving a power-on request from the camera driver, the camera performs a power-on operation.

[0206] S906, Camera 1 sends a power-on completion message to the camera driver. The power-on completion message indicates that the camera has been powered on.

[0207] S907, the camera driver sends a power-on completion message to the camera kernel layer.

[0208] S908, the camera kernel layer sends a power-on completion message to the camera HAL.

[0209] After receiving the power-on completion message, the camera HAL of the S909 creates the corresponding data flow path according to the preset distribution parameters.

[0210] Specifically, the camera HAL can select available nodes based on the camera identifier of camera 1, image resolution 1, and data stream format 1, and then create corresponding data stream paths based on the available nodes. For example, continuing with the example of image resolution 1 being 1280x2898 pixels, data stream format 1 being RAW16, and camera identifier being 3, sensor nodes and IFE nodes can be selected.

[0211] S910: The camera HAL sends an initialization request to the camera kernel layer, which carries preset flow parameters and the camera identifier.

[0212] Specifically, after the camera HAL successfully creates the data flow path, it sends an initialization request to the camera kernel layer. The initialization request can carry preset flow parameters and the camera identifier of the camera to send the preset flow parameters to the camera.

[0213] For example, the camera HAL calls the StartSensorInitSetting() function to send an initialization request.

[0214] S911: After receiving the initialization request sent by the camera HAL, the camera kernel layer sends an initialization request to the camera driver.

[0215] For example, the camera kernel layer calls the StartSensorInitSetting() function to send an initialization request.

[0216] S912: After receiving the initialization request sent by the camera kernel layer, the camera driver sends an initialization request to the camera.

[0217] For example, the camera driver calls the StartSensorInitSetting() function to send an initialization request.

[0218] S913: After receiving the initialization request sent by the camera driver, the camera sets the preset distribution parameters.

[0219] Specifically, in response to the initialization request, the camera writes the information in the preset streaming parameters (such as image resolution 1 and data stream format 1) into the camera's register, thus completing the parameter configuration.

[0220] S914, the camera sends an initialization completion message to the camera driver. This initialization completion message indicates that the camera has completed its initialization configuration, meaning that the camera's power-on and parameter configuration have been completed.

[0221] S915, the camera driver sends an initialization completion message to the camera kernel layer.

[0222] S916, the camera kernel layer sends an initialization complete message to the camera HAL.

[0223] The above is a schematic illustration of the process of powering on and configuring parameters of the camera controlled by the HAL system.

[0224] In some embodiments, according to the description of S611 above, after receiving the camera opening request sent by the camera service, the camera HAL determines whether the current state of the camera is powered on. If the current state of the camera is not powered on, it means that the camera has not completed power-on and parameter configuration. At this time, the original configuration and image output process is executed, that is, the camera is first controlled to power on and configure parameters. After the camera has completed power-on and parameter configuration, the camera is controlled to load parameters and acquire image data.

[0225] like Figure 11 As shown, the process of generating a diagram using the original configuration may include the following steps:

[0226] S1101, after receiving the camera opening request sent by the camera service, the camera HAL determines whether the current state of the camera is powered on. If the current state of the camera is powered on, then S612 is executed; otherwise, S1103 is executed.

[0227] S1102, Camera HAL controls the camera to power on.

[0228] Specifically, the camera HAL sends a power-on command to the camera kernel layer. Upon receiving the power-on command from the camera HAL, the camera kernel layer sends a power-on command to the camera driver. Upon receiving the power-on command from the camera kernel layer, the camera driver sends a power-on command to the camera. Upon receiving the power-on command from the camera driver, the camera performs the power-on operation.

[0229] Based on the preceding description of the camera configuration and image output process, the startup phase can include two stages: pre-acquisition and camera acquisition. In specific implementation, the camera HAL calls the `PreAcquiredDevice()` function to execute the pre-acquisition instruction; the camera kernel layer calls the `PreAcquiredDevice()` function to execute the pre-acquisition instruction; and the camera driver calls the `PreAcquiredDevice()` function to execute the pre-acquisition instruction. After executing the pre-acquisition instruction, the camera HAL calls the `AcquiredDevice()` function to execute the pre-acquisition instruction; the camera kernel layer calls the `AcquiredDevice()` function to execute the pre-acquisition instruction; and the camera driver calls the `AcquiredDevice()` function to acquire the camera resources.

[0230] In some examples, when the camera HAL executes the PreAcquiredDevice() function, it updates the camera's initial state to state 1, which can indicate that the camera has not been acquired or is being acquired (i.e., UNACQUIRED_OR_ACQUIRING). When the camera HAL executes the AcquiredDevice() function, it updates the camera's state from state 1 to state 2, which can indicate that the camera has been acquired and is waiting for a streaming request (i.e., ACQUIRED_AND_WAIT).

[0231] S1103, the camera HAL receives the power-on complete message from the camera and sends a power-on complete message to the camera service.

[0232] S1104, upon receiving the power-on completion message from the camera HAL, the camera service sends a stream allocation request to the camera HAL. This stream allocation request is used to request the camera to perform initialization settings and load stream allocation parameters.

[0233] S1105, after receiving the streaming request sent by the camera service, the camera HAL controls the camera to perform initialization settings.

[0234] Specifically, the camera HAL calls the `StartSensorInitSetting()` function to load initialization instructions; the camera kernel layer also calls the `StartSensorInitSetting()` function to load initialization instructions. The camera driver calls the `StartSensorInitSetting()` function to load initialization instructions, thereby driving the camera to set its streaming parameters, i.e., performing initialization settings. These streaming parameters can include image resolution (1) and data stream format (1).

[0235] In some examples, when the camera HAL executes the StartSensorInitSetting() function, it updates the camera's state from state 2 to state 3 (i.e., the power-on complete state mentioned earlier). State 3 can indicate that initialization settings are in progress (i.e., DO_INIT_AFTER_ACQUIRED).

[0236] S1106, when the camera HAL receives the initialization completion message sent by the camera, the camera HAL controls the camera to load the distribution parameters.

[0237] Specifically, the camera HAL calls the LoadSensorInitCfgCmd() function to execute the loading instruction; the camera kernel layer calls the LoadSensorInitCfgCmd() function to execute the loading instruction; and the camera driver calls the LoadSensorInitCfgCmd() function to execute the loading instruction, thereby driving the camera to load the distribution parameters.

[0238] In some examples, after the camera HAL executes the LoadSensorInitCfgCmd() function and sends SensorInitSetting, it updates the camera's state from state 3 to state 4. State 4 can indicate that the initialization settings are complete and the camera is waiting for the data stream request (i.e., INIT_AND_WAIT).

[0239] S1107, the camera HAL receives the loading complete message from the camera and sends a loading complete message to the camera service.

[0240] After the camera service receives the loading completion message, the camera service sends an image acquisition request to the camera HAL, requesting the camera to acquire the user's facial image, and then continues to execute S614-S616.

[0241] The above section provides an illustrative explanation of the camera configuration and image output stage. The following section provides an illustrative explanation of the face recognition stage.

[0242] 2. Facial Recognition Stage

[0243] Figure 12 This is a flowchart illustrating an example of a face recognition process provided in an embodiment of this application. Figure 12 As shown, the face recognition process may include the following steps:

[0244] S1201, the camera service sends image acquisition results to the face recognition control module through the VNDK interface.

[0245] After the camera service receives the image acquisition result (S616) sent by the camera HAL, it sends the image acquisition result to the face recognition control module through the VNDK interface. The image acquisition result may contain the image storage address corresponding to the facial image captured by the camera.

[0246] S1202, the face recognition control module sends the image acquisition result to the face TA.

[0247] S1203, the face recognition sensor retrieves the user's facial image from secure memory based on the storage address.

[0248] S1204, Face TA determines the face recognition result based on the user's facial image.

[0249] S1205, the face recognition TA sends the face recognition result to the face recognition control module.

[0250] S1206 After receiving the face recognition result sent by the face recognition TA, the face recognition control module sends the face recognition result to the face recognition service.

[0251] The face recognition control module transmits the face recognition result to the face recognition service based on the callback registered with the face recognition service.

[0252] S1207 After receiving the face recognition result sent by the face recognition control module, the face recognition service sends the face recognition result to the face recognition SDK.

[0253] The face recognition service uses a callback registered with the face recognition SDK to pass the face recognition result to the face recognition SDK.

[0254] S1208: After receiving the face recognition result from the face recognition service, the face recognition SDK sends the face recognition result to the lock screen application.

[0255] The face recognition SDK uses a callback registered with the lock screen application to pass the face recognition result to the lock screen application.

[0256] S1209: After receiving the face recognition result sent by the face recognition service, the lock screen application determines whether to unlock based on the face recognition result.

[0257] Specifically, if the face recognition is successful, the lock screen app executes the unlock command. If the face recognition fails, the lock screen app executes the unlock command or outputs a prompt message indicating that the face recognition failed.

[0258] In this embodiment, after receiving a user's face unlock operation, the lock screen application can interact with the camera HAL to notify the camera HAL to control the camera to power on and configure parameters, thereby enabling the camera to power on and configure parameters in advance. Simultaneously, the lock screen application can interact with the face recognition control module of the hardware abstraction layer to notify the face recognition control module to execute the face recognition process. During face recognition, the face recognition control module can send a camera open request to the camera HAL. Upon receiving the camera open request, the camera HAL can control the camera to load parameters and acquire image data, and then perform face recognition based on the image data.

[0259] Finally combined Figure 13 This paper provides an illustrative explanation of the camera status update process during the camera configuration and image output process.

[0260] Figure 13 This is a schematic diagram illustrating the state changes of a camera during the image output process, as provided in an embodiment of this application. For example... Figure 13 As shown, taking camera 1 as an example, during the process of controlling camera 1 to configure and output images according to the original process (i.e., startup - initialization settings - loading parameters), the state update process of camera 1 (i.e., ... Figure 13 The content within the box is:

[0261] First, the camera HAL calls the PreAcquiredDevice() function to load pre-acquire instructions, checking the availability of resources for camera 1 and allocating resources for camera 1. When the camera HAL executes the PreAcquiredDevice() function, it updates the camera's initial state (m_status) to state 1. State 1 can indicate that the camera is not being acquired or is being acquired (i.e., UNACQUIRED_OR_ACQUIRING).

[0262] After executing the PreAcquiredDevice() function, the camera HAL calls the AcquiredDevice() function to load the acquisition command and acquire the resources of camera 1. Specifically, when the camera HAL executes the AcquiredDevice() function, it updates the camera's state from state 1 to state 2. State 2 indicates that the camera has been acquired and is waiting for a streaming request (i.e., ACQUIRED_AND_WAIT).

[0263] After executing the `AcquiredDevice()` function, the camera HAL calls the `StartSensorInitSetting()` function to load initialization settings instructions to control the camera to perform initialization settings (i.e., set the streaming parameters). When the camera HAL executes the `StartSensorInitSetting()` function, it updates the camera's state from state 2 to state 3 (i.e., the power-on complete state mentioned earlier). State 3 indicates that initialization settings are in progress (i.e., `DO_INIT_AFTER_ACQUIRED`).

[0264] After executing the `StartSensorInitSetting()` function, the camera HAL calls the `LoadSensorInitCfgCmd()` function to execute the loading command, controlling the camera to load the streaming parameters. Specifically, after executing `LoadSensorInitCfgCmd()` and sending the `SensorInitSetting` command, the camera HAL updates the camera's state from state 3 to state 4. State 4 indicates that initialization is complete and the camera is waiting for a data stream request (i.e., `INIT_AND_WAIT`).

[0265] In some examples, when the camera is in state 4, if the mode is switched and the stream is re-distributed, the camera HAL calls the StartSensorInitSetting() function again, and when the StartSensorInitSetting() function is executed, the camera state is updated from state 4 to state 3.

[0266] In this embodiment, a state machine is established for the camera in the hardware layer to manage and track the various states of the camera. By using a state machine to maintain the camera's state, it is ensured that the normal operation of the camera does not interfere with other processes that call the camera (such as lock screen applications calling the camera for face unlock), thereby ensuring that the electronic device can function normally.

[0267] like Figure 13As shown, continuing with camera 1 as an example, when controlling camera 1 to pre-configure power-on, after receiving the face unlock request (i.e., Get Command) sent by the lock screen application through the AIDL interface, the camera HAL directly sets the state of camera 1 from the initial state (m_status) to state 3 (i.e., the power-on completed state mentioned earlier). State 3 can indicate that initialization settings are in progress (i.e., DO_INIT_AFTER_ACQUIRED). Based on this, after receiving the camera opening request sent by the camera service, the camera HAL determines whether the current state of the camera is state 3. If the current state of the camera is state 3, the LoadSensorInitCfgCmd() function is executed; otherwise, the camera is first controlled to power on and configure parameters. After the camera completes power-on and parameter configuration, the camera is controlled to load parameters and acquire images, that is, the PreAcquiredDevice() function, AcquiredDevice() function, StartSensorInitSetting() function, and LoadSensorInitCfgCmd() function are called in sequence.

[0268] In this way, after receiving a request to turn on the camera, the camera's HAL can obtain the camera's status and, if the camera is powered on, control the camera to load parameters. This ensures that parameters are loaded immediately after powering on the camera, preventing camera malfunctions and improving the user experience.

[0269] Additionally, after the camera's HAL controls the camera to complete image acquisition, the ReleaseDevice() function is called to shut down the camera. During the execution of ReleaseDevice(), the camera's state is updated from state 4 to state 5. State 5 indicates that the camera is exiting (i.e., DO_EXIT_AFTER_ACQUIRED_OR_INITED).

[0270] It's important to note that the current state of a camera can be represented by a status parameter. Understandably, different cameras in the hardware layer can be tracked and recorded using different status parameters. Specifically, the status parameter can carry a camera ID to distinguish different cameras in the hardware layer. For example, the hardware layer includes camera 1 (e.g., a front-facing camera) and camera 2 (e.g., a rear-facing camera). The current state of camera 1 is tracked and recorded using status parameter 1 (m_status(camera 1)), and the current state of camera 2 is tracked and recorded using status parameter 2 (m_status(camera 2)).

[0271] The foregoing has detailed examples of the face recognition method provided in the embodiments of this application. It is understood that, in order to achieve the above functions, the electronic device includes hardware and / or software modules corresponding to the execution of each function. Those skilled in the art should readily recognize that, based on the units and algorithm steps of the examples described in conjunction with the embodiments disclosed herein, this application can be implemented in hardware or a combination of hardware and computer software. Whether a function is executed by hardware or by computer software driving hardware depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application in conjunction with the embodiments, but such implementation should not be considered beyond the scope of this application.

[0272] This application embodiment can divide the electronic device into functional modules according to the above method example. For example, each function can be divided into a separate functional module, such as a detection unit, a processing unit, a display unit, etc., or two or more functions can be integrated into one module. The integrated module can be implemented in hardware or as a software functional module. It should be noted that the module division in this application embodiment is illustrative and only represents one logical functional division. In actual implementation, there may be other division methods.

[0273] It should be noted that all relevant content of each step involved in the above method embodiments can be referenced from the functional description of the corresponding functional module, and will not be repeated here.

[0274] The electronic device provided in this embodiment is used to execute the above-described face recognition method, and therefore can achieve the same effect as the above-described implementation method.

[0275] When using integrated units, the electronic device may further include a processing module, a storage module, and a communication module. The processing module is used to control and manage the operation of the electronic device. The storage module supports the execution of stored program code and data. The communication module supports communication between the electronic device and other devices.

[0276] The processing module can be a processor or a controller. It can implement or execute various exemplary logic blocks, modules, and circuits described in conjunction with the disclosure of this application. The processor can also be a combination that implements computing functions, such as a combination of one or more microprocessors, a digital signal processor (DSP), and a microprocessor, etc. The storage module can be a memory. The communication module can specifically be a radio frequency circuit, a Bluetooth chip, a Wi-Fi chip, or other devices that interact with other electronic devices.

[0277] In one embodiment, when the processing module is a processor and the storage module is a memory, the electronic device involved in this embodiment can be a device having... Figure 2 The device with the structure shown.

[0278] This application also provides a computer-readable storage medium storing a computer program that, when executed by a processor, causes the processor to perform the face recognition method of any of the above embodiments.

[0279] This application also provides a computer program product that, when run on a computer, causes the computer to perform the aforementioned steps to implement the face recognition method described in the above embodiments.

[0280] In addition, embodiments of this application also provide an apparatus, which may specifically be a chip, component or module. The apparatus may include a connected processor and a memory; wherein the memory is used to store computer execution instructions, and when the apparatus is running, the processor may execute the computer execution instructions stored in the memory to cause the chip to execute the face recognition methods in the above method embodiments.

[0281] In this embodiment, the electronic device, computer-readable storage medium, computer program product or chip are all used to execute the corresponding methods provided above. Therefore, the beneficial effects that can be achieved can be referred to the beneficial effects of the corresponding methods provided above, and will not be repeated here.

[0282] Through the above description of the embodiments, those skilled in the art will understand that, for the sake of convenience and brevity, only the division of the above functional modules is used as an example. In actual applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above.

[0283] In the several embodiments provided in this application, it should be understood that the disclosed apparatus and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of modules or units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another device, or some features may be ignored or not executed. Furthermore, the mutual coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between devices or units may be electrical, mechanical, or other forms.

[0284] The units described as separate components may or may not be physically separate. A component shown as a unit can be one or more physical units; that is, it can be located in one place or distributed in multiple different locations. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0285] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0286] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a readable storage medium. Based on this understanding, the technical solutions of the embodiments of this application, in essence, or the parts that contribute to the prior art, or all or part of the technical solutions, can be embodied in the form of a software product. This software product is stored in a storage medium and includes several instructions to cause a device (which may be a microcontroller, chip, etc.) or processor to execute all or part of the steps of the methods of the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0287] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A face recognition method applied to an electronic device, the electronic device including a camera, characterized in that, The method includes: Receive the user's first operation, which is used to trigger face recognition; In response to the first operation, power is supplied to the camera; After powering on the camera, a configuration command is loaded, which is used to instruct the camera to load camera parameters; the time difference between the end time of powering on the camera and the start time of loading the configuration command is less than a preset time threshold. Load camera parameters according to the configuration command; Images are acquired based on the camera parameters; Face recognition is performed based on the image.

2. The method according to claim 1, characterized in that, The first operation is pressing the power button while the screen is off.

3. The method according to claim 1 or 2, characterized in that, The electronic device includes a first application located at the application layer and a camera hardware abstraction layer located at the hardware abstraction layer. The application layer includes an AIDL interface. The step of powering on the camera in response to the first operation includes: The first application sends a first instruction to the camera hardware abstraction layer through the AIDL interface, the first instruction being used to instruct the camera to power on; The camera hardware abstraction layer responds to the first instruction and controls the camera to power on.

4. The method according to claim 3, characterized in that, The control of powering on the camera includes: The camera hardware abstraction layer controls the camera to power on by calling the PreAcquireDevice() and AcquireDevice() functions.

5. The method according to claim 3 or 4, characterized in that, The electronic device also includes a face recognition control module located in the hardware abstraction layer, and a camera service located in the application framework layer. The loading configuration command includes: The first application sends a second instruction to the camera service through the face recognition control module, the second instruction being used to instruct face recognition; In response to the second instruction, the camera service sends a third instruction to the camera hardware abstraction layer, the third instruction being used to instruct the camera to power on. The camera hardware abstraction layer responds to the third instruction by loading the configuration command when it determines that the camera is powered on.

6. The method according to claim 5, characterized in that, The method further includes: The camera hardware abstraction layer responds to the first instruction by setting the value of the first state parameter to a first value; The step of loading the configuration command when it is determined that the camera is powered on includes: When the camera hardware abstraction layer determines that the value of the first state parameter is the first value, it loads the configuration command.

7. The method according to claim 6, characterized in that, The method further includes: In response to the first instruction, after the camera is powered on, the camera hardware abstraction layer initializes the camera configuration; When the camera hardware abstraction layer determines that the value of the first state parameter is the first value, it loads the configuration command, including: When the camera hardware abstraction layer determines that the value of the first state parameter is the first value and the initialization configuration is completed, it loads the configuration command.

8. The method according to claim 6 or 7, characterized in that, The method further includes: After loading the configuration command, the camera hardware abstraction layer sets the value of the first state parameter to an initial value.

9. The method according to any one of claims 5 to 8, characterized in that, The command to load the configuration includes: The camera hardware abstraction layer calls LoadSensorInitCfgCmd() to load the configuration command.

10. The method according to any one of claims 3 to 9, characterized in that, The first application is a lock screen application, a payment application, a shopping application, a chat application, or a banking application.

11. An electronic device, characterized in that, include: Processor, memory, and interface; The processor, the memory, and the interface cooperate with each other to enable the electronic device to perform the method as described in any one of claims 1 to 10.

12. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a processor, causes the processor to perform the method of any one of claims 1 to 10.

Images