Facial recognition method and electronic device

Abstract

The present application relates to the technical field of terminals. Disclosed are a facial recognition method and an electronic device, which are used for improving the user experience. The method comprises: in response to a first operation, an electronic device collecting a first facial image of a user; when the face in the first facial image is incomplete, the electronic device performing facial recognition on the facial image on the basis of a first template image, so as to obtain a facial recognition result; and on the basis of the facial recognition result, the electronic device executing a preset operation, wherein the first template image is a template image for incomplete faces.

Description

Face recognition method and electronic device

[0001] This application claims priority to the Chinese patent application filed with the State Intellectual Property Office on December 28, 2023, with application number 202311837485.5 and invention name “Face Recognition Method and Electronic Device”, the entire contents of which are incorporated by reference into this application. Technical Field

[0002] The present application relates to the field of terminal technology, and in particular to a face recognition method and electronic device. Background Art

[0003] Facial recognition technology refers to a method of identity authentication using facial images. With the rapid development of computer and network technologies, facial recognition technology has been widely used in many industries and fields, including smart access control, smart door locks, mobile terminals, public security, and entertainment.

[0004] Typically, when a device uses facial recognition technology to identify a facial image, it matches the captured facial image with a pre-stored facial image. If the captured facial image matches the pre-stored facial image, facial recognition is successful; if not, facial recognition fails. The facial image used for facial recognition is typically complete or free of obstructions. Therefore, if the captured facial image is incomplete, facial recognition may fail.

[0005] Summary of the Invention

[0006] In view of this, the present application provides a face recognition method and electronic device for improving user experience.

[0007] In a first aspect, the present application provides a face recognition method, comprising: an electronic device, in response to a first operation, collecting a first face image of a user. If the face in the first face image is incomplete, the electronic device performs face recognition on the first face image based on a first template image to obtain a face recognition result, and based on the face recognition result, the electronic device performs a preset operation. The first template image is a template image with an incomplete face. In this way, if the face in the collected face image is incomplete, the electronic device can use the incomplete template image to perform face recognition on the collected face image to obtain a face recognition result. Based on the face recognition result, the electronic device can perform a corresponding operation.

[0008] In one scenario, the first operation is an unlocking operation. Accordingly, when the face recognition result is a successful face recognition, the electronic device can perform the unlocking operation; when the face recognition result is a failed face recognition, the electronic device can save the lock screen state or output a prompt message of face recognition failure.

[0009] In one possible implementation of the first aspect, the electronic device may obtain a first template image from a storage module. The first template image is pre-obtained based on a second template image, and the second template image is a template image of a complete face. That is, the electronic device may pre-obtain an incomplete template image of a face and subsequently perform face recognition based on the incomplete template image, which is fast and convenient.

[0010] In another possible implementation manner of the first aspect, the electronic device may acquire the first template image based on a pre-stored second template image.

[0011] In one example, the first template image can be obtained by: blocking the second template image to obtain the first template image.

[0012] In another example, if the first facial image contains an incomplete face, the electronic device can determine the facial orientation in the first facial image and, based on the facial orientation in the first facial image, determine the missing region of the first facial image. Accordingly, the first template image can be obtained by occluding the facial portion corresponding to the missing region in the second template image to obtain the first template image.

[0013] Specifically, the electronic device may determine the facial part corresponding to the abnormal key point of the first facial image based on the facial direction in the first facial image. The abnormal key point is a key point that is not included in the area where the first facial image is located among multiple key points of the first facial image.

[0014] The height of the blocked facial portion in the first template image is the same as the height of the missing region in the first facial image.

[0015] In another possible implementation of the first aspect, the electronic device processes the feature vector of the second template image to obtain the feature vector of the first template image, and performs face recognition on the first face image based on the feature vector of the first template image to obtain a face recognition result.

[0016] In another possible implementation of the first aspect, before performing face recognition on the first facial image based on the first template image, the electronic device may perform face recognition on the first facial image based on the second template image. If the face recognition of the first facial image based on the second template image fails, the electronic device performs face recognition on the first facial image based on the first template image to obtain a face recognition result.

[0017] In another possible implementation of the first aspect, when the similarity between the first facial image and the second template image is less than or equal to a first threshold and greater than a second threshold, the electronic device determines that face recognition of the first facial image based on the second template image has failed, where the first threshold is greater than the second threshold.

[0018] In another possible implementation of the first aspect, when the similarity between the first facial image and the first template image is greater than a third threshold, the facial recognition result is successful facial recognition, wherein the third threshold is greater than the first threshold.

[0019] In another possible implementation of the first aspect, the electronic device may obtain multiple key points of the first facial image, and determine that the face in the first facial image is incomplete if there are abnormal key points among the multiple key points.

[0020] In a second aspect, the present application provides an electronic device, comprising a camera, a memory, a communication module, and one or more processors. The camera, the memory, the communication module, and the processor are coupled.

[0021] The camera is configured to capture a first facial image of a user. The communication module is configured to communicate with other devices. The memory is configured to store data in the electronic device. The memory is further configured to store computer program code, which includes computer instructions. When the processor executes the computer instructions, the electronic device executes the method performed by the electronic device in the first aspect and any possible design thereof.

[0022] In a third aspect, a computer-readable storage medium is provided, comprising a program code. When the program code is run on an electronic device, the electronic device executes any one of the methods provided in the first aspect.

[0023] In a fourth aspect, a computer program product is provided, comprising a program code, which, when executed on an electronic device, enables the electronic device to execute any one of the methods provided in the first aspect.

[0024] In a fifth aspect, an embodiment of the present application provides a chip system, which is applied to an electronic device including a display screen, a communication module, and a memory. The chip system includes one or more interface circuits and one or more processors. The interface circuit and the processor are interconnected by a line. The interface circuit is used to receive a signal from the memory of the electronic device and send the signal to the processor, where the signal includes a computer instruction stored in the memory. When the processor executes the computer instruction, the electronic device performs the method performed by the electronic device in the first aspect and any possible design thereof.

[0025] In a sixth aspect, embodiments of the present application provide a chip system, which is applied to a management device including a communication module and a memory. The chip system includes one or more interface circuits and one or more processors. The interface circuits and processors are interconnected via a circuit. The interface circuits receive signals and send them to the processors. The signals include computer instructions stored in the memory. When the processors execute the computer instructions, the electronic device performs the method described in the first aspect and any possible design thereof.

[0026] It can be understood that the beneficial effects that can be achieved by the electronic device described in the second aspect, the management device described in the third aspect and any possible design method thereof, the chip system described in the fourth aspect, the computer storage medium described in the fifth aspect, and the computer program product described in the sixth aspect provided above can be referred to the beneficial effects in the first aspect and any possible design method thereof, and will not be repeated here. BRIEF DESCRIPTION OF THE DRAWINGS

[0027] FIG1 is a schematic diagram of face recognition in an unlocking scenario provided by an embodiment of the present application;

[0028] FIG2 is a schematic diagram of a facial image captured by a user holding a mobile phone horizontally, according to an embodiment of the present application;

[0029] FIG3 is a schematic diagram of a facial image captured when the pitch angle of an electronic device is too large, provided by an embodiment of the present application;

[0030] FIG4 is a schematic diagram of face recognition based on a face recognition model provided in an embodiment of the present application;

[0031] FIG5 is a schematic diagram of the structure of an electronic device provided in an embodiment of the present application;

[0032] FIG6 is a flow chart of a face recognition method provided in an embodiment of the present application;

[0033] FIG7 is an example diagram of facial key points provided in an embodiment of the present application;

[0034] FIG8 is a schematic diagram of key points of a face in an incomplete face according to an embodiment of the present application;

[0035] FIG9a is a schematic diagram of a coordinate axis established based on a facial image according to an embodiment of the present application;

[0036] FIG9b is a schematic diagram of the coordinates of key points of a face in an incomplete face according to an embodiment of the present application;

[0037] FIG9c is a schematic diagram of the coordinates of facial key points in another embodiment of the present application in a case where the face is incomplete;

[0038] FIG10 is a schematic diagram of face recognition provided by an embodiment of the present application;

[0039] FIG11 is a schematic diagram of face alignment provided in an embodiment of the present application;

[0040] FIG12 is a flow chart of a face recognition method provided in an embodiment of the present application;

[0041] FIG13 is a schematic diagram of performing occlusion processing on a second template image to obtain a first template image according to an embodiment of the present application;

[0042] FIG14 is a flow chart of another face recognition method provided in an embodiment of the present application;

[0043] FIG15 is a schematic diagram of performing occlusion processing on a second template image according to an embodiment of the present application;

[0044] FIG16 is a schematic diagram of face recognition provided by an embodiment of the present application;

[0045] FIG17 is a flowchart of another face recognition method provided in an embodiment of the present application;

[0046] FIG18 is a schematic diagram of the verification effect of the technical solution provided in the embodiment of the present application;

[0047] FIG19 is a schematic diagram of the structural composition of a chip system provided in an embodiment of the present application. DETAILED DESCRIPTION

[0048] In the following, the terms "first" and "second" are used for descriptive purposes only and should not be understood to indicate or imply relative importance or implicitly specify the number of the technical features indicated. Therefore, a feature defined as "first" or "second" may explicitly or implicitly include one or more of the features. In the description of this embodiment, unless otherwise specified, "plurality" means two or more.

[0049] Generally, facial recognition technologies include two-dimensional (2D) and three-dimensional (3D) facial recognition technologies. Facial images used in 2D facial recognition technologies can include RGB images and always-on (AO) images. Facial images used in 3D facial recognition technologies can include infrared (IR) images and depth images.

[0050] A depth image is an image that uses the distance (depth) values ​​of each point in the scene as pixel values. This image can intuitively reflect the depth information of an object. Therefore, depth images are generally used for liveness detection in face recognition. IR images can be used for recognition in low-light and backlit scenes, and therefore, IR images can assist in face recognition. Of course, IR images can also be used in other algorithms, such as thermal imaging algorithms, without limitation.

[0051] In conjunction with the above-mentioned images, please refer to Figure 1, which shows a flowchart of face recognition provided by an embodiment of the present application. In Figure 1, after the electronic device acquires the face image, it can perform face detection (such as eye opening and closing detection, gaze detection, liveness detection, key point detection, etc.) on the face image, and perform face feature extraction to obtain a feature vector of the face image. After obtaining the feature vector of the face image, face recognition is performed. Among them, the face image acquired by the electronic device can be the above-mentioned AO image or depth image. For example, the face image can be an image taken by the electronic device in a backlit, dark or frontlit environment.

[0052] In one application scenario, the facial recognition described above is used in an unlocking scenario. In response to an unlocking operation, the electronic device captures a facial image and performs facial recognition on the facial image. If facial recognition is successful, the electronic device executes the unlocking operation.

[0053] Facial recognition can refer to calculating the similarity between the feature vector of a captured facial image and the feature vector of a template image. Facial recognition is successful if the similarity between the feature vector of the captured facial image and the template image is greater than a preset value. The template image can be a facial image pre-set by the electronic device, such as the unlock image shown in Figure 1. In other words, the electronic device can perform facial recognition based on the template image.

[0054] The image feature vector can be a multidimensional vector, for example, a facial image feature vector can be a 512x1 vector. For example, the electronic device can input the facial image into a feature vector extraction model to obtain the image feature vector. The feature vector extraction model has the function of extracting the image feature vector. Subsequent methods for obtaining the image feature vector can refer to this description.

[0055] However, in some scenarios, such as when a user is holding their phone for face recognition, the phone may not capture the entire face due to the phone's tilt angle being too large or the camera not directly facing the face. Incomplete faces in the face image may result in a low face recognition rate.

[0056] For example, when a user holds an electronic device horizontally for facial recognition, the captured facial image may be missing portions of the user's face because the device isn't facing the user's face. Please refer to Figure 2, which shows a facial image captured by the electronic device when the user holds the phone horizontally for facial recognition. As can be seen in Figure 2, the facial image captured by the electronic device is missing the mouth area.

[0057] For example, when the electronic device's pitch angle is too large, the captured facial image may be missing parts of the face because the electronic device's shooting angle cannot fully cover the face area. Please refer to Figure 3, which shows a facial image captured when the electronic device's pitch angle is too large. As can be seen from Figure 3, the facial image captured by the electronic device also lacks the image of the mouth area.

[0058] In combination with the above scenarios, in general, the template images used in face recognition are complete face images, so that face detection can be performed accurately. However, when the collected face image is incomplete, face recognition may fail.

[0059] A solution to the above problem provided by the related art is shown in FIG4 . The template image can be occluded to obtain multiple template images with incomplete faces. The template image and multiple template images with incomplete faces are then trained to obtain a face recognition model. Subsequently, the electronic device can perform face recognition based on the face recognition model. For example, when performing face recognition, if an incomplete face image is collected, such as a face image with a missing mouth area as shown in FIG4 , the face recognition model is input for face recognition. This can reduce the probability of face recognition failure and improve the recognition rate in scenarios with incomplete faces.

[0060] Among them, the occlusion areas and / or occlusion widths of any two images among the multiple incomplete face template images are different.

[0061] Although this related technology can improve the recognition rate in scenes with incomplete faces, it will also reduce the recognition rate in normal scenes.

[0062] In view of this, an embodiment of the present application provides a face recognition method, which is used to not only accurately perform face recognition when an incomplete face is detected in a face image to improve the accuracy of face recognition, but also will not reduce the recognition rate in normal scenarios, thereby improving the user experience.

[0063] For example, the face recognition method provided in the embodiments of the present application can be applied to electronic devices. The electronic devices can be mobile phones, tablet computers, laptops, wearable devices (such as smart watches), handheld computers, notebook computers, ultra-mobile personal computers (UMPCs), netbooks, as well as cellular phones, personal digital assistants (PDAs), augmented reality (AR) and virtual reality (VR) devices, and other devices with face recognition capabilities. The embodiments of the present application do not impose any particular restrictions on the specific form of the electronic devices.

[0064] In the embodiment of the present application, the electronic device is a mobile phone as an example to illustrate the structure of the electronic device provided in the embodiment of the present application. As shown in Figure 5, the mobile phone may include: a processor 510, an external memory interface 520, an internal memory, a universal serial bus (USB) interface 530, a charging management module 540, a power management module 541, a battery 542, an antenna 1, an antenna 2, a mobile communication module 550, a wireless communication module 560, an audio module 570, a speaker 570A, a receiver 570B, a microphone 570C, an earphone interface 570D, a sensor module 580, a button 590, a motor 591, an indicator 592, a camera 593, a display 594, and a subscriber identification module (SIM) card interface 595, etc.

[0065] Among them, the above-mentioned sensor module 580 may include sensors such as pressure sensor, gyroscope sensor, air pressure sensor, magnetic sensor, acceleration sensor, distance sensor, proximity light sensor, fingerprint sensor, temperature sensor, touch sensor, ambient light sensor and bone conduction sensor.

[0066] In the embodiment of the present application, the electronic device can capture a facial image through the camera 593 and perform facial recognition on the captured facial image. The electronic device can also capture a template image through the camera 593 for facial recognition, or the electronic device can obtain a template image from another device for facial recognition.

[0067] It should be understood that the structure illustrated in this embodiment does not constitute a specific limitation on the electronic device. In other embodiments, the electronic device may include more or fewer components than shown, or may combine or separate certain components, or have different component arrangements. The illustrated components may be implemented in hardware, software, or a combination of software and hardware.

[0068] The processor 510 may include one or more processing units. For example, the processor 510 may include an application processor (AP), a modem processor, a graphics processing unit (GPU), an image signal processor (ISP), a controller, a memory, a video codec, a digital signal processor (DSP), a baseband processor, and / or a neural-network processing unit (NPU). Different processing units may be independent devices or integrated into one or more processors.

[0069] The controller can be the nerve center and command center of the electronic device. The controller can generate operation control signals based on the instruction opcode and timing signals to complete the control of instruction fetching and execution.

[0070] The processor 510 may also be provided with a memory for storing instructions and data. In some embodiments, the memory in the processor 510 is a cache memory. This memory can store instructions or data that the processor 510 has just used or is reusing. If the processor 510 needs to use the instruction or data again, it can directly call it from the memory. This avoids repeated accesses, reduces the waiting time of the processor 510, and thus improves the efficiency of the system. In an embodiment of the present application, the processor 510 can be used to perform facial recognition on collected facial images.

[0071] In some embodiments, the processor 510 may include one or more interfaces. The interfaces may include an inter-integrated circuit (I2C) interface, an inter-integrated circuit sound (I2S) interface, a pulse code modulation (PCM) interface, a universal asynchronous receiver / transmitter (UART) interface, a mobile industry processor interface (MIPI), a general-purpose input / output (GPIO) interface, a subscriber identity module (SIM) interface, and / or a universal serial bus (USB) interface.

[0072] It is understood that the interface connection relationship between the modules illustrated in this embodiment is only for illustrative purposes and does not constitute a structural limitation on the electronic device. In other embodiments, the electronic device may also adopt different interface connection methods from the above embodiments, or a combination of multiple interface connection methods.

[0073] The charging management module 540 is configured to receive charging input from a charger. The charger can be either a wireless charger or a wired charger. In some wired charging embodiments, the charging management module 540 can receive charging input from the wired charger via the USB interface 530. In some wireless charging embodiments, the charging management module 540 can receive wireless charging input via the wireless charging coil of the mobile phone 500. While charging the battery 542, the charging management module 540 can also power the electronic device via the power management module 541.

[0074] The power management module 541 is used to connect the battery 542, the charging management module 540, and the processor 510. The power management module 541 receives input from the battery 542 and / or the charging management module 540 and provides power to the processor 510, internal memory, external memory, display 594, camera 593, and wireless communication module 560. The power management module 541 can also be used to monitor parameters such as battery capacity, battery cycle count, and battery health status (leakage, impedance). In some other embodiments, the power management module 541 can also be set in the processor 510. In other embodiments, the power management module 541 and the charging management module 540 can also be set in the same device.

[0075] The wireless communication function of the electronic device can be implemented through antenna 1, antenna 2, mobile communication module 550, wireless communication module 560, modem processor and baseband processor.

[0076] Antenna 1 and Antenna 2 are used to transmit and receive electromagnetic wave signals. Each antenna in an electronic device can be used to cover a single or multiple communication frequency bands. Different antennas can also be reused to improve antenna utilization. For example, antenna 1 can be reused as a diversity antenna for a wireless local area network. In other embodiments, the antennas can be used in conjunction with a tuning switch.

[0077] The mobile communication module 550 can provide wireless communication solutions for electronic devices, including 2G / 3G / 4G / 5G, etc. The mobile communication module 550 can include at least one filter, a switch, a power amplifier, a low noise amplifier (LNA), etc.

[0078] The antenna 1 of the electronic device is coupled to the mobile communication module 550. The mobile communication module 550 can receive electromagnetic waves from the antenna 1, filter, amplify, and perform other processing on the received electromagnetic waves, and transmit them to the modem processor for demodulation. The mobile communication module 550 can also amplify the signal modulated by the modem processor and convert it into electromagnetic waves for radiation via the antenna 1. In some embodiments, at least some of the functional modules of the mobile communication module 550 can be provided in the processor 510. In some embodiments, at least some of the functional modules of the mobile communication module 550 can be provided in the same device as at least some of the modules of the processor 510.

[0079] The wireless communication module 560 can provide wireless communication solutions for electronic devices, such as wireless local area networks (WLAN), Bluetooth (BT), global navigation satellite system (GNSS), frequency modulation (FM), near field communication (NFC), and infrared (IR). For example, the WLAN can be a wireless fidelity (Wi-Fi) network.

[0080] Antenna 2 of the electronic device is coupled to wireless communication module 560. Wireless communication module 560 can be one or more devices integrating at least one communication processing module. Wireless communication module 560 receives electromagnetic waves via antenna 2, frequency-modulates and filters the electromagnetic wave signals, and transmits the processed signals to processor 510. Wireless communication module 560 can also receive signals to be transmitted from processor 510, frequency-modulate and amplify them, and then convert them into electromagnetic waves for radiation via antenna 2.

[0081] The electronic device implements display functionality through a GPU, display screen 594, and an application processor. The GPU is a microprocessor for image processing that connects the display screen 594 and the application processor. The GPU is used to perform mathematical and geometric calculations for graphics rendering. The processor 510 may include one or more GPUs that execute program instructions to generate or modify display information. The display screen 594 may be a touchscreen, configured to display images, videos, and the like. The display screen 594 includes a display panel.

[0082] The electronic device can implement a camera function using an ISP, a camera 593, a video codec, a GPU, a display 594, and an application processor. The ISP is used to process data fed back by the camera 593. In some embodiments, the ISP can be installed in the camera 593. The camera 593 is used to capture still images or videos. In some embodiments, the electronic device can include one or N cameras 593, where N is a positive integer greater than 1.

[0083] The external memory interface 520 can be used to connect an external memory card, such as a Micro SD card, to expand the storage capacity of the electronic device. The external memory card communicates with the processor 510 via the external memory interface 520 to implement data storage functions. For example, files such as music and videos can be stored on the external memory card.

[0084] The internal memory can be used to store computer executable program code, which includes instructions. The processor 510 executes various functional applications and data processing of the electronic device by running the instructions stored in the internal memory. For example, in an embodiment of the present application, the processor 510 can execute instructions stored in the internal memory, and the internal memory can include a program storage area and a data storage area.

[0085] The program storage area can store an operating system and at least one application required for a function (such as a sound playback function or an image playback function). The data storage area can store data created during the use of the electronic device (such as audio data, a phone book, etc.). In addition, the internal memory can include high-speed random access memory and non-volatile memory, such as at least one disk storage device, a flash memory device, or universal flash storage (UFS).

[0086] The electronic device can implement audio functions such as music playback and recording through the audio module 570, the speaker 570A, the receiver 570B, the microphone 570C, the headphone jack 570D, and the application processor.

[0087] Audio module 570 is used to convert digital audio information into analog audio signal output and also to convert analog audio input into digital audio signals. Headphone jack 570D is used to connect wired headphones. Headphone jack 570D can be USB interface 530 or a 3.5mm Open Mobile Terminal Platform (OMTP) standard interface or a Cellular Telecommunications Industry Association of the USA (CTIA) standard interface.

[0088] The buttons 590 include a power button, a volume button, and the like. The buttons 590 may be mechanical buttons. They may also be touch buttons. The motor 591 may generate a vibration prompt. The motor 591 may be used for incoming call vibration prompts or for touch vibration feedback. The indicator 592 may be an indicator light that may be used to indicate the charging status, power changes, messages, missed calls, notifications, and the like. The SIM card interface 595 is used to connect a SIM card. The SIM card may be inserted into or removed from the SIM card interface 595 to achieve contact and separation with the electronic device. The electronic device may support 1 or N SIM card interfaces, where N is a positive integer greater than 1. The SIM card interface 5795 may support Nano SIM cards, Micro SIM cards, SIM cards, and the like.

[0089] The methods in the following embodiments can all be implemented in an electronic device having the above hardware structure.

[0090] An embodiment of the present application provides a face recognition method, as shown in FIG6 , which may include the following steps.

[0091] S601: In response to a first operation, the electronic device collects a first facial image of a user.

[0092] The first facial image refers to an image that needs to be recognized. The first operation may refer to an operation that can trigger the electronic device to capture a facial image. For example, the first operation may be an unlocking operation or an operation to open an access control device.

[0093] In one example, an electronic device may activate a camera to capture a picture when a predetermined condition is met, such as in response to a trigger operation or detecting a user approaching. The camera may be a front-facing camera or a rear-facing camera of the electronic device. The trigger operation may be pressing a button (such as a power button) of the electronic device or touching the display screen of the electronic device.

[0094] In one scenario, for example, the electronic device is a mobile phone or tablet computer, and the electronic device is in a locked screen state. The electronic device can respond to an unlocking operation (e.g., pressing a power button or long pressing a finger on the display screen) by activating the front camera to capture a user's facial image (i.e., a first facial image). The facial image can be used by the electronic device to determine whether to allow the electronic device to be unlocked.

[0095] In another scenario, for example, the electronic device is a fixed-position facial recognition device (such as an access control device). In response to a user approaching or detecting an ID, the electronic device can activate a camera to capture and obtain a first facial image.

[0096] The electronic device can detect the distance between itself and the user using infrared light. When the electronic device detects that the distance between itself and the user is less than a preset distance, it can control the camera to capture the image. The preset distance can be set as needed, for example, 50 centimeters (cm), 100 cm, etc., without limitation.

[0097] In some examples, the access control device may be an access control device in a public place such as an airport or a train station. The certificate may be a user's identification document. The user's identification document may store relevant information about the user, such as an identification number, a facial image, and the like.

[0098] S602: When the face in the first facial image is incomplete, the electronic device performs facial recognition on the first facial image based on the first template image to obtain a facial recognition result.

[0099] Among them, incomplete face means that the face in the face image lacks some features or is missing some features. The lack of some features of the face may be caused by the incomplete collected face image (for example, the face image can be as shown in Figure 2 or Figure 3 above), or it may be caused by the collected face image being complete but the face is blocked (for example, the face is wearing a mask or sunglasses). The first template image is a template image with an incomplete face. The first template image can be an image pre-configured for the electronic device, or an image obtained by the electronic device from other devices. For specific implementation, please refer to the explanation of related content in the subsequent section, which will not be elaborated here.

[0100] In one possible implementation, the electronic device may determine whether the first facial image is an incomplete face image based on multiple key points of the face.

[0101] In one example, the electronic device may input the first facial image into a facial key point detection model to obtain coordinates of multiple key points of the first facial image. The facial key point detection model has the function of identifying key points of the facial image and the coordinates of the key points.

[0102] Among them, the key points of a face can be used to identify multiple parts of the face. Please refer to Figure 7, which shows multiple key points of a complete face. The multiple key points can include five key points (black origins in the figure), such as the left eye, right eye, nose, left corner of the mouth, and right corner of the mouth.

[0103] It should be noted that in the embodiments of the present application, when the face in the face image is incomplete, the facial key point detection model can also output the above-mentioned multiple key points of the face and the coordinates of the key points. Key points with abnormal coordinates among the multiple key points can be referred to as abnormal key points in this embodiment.

[0104] For example, if the incomplete face image in Figure 3 (the image without glasses) is input into the facial key point detection model, the obtained facial key points can be shown in Figure 8. As shown in Figure 8, the key points of the mouth corners of the face are not included in the area where the facial image is located.

[0105] In one example, based on the multiple key points of the face image to be identified and the coordinates of the key points output by the above-mentioned face detection model, the electronic device can detect whether there are abnormal key points among the multiple key points.

[0106] For example, the range of the horizontal coordinates of the points in the area where the facial image is located is [x1, x2], and the range of the vertical coordinates is [y1, y2]. x1 is the horizontal coordinate of the point on the left edge of the area where the facial image is located, x2 is the horizontal coordinate of the point on the right edge of the area where the facial image is located, y1 is the vertical coordinate of the point on the upper edge of the area where the facial image is located, and x2 is the vertical coordinate of the point on the lower edge of the area where the facial image is located. The width of the facial image w = x2-x1, and the height of the facial image h = y2-y1. It will be understood that in the embodiments of the present application, the facial image is a regular square image as an example for illustration, but is not limited to this.

[0107] Please refer to Figure 9a, which shows a coordinate system established based on a face image provided by an embodiment of the present application. The origin of the coordinate system is the lower left corner of the face image, the x-axis of the coordinate system is the lower edge of the face image, and the y-axis is the left edge of the face image. In Figure 9a, the range of the horizontal coordinates of the key points in the area where the face image is located is [0, w], and the range of the vertical coordinates of the key points is [0, h]. In other words, if the coordinates of the key points output by the face detection mode are included in this range, the key points can be considered to be abnormal key points. For example, abnormal key points may include any one or more of the following: key points whose horizontal coordinates are less than 0, horizontal coordinates are greater than w, vertical coordinates are less than 0, or vertical coordinates are greater than h.

[0108] In one scenario, as shown in FIG9b , the face in the face image is in a straight-on posture. In combination with the coordinate system shown in FIG9a and the position of the face image in the coordinate system, when the face in the face image lacks mouth features (the dotted line portion in the figure indicates the missing portion), the vertical coordinate w1 of the key point of the corner of the mouth is less than 0, and the key point of the corner of the mouth is an abnormal key point. When the face in the face image lacks eye features of both eyes, the vertical coordinate h2 of the key point of the eye of the face image is greater than h, and the key point of the eye is an abnormal key point. When the face in the face image lacks left eye features, the horizontal coordinate w2 of the key point of the left eye of the face image is less than 0, and the key point of the left eye is an abnormal key point. When the face in the face image lacks right eye features, the horizontal coordinate w3 of the key point of the right eye of the face image is greater than w, and the key point of the right eye is an abnormal key point.

[0109] In another scenario, as shown in FIG9c , the face in the face image is in a straight-facing posture. Combined with the coordinate system shown in FIG9a and the position of the face image in the coordinate system. When the face in the face image lacks mouth features, the horizontal coordinate w4 of the corner of the mouth is less than 0, and the key point of the corner of the mouth is an abnormal key point. When the face in the face image lacks eye features of both eyes, the horizontal coordinate w5 of the key point of both eyes is greater than w, and the key point of the left eye and the key point of the right eye are both abnormal key points. When the face in the face image lacks right eye features, the vertical coordinate h3 of the key point of the right eye of the face image is greater than h, and the key point of the right eye is an abnormal key point. When the face in the face image lacks left eye features, the vertical coordinate of the key point of the left eye is less than 0, and the key point of the left eye is an abnormal key point.

[0110] If there are abnormal key points among the multiple key points of the first facial image, the electronic device can determine that the face in the first facial image is incomplete. Correspondingly, if there are no abnormal key points among the multiple key points of the first facial image, the electronic device can determine that the face in the first facial image is complete.

[0111] In one possible implementation, if it is determined that the first facial image contains an incomplete face, the electronic device may perform facial recognition on the first facial image based on the first template image to obtain a facial recognition result. The specific facial recognition process can be referred to the relevant description in the following embodiments and will not be detailed here.

[0112] In one example, the electronic device may perform face recognition on the first face image based on the feature vector of the first template image to obtain a face recognition result.

[0113] For example, if the similarity between the feature vector of the first template image and the feature vector of the first face image is greater than a preset threshold, face recognition is successful; if the similarity between the feature vector of the first template image and the feature vector of the first face image is less than or equal to the preset threshold, face recognition fails.

[0114] In a possible scenario, after acquiring the first template image, the electronic device may acquire a feature vector of the first template image, and perform face recognition on the first face image based on the feature vector of the first template image.

[0115] In another possible scenario, after acquiring the feature vector of the second template image, the electronic device may obtain the feature vector of the first template image based on the feature vector of the second template image, and perform face recognition on the first face image based on the feature vector of the first template image.

[0116] In some further embodiments, referring to FIG10 , when the electronic device determines that the face in the first facial image is incomplete, it can input the position where the mask can be added into the embedding vector of the second template image based on the embedding vector to obtain the embedding vector of the first template image. The electronic device can perform face recognition on the first facial image based on the embedding vector of the first template image. The embedding vector of the image can be a multidimensional vector of the image, i.e., a feature vector. The embedding vector can refer to the existing technology and will not be described in detail.

[0117] S603: Based on the face recognition result, the electronic device performs a preset operation.

[0118] The preset operation may match the first operation. For example, if the first operation is an unlock operation, the preset operation may be unlocking or prompting an unlock failure. For another example, if the first operation is an access control device opening operation, the preset operation may be opening the access control device or prompting an unlock failure.

[0119] In one scenario, the electronic device is a mobile phone, tablet computer, or other device, and the electronic device is in a locked screen state. In response to an unlocking operation, the electronic device can use a camera to capture a facial image of the user. If the captured facial image does not contain a complete face, the electronic device can perform facial recognition on the captured facial image based on the first facial image to obtain a facial recognition result. If facial recognition is successful, the electronic device is unlocked. In this way, the user can use the functions of the electronic device normally. If facial recognition fails, the electronic device can continue to be in a locked screen state, or the electronic device can continue to be in a locked screen state and output a prompt message indicating that facial recognition failed.

[0120] In another scenario, the application of the electronic device is in a non-logged-in state. In response to a click to log in to the application, the electronic device can use a camera to capture a facial image of the user. In the case where the captured facial image is incomplete, the electronic device can perform facial recognition on the captured facial image based on the first facial image to obtain a facial recognition result. If the facial recognition is successful, the application is logged in. In this way, the user can use the functions of the application normally. If the facial recognition fails, the electronic device can output a prompt message indicating that the login failed. The application can be a multimedia application (such as a banking client, an instant messaging application, etc.).

[0121] Continuing with this scenario, after the electronic device logs into the application, the electronic device can respond to the user's operation (such as transfer operation, operation to modify user information, password modification, etc.), continue to capture the user's facial image, and when the face in the captured facial image is incomplete, perform facial recognition on the captured facial image based on the first facial image to obtain a facial recognition result. If the facial recognition is successful, the electronic device can perform the operation (such as transfer, modification of user information, modification of password). If the facial recognition fails, the electronic device may not perform the operation, or the electronic device may not perform the operation and output a prompt message of execution failure (such as prompt of transfer failure, prompt of user information modification failure, and password modification failure).

[0122] In another scenario, the electronic device is an access control device. Upon detecting the user's approach or an identification document, the electronic device can use a camera to capture a facial image of the user. If the captured facial image is incomplete, facial recognition is performed on the captured facial image based on the first facial image to obtain a facial recognition result. If facial recognition is successful, the access control is unlocked. In this way, the user can pass normally. If facial recognition fails, the electronic device can continue to close the access control, or it can continue to close the access control and output a prompt message indicating that facial recognition failed.

[0123] Based on the technical solution of Figure 6, after the electronic device acquires a facial image to be recognized, if it detects that the face in the facial image to be recognized is incomplete, the electronic device can perform facial recognition on the facial image to be recognized based on the occluded template image, and perform a preset operation if the occluded template image matches the facial image to be recognized. In this way, even if the face in the facial image captured by the electronic device is incomplete, the electronic device can still accurately perform facial recognition on the facial image, and at the same time, the recognition rate of faces in normal scenarios is not applied, thereby improving the user experience.

[0124] In some embodiments, the electronic device may be pre-configured with the first template image or obtain the first template image from another device.

[0125] In one scenario, the electronic device is a device such as a mobile phone or a tablet computer. The electronic device may pre-store a first template image. The first template image may be an image entered by the user when setting up the electronic device. For example, when setting the initial password for the lock screen, the electronic device may capture a facial image of the user and detect the captured facial image. When it is determined that the face in the captured facial image is complete, the facial image is stored in the electronic device as a second template image. In addition, the electronic device may also obtain the first template image based on the second template image. Of course, the facial integrity judgment may also be performed before the facial image is captured, which is not limited in this embodiment.

[0126] In one possible implementation, the electronic device can perform key point detection on a captured facial image. For example, the electronic device can input the captured facial image into a facial key point detection model to obtain multiple key points of the face. It is understood that if all of the multiple key points are located in the area of ​​the facial image, the face in the facial image is complete; otherwise, the face is incomplete.

[0127] In another scenario, the electronic device is a fixed-position facial recognition device. The electronic device can obtain the second template image from another device, or read the second template image from an ID card, and obtain the first template image based on the second template image.

[0128] For example, when the electronic device is a door access control device of a residential complex, the electronic device obtains a second template image of the user from the management device. The second template image may be the second template image of the user captured and stored by the management device via a camera in response to a capture operation when the user checks in.

[0129] For another example, when the electronic device is an access control device in a public place such as an airport or a station, the electronic device can read the second template image from a chip that detects a document, such as an identity card.

[0130] Furthermore, in an embodiment of the present application, if the face in the facial image is in a tilted posture, the electronic device or management device can align the facial image. Please refer to Figure 11, which shows an example diagram of aligning a facial image. As can be seen from Figure 11, the face in the facial image after alignment is in a straight-on state. This facilitates key point extraction and face recognition. Alignment processing can refer to aligning the face in the facial image based on a face detection frame. For details, please refer to the introduction of the corresponding content in the relevant technology and will not be repeated here.

[0131] The facial image may be a facial image collected when acquiring the second template image, or may be a facial image collected when performing face recognition.

[0132] Based on this embodiment, the electronic device may pre-configure a second template image or obtain the second template image from another device for subsequent face recognition.

[0133] Taking the first template image obtained by pre-processing the second template image by the electronic device as an example, the embodiment of the present application is described. Please refer to Figure 12, which shows another face recognition method provided by the embodiment of the present application, including S1201 to S1205.

[0134] S1201: The electronic device obtains a second template image.

[0135] The method for obtaining the second template image may refer to the relevant description in the above embodiment and will not be repeated here.

[0136] S1202: The electronic device performs occlusion processing on the second template image to obtain a first template image.

[0137] Among them, performing occlusion processing on the second template image may refer to adding a mask to the second template image or removing part of the image in the second template image. The first template image may be obtained by the electronic device by randomly occluding the face in the second template image, or by occluding key parts of the face in the second template image. The number of first template images may be one or more. When there are multiple first template images, the electronic device may adjust the position of the random occlusion in the first template image, or obtain the image by occluding multiple different key parts. For example, the mouth may be occluded, the eyes may be occluded, the left half of the face may be occluded, the right half of the face may be occluded, etc.

[0138] Please refer to Figure 13, which shows a first template image. In Figure 13, the first template image is a plurality of images. The occluded portions of the plurality of images are different.

[0139] S1203: In response to the first operation, the electronic device obtains a first facial image.

[0140] S1204: When the face in the first facial image is incomplete, the electronic device performs facial recognition on the first facial image based on the first template image to obtain a facial recognition result.

[0141] S1205: Based on the face recognition result, the electronic device performs a preset operation.

[0142] Among them, S1203-S1205 can refer to the description of the corresponding content in the above S601-S603, which will not be repeated here.

[0143] In one example, in S1202 above, there are multiple first template images. The electronic device may calculate the similarity between the first facial image and each first template image to obtain multiple similarities. If there is a similarity greater than a preset threshold among the multiple similarities, or the number of similarities greater than the preset threshold among the multiple similarities exceeds a preset number, the electronic device may determine that the face recognition is successful. If the multiple similarities are all less than or equal to the preset threshold, or the number of similarities greater than the preset threshold among the multiple similarities does not exceed a preset number, the electronic device may determine that the face recognition has failed.

[0144] In another example, in S1202 above, the number of first template images is one. The electronic device may directly calculate the similarity between the first facial image and the first template image. If the similarity is greater than a preset threshold, the electronic device may determine that facial recognition is successful. If the similarity is less than or equal to the preset threshold, the electronic device may determine that facial recognition has failed.

[0145] Based on the technical solution shown in Figure 12, the electronic device can pre-occlude the second template image to obtain the first template image. In this way, after the electronic device obtains an incomplete face image to be recognized, it can directly use the first template image to perform face recognition on the obtained face image, which is fast and convenient.

[0146] The embodiment of the present application is described by taking as an example a first template image obtained by performing occlusion processing on a second template image when the electronic device determines that the face in the first face image is incomplete. Please refer to FIG14 , which shows another face recognition method provided by an embodiment of the present application, including S1401 to S1403.

[0147] S1401. In response to a first operation, the electronic device obtains a first facial image.

[0148] Among them, S1401 can refer to the relevant description in the above S601, and will not be repeated here.

[0149] S1402: When it is determined that the face in the first face image is incomplete, obtain a second template image, and perform occlusion processing on the second template image to obtain a first template image.

[0150] The method for determining whether the face in the first facial image is incomplete may refer to the description of S602 above, which will not be repeated here.

[0151] In a possible implementation, the electronic device may perform occlusion processing on the second template image according to the missing area of ​​the first facial image to obtain the first template image.

[0152] In one example, the electronic device may detect the first facial image, determine a missing area of ​​the first facial image, and block or delete the missing area in the second template image to obtain the first template image.

[0153] It should be pointed out that if the second template image also includes other parts besides the face (such as the neck, hair, etc.), the electronic device can detect the second template image based on the face detection frame, determine the face image in the second template image, and block the obtained face image or delete some areas to obtain the first template image.

[0154] In one possible implementation, the electronic device can determine the missing region in the first facial image based on the abnormal key point among the multiple key points in the first facial image. It is understandable that the facial part corresponding to the abnormal key point is the missing region in the first facial image.

[0155] For example, the electronic device can determine the facial part corresponding to the abnormal key point of the first facial image based on the facial direction of the first facial image, and determine the missing area in the first facial image based on the facial part corresponding to the abnormal key point.

[0156] In an embodiment of the present application, the electronic device can determine the face direction of the first face image based on a face direction detection model, or the electronic device can also determine the face direction of the first face image based on the position of key points in the first face image. It is understandable that the positions of key points of different face images may be different, but the gaps between key points or key points of the same type are relatively close in the area where the entire face is located. Therefore, for a face image with an incomplete face, the electronic device can determine the face direction in the face image based on the position of normal key points among multiple key points of the face image in the face image. Specifically, reference can be made to the existing technology without limitation.

[0157] After determining that there is an abnormal key point among the multiple key points, the electronic device can determine the facial part of the first facial image corresponding to the abnormal key point based on the facial orientation in the first facial image. In this way, the electronic device can determine the missing area of ​​the first facial image based on the facial part of the abnormal key point.

[0158] The face direction may include upward, downward, leftward, and rightward. Upward means that the mouth of the face is below the eyes in the face image, downward means that the mouth of the face is above the eyes in the face image, leftward means that the right eye of the face is above the left eye in the face image, and rightward means that the left eye of the face is above the right eye in the face image.

[0159] It can be understood that the face direction in the first face direction is upward. If the abnormal key point is located above the face, the face part corresponding to the abnormal key point is the eye; if the abnormal key point is located below the face, the face part corresponding to the abnormal key point is the mouth.

[0160] The face direction in the first face direction is downward. If the abnormal keypoint is located above the face, the facial part corresponding to the abnormal keypoint is the mouth; if the abnormal keypoint is located below the face, the facial part corresponding to the abnormal keypoint is the eye.

[0161] The face direction in the first face direction is left. If the abnormal keypoint is on the left side of the face, the facial part corresponding to the abnormal keypoint is the mouth; if the abnormal keypoint is on the right side of the face, the facial part corresponding to the abnormal keypoint is the eye.

[0162] The face direction in the first face direction is rightward. If the abnormal keypoint is on the left side of the face, the facial part corresponding to the abnormal keypoint is the eye; if the abnormal keypoint is on the right side of the face, the facial part corresponding to the abnormal keypoint is the mouth.

[0163] Furthermore, after determining the facial part corresponding to the abnormal key point of the first facial image, the electronic device can determine the offset width of the face erasing in the first facial image based on the coordinates of the abnormal key point, and based on the offset width, perform occlusion processing on the second template image to obtain the first template image.

[0164] The offset width is the sum of the abnormal value in the coordinates of the abnormal key point and the distance from the abnormal key point to the corresponding edge. The width of the blocked area on the first template image is the same as the offset width.

[0165] For example, in combination with the first face image with the corners of the mouth missing in Figure 9b above, as shown in Figure 15, the electronic device can determine the offset width based on the coordinates of the key points of the corners of the mouth. For example, the coordinates of the key point of the left corner of the mouth are (x1, y1), and the distance between the key point of the left corner of the mouth and the lower edge of the face is a, then the offset width is equal to |y1|+a. In this way, the electronic device can block the lower edge area of ​​the second template image, and the width of the blocked area is equal to |y1|+a. Among them, the lower edge of the face can be the lower edge of the face in the virtual first face image.

[0166] Similarly, when the missing area of ​​the first facial image is the eye area, the offset width can be the difference between the vertical coordinate of the key point of the eye and the height of the first facial image, and the sum of the distance from the key point of the eye to the upper edge of the face. For example, the coordinates of the key point of the eye are (x2, y2), and the distance between the key point of the left eye and the upper edge of the face is b, then the offset width is equal to |y2-h|+b. Correspondingly, the width of the occluded area of ​​the first template image is equal to |y2-h|+b. The upper edge of the face can be the upper edge of the face in the virtual first facial image.

[0167] The electronic device may block the upper edge area of ​​the second template image to obtain the first template image. The width of the blocked area is the difference. It is understood that the first template image and the first facial image have the same missing features.

[0168] In conjunction with FIG. 11 , for a first facial image in which the face is in a tilted posture, the electronic device may perform alignment processing on the first facial image and then determine an offset width based on abnormal key points in the aligned first facial image, so as to perform occlusion processing on the second template image based on the determined offset width to obtain the first template image.

[0169] S1403: Perform facial recognition on the first facial image based on the first template image to obtain a facial recognition result, and based on the facial recognition result, the electronic device executes a preset operation.

[0170] Among them, S1403 can refer to the relevant descriptions in the above S602 and S603, and will not be repeated here.

[0171] Based on the technical solution in Figure 14, after the electronic device acquires the first facial image, it can determine the missing region of the first facial image and, based on the missing region, perform occlusion processing on the second template image to obtain a first template image that is close to the first facial image. In other words, if the occluded portion of the first template image is close to the missing region of the first facial image, then when the electronic device uses the first template image to perform facial recognition on the first facial image, the missing facial features are the same, resulting in a more accurate facial recognition result.

[0172] In some embodiments, please refer to Figure 16, the electronic device is pre-configured with a second template image and a first template image. After acquiring the first facial image, the electronic device may first calculate the similarity between the first facial image and the second template image. For example, the similarity between the feature vector of the first facial image and the feature vector of the second template image is calculated. If the similarity between the first facial image and the second template image is less than the first threshold but greater than the second threshold, the electronic device calculates the similarity between the first facial image and the first template image. If the similarity between the first facial image and the first template image is greater than the third threshold, the facial recognition is successful.

[0173] The first threshold is greater than the second threshold, and the first threshold is less than the third threshold.

[0174] Alternatively, the electronic device may calculate the similarity between the first facial image and the first template image and the second template image, respectively. If the similarity between the first facial image and the second template image is less than a first threshold but greater than a second threshold, and the similarity between the first facial image and the first template image is greater than a third threshold, then facial recognition is successful.

[0175] In some other embodiments, please refer to Figure 17, the face recognition method provided in the embodiment of the present application may include S1701 to S1708.

[0176] S1701. In response to a first operation, the electronic device collects a first facial image of a user.

[0177] Among them, S1701 can refer to the above-mentioned S601 and will not be described in detail here.

[0178] S1702: The electronic device performs face detection on the first face image.

[0179] Among them, face detection includes detecting the direction of the face, determining whether the face is complete, face recognition, etc.

[0180] In one example, if the face detection is face recognition, S1703 is executed; if the face detection is to determine whether the face is complete, S1704 is executed; if the face detection is to detect the direction of the face, S1705 is executed. Alternatively, the electronic device may directly execute S1706.

[0181] S1703: The electronic device calculates the similarity between the second template image and the first facial image.

[0182] The second template image is obtained from a user's face image. The second template image can be stored in a storage module. The storage module can be a storage module of an electronic device or a storage module of another device.

[0183] If the similarity is greater than the first threshold, the face recognition is successful; if the similarity is less than or equal to the first threshold and greater than the second threshold, execute S1704.

[0184] S1704: The electronic device determines whether the face in the first facial image is complete.

[0185] If the face in the first face image is complete, face recognition fails; if the face in the first face image is incomplete, step S1705 is executed.

[0186] S1705: The electronic device detects a face direction in the first face image.

[0187] Among them, S1705 can refer to the relevant description of the above embodiment and will not be repeated here.

[0188] S1706: The electronic device determines a missing area of ​​the first facial image, and determines an offset width based on the missing area of ​​the first facial image.

[0189] Among them, S1706 can refer to the relevant description of the above embodiment and will not be repeated here.

[0190] S1707: The electronic device performs occlusion processing on the second template image according to the offset width to obtain the first template image.

[0191] S1708: The electronic device calculates the similarity between the first template image and the first facial image.

[0192] If the similarity is greater than the third threshold, the face recognition is successful; if the similarity is less than or equal to the third threshold, the face recognition fails.

[0193] Based on the technical solution of FIG17 , after obtaining a user's facial image, the electronic device can detect the face and perform facial recognition based on a variety of detection methods. If the user's facial image contains an incomplete face, the electronic device can also perform detection based on an incomplete face template image. In this way, the technical solution of the embodiment of the present application can not only recognize incomplete facial images, but also maintain the recognition rate of complete facial images.

[0194] In some embodiments, please refer to Figure 18, which shows the results of verifying the technical solutions of the embodiments of the present application. As can be seen from Figure 18, compared with other face recognition technologies, the technical solutions of the embodiments of the present application have a relatively high matching score and can successfully recognize facial images missing the mouth and other incomplete facial images (such as those wearing hats).

[0195] An embodiment of the present application also provides a chip system, which can be applied to the above-mentioned electronic device or management device. As shown in Figure 19, the chip system 1900 includes at least one processor 1901 and at least one interface circuit 1902. The processor 1901 and the interface circuit 1902 can be interconnected via lines. For example, the interface circuit 1902 can be used to receive signals from other devices (such as the memory of an electronic device or a speaker). For another example, the interface circuit 1902 can be used to send signals to other devices (such as the processor 1901 of an electronic device or a speaker). Exemplarily, the interface circuit 1902 can read instructions stored in the memory and send the instructions to the processor 1901. Of course, the chip system can also include other discrete components, which is not specifically limited in the embodiment of the present application.

[0196] When the instructions are executed by the processor 1901, the electronic device may execute the steps of the electronic device in the above embodiment.

[0197] When the instructions are executed by the processor 1901, the electronic device may execute the steps of the electronic device in the above embodiment.

[0198] An embodiment of the present application also provides a computer storage medium, which includes computer instructions. When the computer instructions are executed on the above-mentioned electronic device, the electronic device executes the various functions or steps executed by the electronic device shown in Figure 6 in the above-mentioned method embodiment.

[0199] An embodiment of the present application further provides a computer program product. When the computer program product is run on a computer, the computer is enabled to execute the functions or steps executed by the electronic device in the above method embodiment.

[0200] An embodiment of the present application further provides a computer storage medium, which includes computer instructions. When the computer instructions are executed on the above-mentioned electronic device, the electronic device executes the various functions or steps executed by the target electronic device in the above-mentioned method embodiment.

[0201] An embodiment of the present application further provides a computer program product. When the computer program product is run on a computer, the computer is enabled to execute the functions or steps executed by the electronic device in the above method embodiment.

[0202] Other embodiments of the present application provide an electronic device that may include the aforementioned touch screen, a memory, and one or more processors. The touch screen, memory, and processor are coupled. The memory is used to store computer program code, which includes computer instructions. When the processor executes the computer instructions, the electronic device may perform the various functions or steps performed by the mobile phone in the aforementioned method embodiments. The structure of the electronic device may refer to the structure of mobile phone 500 shown in Figure 5.

[0203] Some other embodiments of the present application provide a display device that can be applied to an electronic device including the above-mentioned touch screen and is used to execute the functions or steps executed by the target electronic device in the above-mentioned method embodiment.

[0204] An embodiment of the present application further provides a computer storage medium, which includes computer instructions. When the computer instructions are executed on the above-mentioned electronic device, the electronic device executes the various functions or steps executed by the mobile phone in the above-mentioned method embodiment.

[0205] The embodiment of the present application further provides a computer program product, which, when executed on a computer, enables the computer to execute the functions or steps executed by the mobile phone in the above method embodiment.

[0206] Through the description of the above implementation methods, technical personnel in the relevant field can clearly understand that for the convenience and simplicity of description, only the division of the above-mentioned functional modules is used as an example. In actual applications, the above-mentioned functions can be assigned to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above.

[0207] In the several embodiments provided in this application, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are merely schematic. For example, the division of the modules or units is merely a logical function division. In actual implementation, there may be other division methods, such as multiple units or components can be combined or integrated into another device, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0208] The units described as separate components may or may not be physically separate, and the components shown as units may be one physical unit or multiple physical units, that is, they may be located in one place or distributed in multiple places. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0209] In addition, the functional units in the various embodiments of the present application may be integrated into a single processing unit, or each unit may exist physically separately, or two or more units may be integrated into a single unit. The aforementioned integrated units may be implemented in the form of hardware or software functional units.

[0210] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a readable storage medium. Based on this understanding, the technical solution of the embodiment of the present application is essentially or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product, which is stored in a storage medium and includes several instructions for enabling a device (which can be a single-chip microcomputer, chip, etc.) or a processor (processor) to execute all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes: various media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk.

[0211] The above content is only a specific embodiment of this application, but the scope of protection of this application is not limited to this. Any changes or replacements within the technical scope disclosed in this application should be included in the scope of protection of this application. Therefore, the scope of protection of this application should be based on the scope of protection of the claims.

Claims

1. A face recognition method, characterized in that, Applied to an electronic device, the method includes: In response to a first operation, collect a first face image of a user; In the case where the face in the first face image is incomplete, perform face recognition on the first face image based on a first template image to obtain a face recognition result; the first template image is a template image with an incomplete face; Execute a preset operation based on the face recognition result.

2. The method according to claim 1, wherein The method further includes: Obtain the first template image from a storage module, where the first template image is obtained in advance based on a second template image, and the second template image is a template image with a complete face.

3. The method according to claim 1, characterized in that The method further includes: Obtain the first template image based on a second template image stored in advance, where the second template image is a template image with a complete face.

4. The method according to claim 2 or 3, characterized in that, The first template image is obtained by: Occlude the second template image to obtain the first template image.

5. The method according to claim 2 or 3, characterized in that, In the case where the face in the first face image is incomplete, the method further includes: Determine the face direction in the first face image; Based on the face direction in the first face image, determine the missing area of the first face image; The first template image is obtained by: Perform an occlusion process on the face part in the second template image corresponding to the missing area to obtain the first template image.

6. The method according to claim 5, wherein The determining the missing area of the first face image based on the face direction in the first face image includes: Based on the face direction in the first face image, determine the face part corresponding to the abnormal key points of the first face image; the abnormal key points are the key points among the multiple key points of the first face image that are not included in the area where the first face image is located; Based on the face part corresponding to the abnormal key points, determine the missing area of the first face image.

7. The method according to claim 6, characterized in that, The height of the occluded face part in the first template image is the same as the height of the missing area.

8. The method according to claim 1, characterized in that, The performing face recognition on the first face image based on the first template image to obtain a face recognition result includes: Process the feature vector of the second template image to obtain the feature vector of the first template image, and perform face recognition on the first face image based on the feature vector of the first template image to obtain the face recognition result; the second template image is a template image with a complete face.

9. The method according to any one of claims 1-8, characterized in that, Before performing face recognition on the first face image based on the first template image, the method further includes: Perform face recognition on the first face image based on the second template image; The performing face recognition on the first face image based on the first template image in the case where the face in the first face image is incomplete to obtain a face recognition result includes: In the case where face recognition on the first face image based on the second template image fails, perform face recognition on the first face image based on the first template image to obtain the face recognition result.

10. The method according to claim 9, characterized in that, The method further includes: In the case where the similarity between the first face image and the second template image is less than or equal to a first threshold and greater than a second threshold, it is determined that the face recognition of the first face image based on the second template image fails; wherein, the first threshold is greater than the second threshold.

11. The method according to claim 10, characterized in that, In the case where the similarity between the first face image and the first template image is greater than a third threshold, the face recognition result is successful face recognition; wherein, the third threshold is greater than the first threshold.

12. The method according to any one of claims 1-11, characterized in that, The method further includes: obtaining a plurality of key points of the first face image; In the case where there are abnormal key points among the plurality of key points, it is determined that the face in the first face image is incomplete; wherein, the abnormal key points are not included in the area where the first face image is located.

13. An electronic device, characterized in that, The electronic device includes: a memory, a communication module, and one or more processors; the memory and the communication module are coupled to the processor; the communication module is used for communicating with other devices, and the memory is used for storing data in the electronic device; the memory is further used for storing computer program code, and the computer program code includes computer instructions, and when the computer instructions are executed by the processor, the electronic device executes the method according to any one of claims 1-12.

14. A chip system, characterized in that, The chip system is applied to an electronic device including a communication module and a memory; the chip system includes one or more interface circuits and one or more processors; the interface circuits and the processors are interconnected by lines; the interface circuits are used for receiving signals from the memory and sending the signals to the processors, and the signals include computer instructions stored in the memory; when the processors execute the computer instructions, the electronic device executes the method according to any one of claims 1-12.

15. A computer storage medium, characterized in that, including computer instructions, and when the computer instructions run on an electronic device, the electronic device executes the method according to any one of claims 1-12.

Images