Face recognition method and electronic device

Abstract

The application discloses a face recognition method and an electronic device, relates to the technical field of terminals, and is used for improving the experience of a user. The method comprises the following steps: in response to a first operation, the electronic device collects a first face image of the user; in the case that the face in the first face image is incomplete, the electronic device performs face recognition on the face image based on a first template image, obtains a face recognition result, and based on the face recognition result, the electronic device performs a preset operation. The first template image is a template image with an incomplete face.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of terminal, in particular to a face recognition method and electronic device BACKGROUND

[0002] Face recognition technology refers to a method of identity authentication through face image. With the rapid development of computer and network technology, face recognition technology has been widely applied to intelligent access control, intelligent door lock, mobile terminal, public security, entertainment and many other industries and fields.

[0003] Generally, when a device identifies a face image based on face recognition technology, the device can match the collected face image with a pre-stored face image. If the collected face image matches the pre-stored face image, the face recognition is successful; if not, the face recognition fails. In the face image used for face recognition, the face is generally a complete face or a face without occlusion. In this way, when the face in the collected face image is incomplete, the face recognition may fail. SUMMARY

[0004] Therefore, the present application provides a face recognition method and electronic device to improve user experience.

[0005] In a first aspect, the present application provides a face recognition method, including: an electronic device collects a first face image of a user in response to a first operation. In the case that the face in the first face image is incomplete, the electronic device performs face recognition on the first face image based on a first template image, obtains a face recognition result, and performs a preset operation based on the face recognition result. The first template image is a template image with incomplete face. In this way, in the case that the face in the collected face image is incomplete, the electronic device can use the template image with incomplete face to perform face recognition on the collected face image, and obtain a face recognition result. Based on the face recognition result, the electronic device can perform a corresponding operation.

[0006] In one scenario, the first operation is an unlocking operation. Correspondingly, in the case that the face recognition result is face recognition success, the electronic device can perform the unlocking operation; in the case that the face recognition result is face recognition failure, the electronic device can save a lock screen state or output a prompt information of face recognition failure.

[0007] In one possible implementation manner of the first aspect, the electronic device can obtain the first template image from a storage module, and the first template image is obtained in advance based on a second template image, and the second template image is a template image with complete face. That is, the electronic device can obtain the template image with incomplete face in advance, and subsequently, the electronic device can perform face recognition based on the template image with incomplete face, which is fast and convenient.

[0008] In a possible implementation of the first aspect, the electronic device can obtain the first template image based on a pre-stored second template image.

[0009] In an example, the first template image can be obtained by occluding the second template image to obtain the first template image.

[0010] In another example, in a case where the face in the first face image is incomplete, the electronic device can determine a face orientation of the first face image, and determine a missing area of the first face image based on the face orientation of the first face image. Correspondingly, the first template image can be obtained by occluding a face part in the second template image corresponding to the missing area to obtain the first template image.

[0011] Specifically, the electronic device can determine a face part corresponding to an abnormal key point of the first face image based on the face orientation of the first face image. The abnormal key point is a key point of the first face image that is not included in a region where the first face image is located.

[0012] The height of the occluded face part in the first template image is the same as the height of the missing area of the first face image.

[0013] In a possible implementation of the first aspect, the electronic device can process a feature vector of the second template image to obtain a feature vector of the first template image, and perform face recognition on the first face image based on the feature vector of the first template image to obtain a face recognition result.

[0014] In a possible implementation of the first aspect, before performing face recognition on the first face image based on the first template image, the electronic device can perform face recognition on the first face image based on the second template image, and in a case where the face recognition on the first face image based on the second template image fails, the electronic device performs face recognition on the first face image based on the first template image to obtain a face recognition result.

[0015] In a possible implementation of the first aspect, in a case where a similarity between the first face image and the second template image is less than or equal to a first threshold value and greater than a second threshold value, the electronic device determines that the face recognition on the first face image based on the second template image fails. The first threshold value is greater than the second threshold value.

[0016] In a possible implementation of the first aspect, in a case where a similarity between the first face image and the first template image is greater than a third threshold value, the face recognition result is that the face recognition succeeds, and the third threshold value is greater than the first threshold value.

[0017] In a possible implementation manner of the first aspect, the electronic device can acquire a plurality of key points of the first face image, and determine that the face in the first face image is incomplete in a case where there is an abnormal key point in the plurality of key points.

[0018] In a second aspect, the present application provides an electronic device, which includes a camera, a memory, a communication module and one or more processors. The camera, the memory, the communication module and the processor are coupled.

[0019] The camera is configured to acquire a first face image of a user. The communication module is configured to communicate with other devices. The memory is configured to store data in the electronic device. The memory is further configured to store computer program code, which includes computer instructions. When the computer instructions are executed by the processor, the electronic device performs the method as described in the first aspect and any possible implementation manner thereof.

[0020] In a third aspect, a computer readable storage medium is provided, which includes program code. When the program code is run on an electronic device, the electronic device performs any method provided in the first aspect.

[0021] In a fourth aspect, a computer program product is provided, which includes program code. When the program code is run on an electronic device, the electronic device performs any method provided in the first aspect.

[0022] In a fifth aspect, an embodiment of the present application provides a chip system, which is applied to an electronic device including a display screen, a communication module and a memory. The chip system includes one or more interface circuits and one or more processors. The interface circuit and the processor are interconnected through a circuit. The interface circuit is configured to receive a signal from the memory of the electronic device and send the signal to the processor. The signal includes computer instructions stored in the memory. When the processor executes the computer instructions, the electronic device performs the method as described in the first aspect and any possible implementation manner thereof.

[0023] In a sixth aspect, an embodiment of the present application provides a chip system, which is applied to a management device including a communication module and a memory. The chip system includes one or more interface circuits and one or more processors. The interface circuit and the processor are interconnected through a circuit. The interface circuit is configured to receive a signal and send the signal to the processor. The signal includes computer instructions stored in the memory. When the processor executes the computer instructions, the electronic device performs the method as described in the first aspect and any possible implementation manner thereof.

[0024] It can be understood that the electronic device provided in the second aspect, the management device provided in the third aspect and any possible design of the third aspect, the chip system provided in the fourth aspect, the computer storage medium provided in the fifth aspect, and the computer program product provided in the sixth aspect can achieve the beneficial effects as described in the first aspect and any possible design of the first aspect, which will not be repeated here. BRIEF DESCRIPTION OF DRAWINGS

[0025] Figure 1 A schematic diagram of face recognition in an unlocking scenario is provided for an embodiment of the present application;

[0026] Figure 2 A schematic diagram of a face image captured when a user holds a mobile phone horizontally is provided for an embodiment of the present application;

[0027] Figure 3 A schematic diagram of a face image captured when the elevation and depression angles of an electronic device are too large is provided for an embodiment of the present application;

[0028] Figure 4 A schematic diagram of face recognition based on a face recognition model is provided for an embodiment of the present application;

[0029] Figure 5 A schematic diagram of an electronic device structure is provided for an embodiment of the present application;

[0030] Figure 6 A flowchart of a face recognition method is provided for an embodiment of the present application;

[0031] Figure 7 An example diagram of face key points is provided for an embodiment of the present application;

[0032] Figure 8 A schematic diagram of face key points in a case of incomplete face is provided for an embodiment of the present application;

[0033] Figure 9a A schematic diagram of a coordinate axis established based on a face image is provided for an embodiment of the present application;

[0034] Figure 9b A schematic diagram of coordinates of face key points in a case of incomplete face is provided for an embodiment of the present application;

[0035] Figure 9c A schematic diagram of coordinates of face key points in another case of incomplete face is provided for an embodiment of the present application;

[0036] Figure 10 A schematic diagram of face recognition is provided for an embodiment of the present application;

[0037] Figure 11A face alignment schematic diagram provided by an embodiment of the present application;

[0038] Figure 12 A flowchart of a face recognition method provided by an embodiment of the present application;

[0039] Figure 13 A schematic diagram of performing occlusion processing on a second template image to obtain a first template image provided by an embodiment of the present application;

[0040] Figure 14 A flowchart of another face recognition method provided by an embodiment of the present application;

[0041] Figure 15 A schematic diagram of performing occlusion processing on a second template image provided by an embodiment of the present application;

[0042] Figure 16 A face recognition schematic diagram provided by an embodiment of the present application;

[0043] Figure 17 A flowchart of another face recognition method provided by an embodiment of the present application;

[0044] Figure 18 A schematic diagram of verification effect of a technical solution provided by an embodiment of the present application;

[0045] Figure 19 A structural composition schematic diagram of a chip system provided by an embodiment of the present application. DETAILED DESCRIPTION

[0046] Hereinafter, the terms "first" and "second" are only used for description purposes, and cannot be understood as indicating or implying relative importance or implicitly indicating the number of the indicated technical features. Therefore, the features defined with "first" and "second" can explicitly or implicitly include one or more of the features. In the description of the present embodiment, unless otherwise specified, the meaning of "a plurality of" is two or more.

[0047] Generally, the face recognition technology can include two-dimensional (2Dimension, 2D) face recognition technology and three-dimensional (3D) face recognition technology. The face image used by the 2D face recognition technology can include an RGB image, an Always-On (AO) image. The face image used by the 3D face recognition technology can include an infrared (infrared ray, IR) image, a depth (Depth) image.

[0048] The depth image is an image in which the distance (depth) value of each point in the collected scene is taken as a pixel value. The depth image can directly reflect the depth information of the object. Therefore, the depth image is generally used for live body recognition detection in face recognition. The IR image can be used for recognition in dark light and backlight scenes. Therefore, the IR image can assist face recognition. Of course, the IR image can also be used for other algorithms, such as thermal imaging algorithms, without limitation.

[0049] In combination with the above image, refer to Figure 1 which shows a flowchart of face recognition provided by an embodiment of the present application. Figure 1 In the above, after the electronic device obtains the face image, the electronic device can perform face detection (such as eye opening and closing detection, gaze detection, live body detection, key point detection, etc.) on the face image and perform face feature extraction to obtain a feature vector of the face image. After obtaining the feature vector of the face image, face recognition is performed. The face image obtained by the electronic device can be the above-mentioned AO image, depth image. For example, the face image can be an image obtained by the electronic device in a backlight, dark light or front light environment.

[0050] In one application scenario, the above-mentioned face recognition is applied to an unlocking scenario. The electronic device collects a face image in response to an unlocking operation and performs face recognition on the face image. In the case of successful face recognition, the electronic device performs the unlocking operation.

[0051] The face recognition can refer to calculating the similarity between the feature vector of the collected face image and the feature vector of the template image. In the case that the similarity between the feature vector of the collected face image and the template image is greater than a preset value, the face recognition is successful. The template image can be a face image pre-set by the electronic device, such as the unlocking image shown in Figure 1 That is, the electronic device can perform face recognition based on the template image.

[0052] The feature vector of the image can be a multi-dimensional vector, such as a 512x1 vector. For example, the electronic device can input the face image into a feature vector extraction model to obtain the feature vector of the image. The feature vector extraction model has the function of extracting the feature vector of the image. Subsequently, the method of obtaining the feature vector of the image can refer to this description.

[0053] However, in some scenarios, such as when the user holds the mobile phone to perform face recognition, if the angle of the mobile phone is too large or the shooting angle is not perpendicular to the face, it may cause the face captured by the mobile phone to be incomplete. In the case that the face in the face image is incomplete, it may cause the face recognition rate to be low.

[0054] For example, when a user holds an electronic device horizontally to perform face recognition, the electronic device cannot face the face directly, so that the captured face image can miss part of the face. Please refer to Figure 2 Fig. 1 shows a face image captured by an electronic device when a user holds the electronic device horizontally to perform face recognition. Figure 2 As shown in Fig. 1, the face image captured by the electronic device misses the image of the mouth region.

[0055] For another example, when the pitch angle of the electronic device is too large, the electronic device cannot cover the face region completely, so that the captured face image can also miss part of the face. Please refer to Figure 3 Fig. 2 shows a face image captured by an electronic device when the pitch angle of the electronic device is too large. Figure 3 As shown in Fig. 2, the face image captured by the electronic device also misses the image of the mouth region.

[0056] In combination with the above scenarios, generally, the template image used for face recognition is a complete face image, so that face detection can be accurately performed. However, when the captured face image is incomplete, face recognition can fail.

[0057] A solution provided by the related art to solve the above problem is shown in Fig. 3. As shown in Fig. 3, the template image can be subjected to occlusion processing to obtain multiple incomplete face template images, and the template image and the multiple incomplete face template images can be used for training to obtain a face recognition model. Subsequently, the electronic device can perform face recognition based on the face recognition model. For example, when a face image with an incomplete face is captured during face recognition, the face recognition model can be input to perform face recognition, so as to reduce the probability of face recognition failure and improve the recognition rate in the incomplete face scenario. Figure 4 Figure 4 As shown in Fig. 3, the face image missing the mouth region.

[0058] In the multiple incomplete face template images, the occlusion regions and / or the occlusion widths of any two images are different.

[0059] Although the related art can improve the recognition rate in the incomplete face scenario, the recognition rate in the normal scenario can be reduced.

[0060] In view of this, the embodiments of the present application provide a face recognition method, which can not only accurately perform face recognition to improve the accuracy of face recognition when detecting that a face in a face image is incomplete, but also will not reduce the recognition rate in the normal scenario, so as to improve the user experience.

[0061] ​For example, the face recognition method provided in this application can be applied to electronic devices. These electronic devices can be mobile phones, tablets, laptops, wearable devices (such as smartwatches), handheld computers, laptops, ultra-mobile personal computers (UMPCs), netbooks, as well as cellular phones, personal digital assistants (PDAs), augmented reality (AR) / virtual reality (VR) devices, and other devices with face recognition capabilities. This application does not impose any special limitations on the specific form of the electronic device.

[0062] This application uses a mobile phone as an example to illustrate the structure of the electronic device provided in this application embodiment. For example... Figure 5 As shown, a mobile phone may include: a processor 510, an external memory interface 520, an internal memory 521, a universal serial bus (USB) interface 530, a charging management module 540, a power management module 541, a battery 542, an antenna 1, an antenna 2, a mobile communication module 550, a wireless communication module 560, an audio module 570, a speaker 570A, a receiver 570B, a microphone 570C, a headphone jack 570D, a sensor module 580, buttons 590, a motor 591, an indicator 592, a camera 593, a display screen 594, and a subscriber identification module (SIM) card interface 595, etc.

[0063] The aforementioned sensor module 580 may include sensors such as pressure sensors, gyroscope sensors, barometric pressure sensors, magnetic sensors, accelerometers, distance sensors, proximity sensors, fingerprint sensors, temperature sensors, touch sensors, ambient light sensors, and bone conduction sensors.

[0064] In this embodiment, the electronic device can capture facial images using camera 593 and perform facial recognition on the captured images. Alternatively, the electronic device can also capture template images using camera 593 for facial recognition, or it can obtain template images from other devices for facial recognition.

[0065] It can be understood that the structure illustrated in the embodiment does not constitute a specific limitation on the electronic device. In other embodiments, the electronic device can include more or fewer components than those illustrated, or combine certain components, or split certain components, or different arrangement of components. The illustrated components can be implemented in hardware, software, or a combination of software and hardware.

[0066] The processor 510 can include one or more processing units, for example: the processor 510 can include an application processor (AP), a modem processor, a graphics processing unit (GPU), an image signal processor (ISP), a controller, a memory, a video codec, a digital signal processor (DSP), a baseband processor, and / or a neural-network processing unit (NPU), etc. Among them, different processing units can be independent devices, or can be integrated in one or more processors.

[0067] The controller can be the nerve center and command center of the electronic device. The controller can generate operation control signals according to instruction operation codes and timing signals, and complete the control of fetching and executing instructions.

[0068] The memory can also be provided in the processor 510, used to store instructions and data. In some embodiments, the memory in the processor 510 is a cache memory. The memory can save instructions or data that the processor 510 has just used or repeatedly uses. If the processor 510 needs to use the instructions or data again, it can be directly called from the memory. Avoiding repeated access, reducing the waiting time of the processor 510, thus improving the efficiency of the system. In the embodiments of the present application, the processor 510 can be used for face recognition on the collected face image.

[0069] In some embodiments, the processor 510 can include one or more interfaces. The interfaces can include an inter-integrated circuit (I2C) interface, an inter-integrated circuit sound (I2S) interface, a pulse code modulation (PCM) interface, a universal asynchronous receiver / transmitter (UART) interface, a mobile industry processor interface (MIPI), a general-purpose input / output (GPIO) interface, a subscriber identity module (SIM) interface, and / or a universal serial bus (USB) interface, etc.

[0070] It can be understood that the interface connection relationship between the modules shown in the embodiments is only illustrative and does not constitute a limitation on the structure of the electronic device. In other embodiments, the electronic device can also use different interface connection manners or combinations of multiple interface connection manners.

[0071] The charging management module 540 is configured to receive charging input from a charger. The charger can be a wireless charger or a wired charger. In some wired charging embodiments, the charging management module 540 can receive charging input from a wired charger through the USB interface 530. In some wireless charging embodiments, the charging management module 540 can receive wireless charging input through a wireless charging coil of the electronic device 500. The charging management module 540 can charge the battery 542 and also supply power to the electronic device through the power management module 541.

[0072] The power management module 541 is configured to connect the battery 542, the charging management module 540, and the processor 510. The power management module 541 receives input from the battery 542 and / or the charging management module 540 to supply power to the processor 510, the internal memory 521, the external memory, the display screen 594, the camera 593, and the wireless communication module 560, etc. The power management module 541 can also be configured to monitor parameters such as battery capacity, battery cycle count, battery health status (leakage, impedance), etc. In other embodiments, the power management module 541 can also be arranged in the processor 510. In other embodiments, the power management module 541 and the charging management module 540 can also be arranged in the same device.

[0073] The wireless communication function of the electronic device can be implemented by the antenna 1, the antenna 2, the mobile communication module 550, the wireless communication module 560, the modem processor, and the baseband processor, etc.

[0074] The antenna 1 and the antenna 2 are used for transmitting and receiving electromagnetic wave signals. Each antenna in the electronic device can be used to cover a single or multiple communication frequency bands. Different antennas can also be multiplexed to improve the utilization of the antennas. For example, the antenna 1 can be multiplexed as a diversity antenna of a wireless local area network. In some other embodiments, the antennas can be used in combination with a tuning switch.

[0075] The mobile communication module 550 can provide a solution for wireless communication including 2G / 3G / 4G / 5G, etc. applied on the electronic device. The mobile communication module 550 can include at least one filter, a switch, a power amplifier, a low noise amplifier (LNA), etc.

[0076] The antenna 1 of the electronic device is coupled with the mobile communication module 550. The mobile communication module 550 can receive electromagnetic waves by the antenna 1, and perform filtering, amplification, etc. on the received electromagnetic waves, and transmit the processed electromagnetic waves to the modem processor for demodulation. The mobile communication module 550 can also amplify the signals modulated by the modem processor, and convert the signals into electromagnetic waves radiated by the antenna 1. In some embodiments, at least part of the functional modules of the mobile communication module 550 can be arranged in the processor 510. In some embodiments, at least part of the functional modules of the mobile communication module 550 and at least part of the modules of the processor 510 can be arranged in the same device.

[0077] The wireless communication module 560 can provide a solution for wireless communication such as wireless local area networks (WLAN), bluetooth (BT), global navigation satellite system (GNSS), frequency modulation (FM), near field communication (NFC), infrared (IR) technology, etc. applied on the electronic device. For example, the above-mentioned WLAN can be a (wireless fidelity, Wi-Fi) network.

[0078] The antenna 2 of the electronic device is coupled with the wireless communication module 560. The wireless communication module 560 can be one or more devices that integrate at least one communication processing module. The wireless communication module 560 receives electromagnetic waves via the antenna 2, frequency-modulates and filters the electromagnetic wave signals, and transmits the processed signals to the processor 510. The wireless communication module 560 can also receive signals to be transmitted from the processor 510, frequency-modulate them, amplify them, and radiate them as electromagnetic waves via the antenna 2.

[0079] The electronic device can implement a display function through a GPU, a display 594, an application processor, and the like. The GPU is a microprocessor for image processing, which is connected to the display 594 and the application processor. The GPU is used to perform mathematical and geometric calculations for graphics rendering. The processor 510 can include one or more GPUs that execute program instructions to generate or change display information. The display 594 can be a touch screen, which is used to display images, videos, and the like. The display 594 includes a display panel.

[0080] The electronic device can implement a photographing function through an ISP, a camera 593, a video codec, a GPU, a display 594, an application processor, and the like. The ISP is used to process data fed back by the camera 593. In some embodiments, the ISP can be disposed in the camera 593. The camera 593 is used to capture still images or videos. In some embodiments, the electronic device can include one or N cameras 593, N being a positive integer greater than 1.

[0081] The external memory interface 520 can be used to connect an external memory card, such as a Micro SD card, to expand the storage capacity of the electronic device. The external memory card communicates with the processor 510 through the external memory interface 520 to implement a data storage function. For example, music, video, and the like files are saved in the external memory card.

[0082] The internal memory 521 can be used to store computer executable program codes, which include instructions. The processor 510 implements various function applications and data processing of the electronic device by running the instructions stored in the internal memory 521. For example, in the embodiments of the present application, the processor 510 can implement the instructions stored in the internal memory 521 to include a program storage area and a data storage area.

[0083] The storage program area can store an operating system, at least one application program required by a function (such as a sound playing function, an image playing function, etc.), and the like. The storage data area can store data (such as audio data, a phone book, etc.) created during use of the electronic device, and the like. In addition, the internal memory 521 can include a high-speed random access memory, and can further include a nonvolatile memory such as at least one of a magnetic disk storage device, a flash memory device, a universal flash storage (UFS), and the like.

[0084] The electronic device can implement an audio function through an audio module 570, a speaker 570A, a receiver 570B, a microphone 570C, an earphone interface 570D, an application processor, and the like. For example, music playing, recording, and the like.

[0085] The audio module 570 is configured to convert digital audio information into an analog audio signal output, and to convert an analog audio input into a digital audio signal. The earphone interface 570D is configured to connect a wired earphone. The earphone interface 570D can be a USB interface 530, or a 3.5 mm open mobile terminal platform (OMTP) standard interface, a cellular telecommunications industry association of the USA (CTIA) standard interface.

[0086] The keys 590 include a power key, a volume key, and the like. The keys 590 can be mechanical keys. Alternatively, the keys 590 can be touch keys. The motor 591 can generate a vibration prompt. The motor 591 can be used for incoming call vibration prompt, or can be used for touch vibration feedback. The indicator 592 can be an indicator light, and can be used for indicating a charging state, a power change, or can be used for indicating a message, a missed call, a notification, and the like. The SIM card interface 595 is configured to connect a SIM card. The SIM card can be inserted into or pulled out of the SIM card interface 595 to achieve contact and separation with the electronic device. The electronic device can support one or N SIM card interfaces, where N is a positive integer greater than 1. The SIM card interface 595 can support a Nano SIM card, a Micro SIM card, a SIM card, and the like.

[0087] The methods in the following embodiments can be implemented in the electronic device with the above hardware structure.

[0088] An embodiment of the present application provides a face recognition method, as shown in the method can include the following steps. Figure 6

[0089] ​S601, in response to a first operation, the electronic device collects a first face image of the user.

[0090] The first face image refers to an image that needs to be identified. The first operation can be an operation that can trigger the electronic device to collect a face image. For example, the first operation can be an unlocking operation or an operation of starting a door access device.

[0091] In an example, the electronic device can start the camera to take a picture when a predetermined condition is met, such as in response to a trigger operation / detection of the user approaching. The camera can be a front camera of the electronic device or can also be a rear camera of the electronic device. The trigger operation is an operation of pressing a key (such as a power key) of the electronic device or touching the display screen of the electronic device.

[0092] In one scenario, for example, the electronic device is a mobile phone or a tablet computer, and the electronic device is in a locked screen state. The electronic device can start the front camera to take a picture in response to an unlocking operation (such as an operation of pressing the power key or an operation of long-pressing the display screen), and obtain a face image (i.e., the first face image) of the user. The face image can be used by the electronic device to determine whether to allow unlocking the electronic device.

[0093] In another scenario, for example, the electronic device is a fixed-position face recognition device (such as a door access device). The electronic device can start the camera to take a picture in response to the user approaching or detecting a certificate, and obtain the first face image.

[0094] The electronic device can detect the distance between the user based on infrared rays. When the electronic device detects that the distance between the user is less than a preset distance, the camera can be controlled to take a picture. The preset distance can be set as needed, for example, 50 centimeters (cm), 100 cm, etc., without limitation.

[0095] In some examples, the door access device can be a door access device in a public place such as an airport or a station. The certificate can be an identity certificate of the user. The identity certificate of the user can store related information of the user, such as an identity card number, a face image, etc.

[0096] S602, in the case that the face in the first face image is not complete, the electronic device performs face recognition on the first face image based on the first template image to obtain a face recognition result.

[0097] The face that is not complete refers to a face in a face image that lacks some features or is missing some features. The face that lacks some features can be caused by the fact that the collected face image is not complete (for example, the face image can be as described above Figure 2 or Figure 3The face image can be a full face image or a face image with a missing face (e.g., a face wearing a mask or glasses). The first template image is a template image with a missing face. The first template image can be a preconfigured image of the electronic device or an image obtained by the electronic device from another device. For details, refer to the subsequent description.

[0098] In a possible implementation, the electronic device can determine whether the first face image is a face image with a missing face based on the plurality of key points of the face.

[0099] In an example, the electronic device can input the first face image into the face key point detection model to obtain coordinates of the plurality of key points of the first face image. The face key point detection model has a function of identifying key points of a face image and coordinates of the key points.

[0100] The key points of the face can be used to identify a plurality of parts of the face. For details, refer to Figure 7 which shows a plurality of key points of a full face. The plurality of key points can include five key points (black dots in the figure), i.e., a left eye, a right eye, a nose, a left corner of a mouth, and a right corner of a mouth.

[0101] It should be noted that in the embodiments of the present application, when the face image has a missing face, the face key point detection model can also output the plurality of key points of the face and the coordinates of the key points. In the embodiments, the key point with an abnormal coordinate in the plurality of key points can be referred to as an abnormal key point.

[0102] For example, inputting the face image with a missing face (an image without glasses) in Figure 3 into the face key point detection model, the obtained face key points can be as shown in Figure 8 It can be known from Figure 8 that the key point of the corner of the mouth of the face is not included in the area where the face image is located.

[0103] In an example, in combination with the plurality of key points of the face image to be identified and the coordinates of the key points output by the face detection model, the electronic device can detect whether there is an abnormal key point in the plurality of key points.

[0104] For example, the horizontal coordinate of a point in the region where the face image is located ranges from x1 to x2, and the vertical coordinate of the point ranges from y1 to y2. x1 is the horizontal coordinate of a point at the left edge in the region where the face image is located, x2 is the horizontal coordinate of a point at the right edge in the region where the face image is located, y1 is the vertical coordinate of a point at the upper edge in the region where the face image is located, and x2 is the vertical coordinate of a point at the lower edge in the region where the face image is located. The width of the face image is w = x2 - x1, and the height of the face image is h = y2 - y1. It can be understood that, in the embodiments of the present application, a regular square image of a face image is taken as an example for illustration, but the present application is not limited thereto.

[0105] Please refer to Figure 9a which shows a coordinate system established based on a face image according to an embodiment of the present application. The origin of the coordinate system is the lower left corner of the face image, the x-axis of the coordinate system is the lower edge of the face image, and the y-axis of the coordinate system is the left edge of the face image. Figure 9a In the coordinate system, the horizontal coordinate of a key point in the region where the face image is located ranges from 0 to w, and the vertical coordinate of the key point ranges from 0 to h. That is, if the coordinate of a key point output by the face detection mode is contained in the range, the key point can be considered as an abnormal key point. For example, the abnormal key point can include any one or more of the following: a key point with a horizontal coordinate less than 0, a key point with a horizontal coordinate greater than w, a key point with a vertical coordinate less than 0, or a key point with a vertical coordinate greater than h.

[0106] In one scenario, as shown in Figure 9b , the face in the face image is in a frontal pose. In combination with the coordinate system shown in Figure 9a and the position of the face image in the coordinate system, when the face in the face image lacks a mouth feature (the missing part is represented by a dashed line in the figure), the vertical coordinate w1 of the key point of the mouth corner is less than 0, and the key point of the mouth corner is an abnormal key point. When the face in the face image lacks eye features of both eyes, the vertical coordinate h2 of the key point of the eye of the face image is greater than h, and the key point of the eye is an abnormal key point. When the face in the face image lacks a left eye feature, the horizontal coordinate w2 of the key point of the left eye of the face image is less than 0, and the key point of the left eye is an abnormal key point. When the face in the face image lacks a right eye feature, the horizontal coordinate w3 of the key point of the right eye of the face image is greater than w, and the key point of the right eye is an abnormal key point.

[0107] In another scenario, as shown in Figure 9c , the face in the face image is in a frontal pose. In combination with the coordinate system shown in Figure 9aThe coordinate system and the position of the face image in the coordinate system are shown. When the face in the face image lacks the mouth feature, the horizontal coordinate w4 of the mouth corner is less than 0, and the key point of the mouth corner is an abnormal key point. When the face in the face image lacks the eye feature of the double eyes, the horizontal coordinate w5 of the key point of the double eyes is greater than w, and the key point of the left eye and the key point of the right eye are both abnormal key points. When the face in the face image lacks the right eye feature, the vertical coordinate h3 of the key point of the right eye of the face image is greater than h, and the key point of the right eye is an abnormal key point. When the face in the face image lacks the left eye feature, the vertical coordinate of the key point of the left eye is less than 0, and the key point of the left eye is an abnormal key point.

[0108] In a case where there is an abnormal key point in the plurality of key points of the first face image, the electronic device can determine that the face in the first face image is incomplete. Correspondingly, in a case where there is no abnormal key point in the plurality of key points of the first face image, the electronic device can determine that the face in the first face image is complete.

[0109] In a possible implementation, in a case where it is determined that the face in the first face image is incomplete, the electronic device can perform face recognition on the first face image based on the first template image, and obtain a face recognition result. The specific process of face recognition can refer to the related description in the following embodiments, and will not be repeated here.

[0110] In an example, the electronic device can perform face recognition on the first face image based on the feature vector of the first template image, and obtain a face recognition result.

[0111] For example, if the similarity between the feature vector of the first template image and the feature vector of the first face image is greater than a preset threshold, the face recognition is successful; if the similarity between the feature vector of the first template image and the feature vector of the first face image is less than or equal to the preset threshold, the face recognition fails.

[0112] In a possible scenario, the electronic device can obtain the feature vector of the first template image after obtaining the first template image, and perform face recognition on the first face image based on the feature vector of the first template image.

[0113] In another possible scenario, the electronic device can obtain the feature vector of the first template image based on the feature vector of the second template image after obtaining the feature vector of the second template image, and perform face recognition on the first face image based on the feature vector of the first template image.

[0114] In some other embodiments, please refer to Figure 10In a case where the electronic device determines that the face in the first face image is not complete, the electronic device can input a position where a mask can be added in the embedding vector of the second template image based on the embedding vector, to obtain an embedding vector of the first template image. The electronic device can perform face recognition on the first face image based on the embedding vector of the first template image. The embedding vector of the image can be a multi-dimensional vector of the image, i.e., a feature vector. The embedding vector can refer to the prior art and will not be described here.

[0115] S603, based on the face recognition result, the electronic device performs a preset operation.

[0116] The preset operation can match the first operation. For example, the first operation is an unlocking operation, and the preset operation can be unlocking or prompting unlocking failure. For another example, the first operation is an operation of opening an access control device, and the preset operation can be opening the access control device or prompting that the access control device cannot be opened.

[0117] In one scenario, the electronic device is a mobile phone, a tablet computer, or the like, and the electronic device is in a locked screen state. The electronic device can capture a face image of a user by using a camera in response to an unlocking operation. In a case where the face in the captured face image is not complete, the electronic device can perform face recognition on the captured face image based on the first face image to obtain a face recognition result. In a case where the face recognition is successful, the electronic device is unlocked. In this way, the user can normally use the functions of the electronic device. In a case where the face recognition fails, the electronic device can continue to be in the locked screen state, or the electronic device can continue to be in the locked screen state and output a prompt information that the face recognition fails.

[0118] In another scenario, an application of the electronic device is in an unlogged state. The electronic device can capture a face image of a user by using a camera in response to an operation of clicking to log in to the application. In a case where the face in the captured face image is not complete, the electronic device can perform face recognition on the captured face image based on the first face image to obtain a face recognition result. In a case where the face recognition is successful, the application is logged in. In this way, the user can normally use the functions of the application. In a case where the face recognition fails, the electronic device can output a prompt information that the logging in fails. The application can be a multimedia application (such as a bank client, an instant chat application, or the like).

[0119] Continuing with the scenario, after the electronic device logs into the application, the electronic device can continue to capture the facial image of the user in response to the operation of the user (e.g., a transfer operation, an operation of modifying user information, a password modification, etc.), and in the case that the face in the captured facial image is not complete, perform face recognition on the captured facial image based on the first facial image to obtain a face recognition result. In the case that the face recognition is successful, the electronic device can perform the operation (e.g., transfer, modify user information, modify password). In the case that the face recognition fails, the electronic device can not perform the operation, or the electronic device does not perform the operation and outputs a prompt information (e.g., prompt transfer failure, prompt user information modification failure, password modification failure) that the operation fails.

[0120] In another scenario, the electronic device is a door access device. The electronic device can capture the facial image of the user using the camera when detecting that the user is close or detecting an identity card. In the case that the face in the captured facial image is not complete, perform face recognition on the captured facial image based on the first facial image to obtain a face recognition result. In the case that the face recognition is successful, open the door access. In this way, the user can pass normally. In the case that the face recognition fails, the electronic device can continue to close the door access, or the electronic device can also continue to close the door access and output a prompt information that the face recognition fails.

[0121] Based on the technical solutions of the above Figure 6 After the electronic device obtains the facial image to be recognized, if it is detected that the face in the facial image to be recognized is not complete, the electronic device can perform face recognition on the facial image to be recognized based on the template image of the occlusion, and in the case that the template image of the occlusion matches the facial image to be recognized, perform a preset operation. In this way, when the face in the facial image captured by the electronic device is not complete, the electronic device can also accurately perform face recognition on the facial image, and the recognition rate of the face in the normal scenario is not used, thereby improving the experience of the user.

[0122] In some embodiments, the electronic device can be preconfigured with the first template image or obtain the first template image from other devices.

[0123] In one scenario, the electronic device is a mobile phone or a tablet computer, etc. The electronic device can pre-store a first template image. The first template image can be an image input by a user when setting up the electronic device. For example, the electronic device can capture a face image of the user when setting up an initial password for a lock screen, and detect the captured face image. In a case where the face in the captured face image is complete, the face image is stored in the electronic device as a second template image. In addition, the electronic device can obtain the first template image based on the second template image. Of course, the face completeness determination can also be performed before capturing the face image, which is not limited in the present embodiment.

[0124] In one possible implementation, the electronic device can perform key point detection on the captured face image. For example, the electronic device can input the captured face image into a face key point detection model to obtain a plurality of key points of the face. It can be understood that, if all the plurality of key points are located in the region where the face image is located, it indicates that the face in the face image is complete; otherwise, it indicates that the face is incomplete.

[0125] In another scenario, the electronic device is a fixed-position face recognition device. The electronic device can obtain the second template image from another device, or read the second template image from a certificate, and obtain the first template image based on the second template image.

[0126] For example, when the electronic device is a door access device of a community, the electronic device obtains the second template image of the user from a management device. The second template image can be a second template image of the user captured and stored by the management device through a camera in response to a shooting operation when the user handles the check-in.

[0127] For another example, when the electronic device is a door access device of a public place such as an airport or a station, the electronic device can read the second template image from a chip of a certificate such as an identity certificate.

[0128] Further, in the present embodiment, if the face in the face image is in a tilted posture, the electronic device or the management device can perform alignment processing on the face image. Please refer to Figure 11 which shows an example of alignment processing on a face image. As can be seen from Figure 11 , the face in the face image after the alignment processing is in a frontal view state. This can facilitate key point extraction and face recognition. The alignment processing can mean aligning the face in the face image based on a face detection frame. For details, please refer to the corresponding content in the related art, which will not be described herein.

[0129] The face image can be a face image captured when obtaining the second template image, or a face image captured when performing face recognition.

[0130] Based on this embodiment, the electronic device can pre-configure the second template image or acquire the second template image from other devices for subsequent face recognition.

[0131] The above is an example of pre-processing the second template image by the electronic device using the first template image. Please refer to Figure 12 which shows another face recognition method provided by the embodiments of the present application, including S1201-S1205.

[0132] S1201, the electronic device acquires a second template image.

[0133] The acquisition method of the second template image can refer to the related description in the above embodiments, which will not be repeated here.

[0134] S1202, the electronic device performs occlusion processing on the second template image to obtain a first template image.

[0135] The occlusion processing on the second template image can mean adding a mask to the second template image or removing part of the image in the second template image. The first template image can be obtained by randomly occluding the face in the second template image, or by occluding the key parts of the face in the second template image. The number of first template images can be one or more. In the case of multiple first template images, the electronic device can adjust the position of random occlusion in the first template image, or occlude multiple different key parts. For example, the mouth can be occluded, the eyes can be occluded, the left half of the face can be occluded, the right half of the face can be occluded, etc.

[0136] Please refer to Figure 13 which shows a first template image. Figure 13 In the above embodiment, the first template image is multiple. The occluded parts of the multiple images are all different.

[0137] S1203, in response to a first operation, the electronic device acquires a first face image.

[0138] S1204, in the case that the face in the first face image is incomplete, the electronic device performs face recognition on the first face image based on the first template image to obtain a face recognition result.

[0139] S1205, based on the face recognition result, the electronic device performs a preset operation.

[0140] S1203-S1205 can refer to the description of the corresponding content in S601-S603 above, which will not be repeated here.

[0141] In an example, in the S1202 above, the number of the first template images is multiple. The electronic device can calculate the similarity between the first face image and each of the first template images to obtain multiple similarities. In a case where there is a similarity greater than a preset threshold in the multiple similarities, or the number of the similarities greater than the preset threshold in the multiple similarities exceeds a preset number, the electronic device can determine that the face recognition is successful. In a case where all the similarities in the multiple similarities are less than or equal to the preset threshold, or the number of the similarities greater than the preset threshold in the multiple similarities does not exceed the preset number, the electronic device can determine that the face recognition fails.

[0142] In another example, in the S1202 above, the number of the first template images is one. The electronic device can directly calculate the similarity between the first face image and the first template image. In a case where the similarity is greater than a preset threshold, the electronic device can determine that the face recognition is successful. In a case where the similarity is less than or equal to the preset threshold, the electronic device can determine that the face recognition fails.

[0143] Based on Figure 12 According to the technical solution shown, the electronic device can perform occlusion processing on the second template image in advance to obtain the first template image. In this way, after the electronic device obtains the face image to be recognized in which the face is incomplete, the electronic device can directly use the first template image to perform face recognition on the obtained face image, which is fast and convenient.

[0144] The embodiments of the present application are exemplarily described taking the first template image as an example, which is obtained by performing occlusion processing on the second template image in a case where the electronic device determines that the face in the first face image is incomplete. Please refer to Figure 14 which shows another face recognition method provided by an embodiment of the present application, including S1401-S1403.

[0145] S1401, in response to a first operation, the electronic device obtains a first face image.

[0146] The S1401 can refer to the related description in the S601 above, and will not be described here again.

[0147] S1402, in a case where it is determined that the face in the first face image is incomplete, a second template image is obtained, and occlusion processing is performed on the second template image to obtain a first template image.

[0148] The method for determining that the face in the first face image is incomplete can refer to the description of the S602 above, and will not be described here again.

[0149] In a possible implementation manner, the electronic device can perform occlusion processing on the second template image according to the missing area of the first face image to obtain the first template image.

[0150] In an example, the electronic device can detect the first face image, determine the missing region of the first face image, and obtain the first template image by performing occlusion processing or deletion on the second template image in the missing region.

[0151] It should be noted that if the second template image further includes other parts (such as the neck, hair, etc.) in addition to the face, the electronic device can detect the second template image based on the face detection frame, determine the face image in the second template image, and perform occlusion processing or delete part of the obtained face image to obtain the first template image.

[0152] In a possible implementation, the electronic device can determine the missing region in the first face image based on the abnormal key point in the plurality of key points of the first face image. It can be understood that the face part corresponding to the abnormal key point is the missing region of the first face image.

[0153] For example, the electronic device can determine the face part corresponding to the abnormal key point of the first face image based on the face direction of the first face image, and determine the missing region in the first face image based on the face part corresponding to the abnormal key point.

[0154] In the embodiments of the present application, the electronic device can determine the face direction of the first face image based on the face direction detection model, or the electronic device can also determine the face direction of the first face image based on the positions of the key points in the first face image. It can be understood that the positions of the key points of different face images can be different, but the gap between the key points or the region where the key points of the same type are located in the whole face is relatively close. Therefore, for the face image with incomplete face, the electronic device can determine the face direction of the face image based on the positions of the normal key points in the plurality of key points of the face image. Specifically, reference can be made to the prior art, which is not limited.

[0155] After determining that there is an abnormal key point in the plurality of key points, the electronic device can determine the face part corresponding to the abnormal key point of the first face image according to the face direction of the first face image. In this way, the electronic device can determine the missing region of the first face image based on the face part of the abnormal key point.

[0156] The face direction can include upward, downward, left, and right. Upward means that the mouth of the face in the face image is below the eye, downward means that the mouth of the face in the face image is above the eye, left means that the right eye of the face in the face image is above the left eye, and right means that the left eye of the face in the face image is above the right eye.

[0157] It can be understood that the face direction in the first face direction is upward. If the abnormal key point is located above the face, the face part corresponding to the abnormal key point is the eye; if the abnormal key point is located below the face, the face part corresponding to the abnormal key point is the mouth.

[0158] The face direction in the first face direction is downward. If the abnormal key point is located above the face, the face part corresponding to the abnormal key point is the mouth; if the abnormal key point is located below the face, the face part corresponding to the abnormal key point is the eye.

[0159] The face direction in the first face direction is leftward. If the abnormal key point is located on the left side of the face, the face part corresponding to the abnormal key point is the mouth; if the abnormal key point is located on the right side of the face, the face part corresponding to the abnormal key point is the eye.

[0160] The face direction in the first face direction is rightward. If the abnormal key point is located on the left side of the face, the face part corresponding to the abnormal key point is the eye; if the abnormal key point is located on the right side of the face, the face part corresponding to the abnormal key point is the mouth.

[0161] Further, after determining the face part corresponding to the abnormal key point of the first face image, the electronic device can determine the offset width of the face erasing in the first face image based on the coordinates of the abnormal key point, and perform occlusion processing on the second template image based on the offset width to obtain the first template image.

[0162] The offset width is the sum of the abnormal value in the coordinates of the abnormal key point and the distance from the abnormal key point to the corresponding edge. The width of the occluded area on the first template image is the same as the offset width.

[0163] For example, in combination with the first face image with missing mouth corners in the above Figure 9b As shown in Figure 15 , the electronic device can determine the offset width based on the coordinates of the key points of the mouth corners. For example, the coordinates of the key point of the left mouth corner are (x1, y1), and the distance between the key point of the left mouth corner and the lower edge of the face is a, then the offset width is equal to |y1|+a. In this way, the electronic device can occlude the lower edge area of the second template image, and the width of the occluded area is equal to |y1|+a. Wherein, the lower edge of the face can be the lower edge of the face in the virtual first face image.

[0164] Similarly, in a case where the missing region of the first face image is an eye region, the offset width can be a sum of a difference between a vertical coordinate of a key point of the eye and a height of the first face image and a distance between the key point of the eye and an upper edge of the face. For example, the coordinate of the key point of the eye is (x2, y2), the distance between the key point of the left eye and the upper edge of the face is b, and the offset width is equal to |y2-h|+b. Correspondingly, the width of the occlusion region of the first template image is equal to |y2-h|+b. The upper edge of the face can be a virtual upper edge of the face in the first face image.

[0165] The electronic device can perform occlusion processing on the upper edge region of the second template image to obtain the first template image. The width of the occlusion region is the difference. It can be understood that the first template image has the same missing features as the first face image.

[0166] In combination with the above Figure 11 For a first face image in which the face is in a tilted posture, the electronic device can determine an offset width according to an abnormal key point of the first face image after alignment processing, so as to perform occlusion processing on a second template image based on the determined offset width to obtain a first template image.

[0167] S1403, performing face recognition on the first face image based on the first template image to obtain a face recognition result, and performing a preset operation based on the face recognition result.

[0168] S1403 can refer to the related description in S602 and S603 described above, and will not be described here.

[0169] Based on Figure 14 the technical solutions, after the electronic device obtains the first face image, the electronic device can determine a missing region of the first face image, and perform occlusion processing on a second template image based on the missing region to obtain a first template image close to the first face image. That is, the occlusion part of the first template image is close to the missing region of the first face image, and then when the electronic device performs face recognition on the first face image using the first template image, the face recognition result is more accurate due to the same missing face features.

[0170] In some embodiments, please refer to Figure 16, the electronic device is pre-configured with the second template image and the first template image. After obtaining the first face image, the electronic device can first calculate the similarity between the first face image and the second template image. For example, the similarity between the feature vector of the first face image and the feature vector of the second template image is calculated. If the similarity between the first face image and the second template image is less than the first threshold but greater than the second threshold, the electronic device calculates the similarity between the first face image and the first template image. If the similarity between the first face image and the first template image is greater than the third threshold, the face recognition is successful.

[0171] wherein the first threshold is greater than the second threshold, and the first threshold is less than the third threshold.

[0172] Alternatively, the electronic device can calculate the similarity between the first face image and the first template image and the second template image respectively. If the similarity between the first face image and the second template image is less than the first threshold but greater than the second threshold, and the similarity between the first face image and the first template image is greater than the third threshold, the face recognition is successful.

[0173] In some other embodiments, please refer to Figure 17 The face recognition method provided by the embodiments of the present application can include S1701-S1708.

[0174] S1701, in response to a first operation, the electronic device acquires a first face image of a user.

[0175] Wherein, S1701 can refer to S601 described above, which will not be repeated here.

[0176] S1702, the electronic device performs face detection on the first face image.

[0177] Wherein, the face detection includes detecting face direction, determining whether the face is complete, face recognition, etc.

[0178] In an example, if the face detection is face recognition, S1703 is performed; if the face detection is to determine whether the face is complete, S1704 is performed; if the face detection is to detect the face direction, S1705 is performed. Alternatively, the electronic device can also directly perform S1706.

[0179] S1703, the electronic device calculates the similarity between the second template image and the first face image.

[0180] Wherein, the second template image is obtained from a user's face image. The second template image can be stored in a storage module. The storage module can be a storage module of the electronic device, or a storage module of another device.

[0181] If the similarity is greater than the first threshold, the face recognition is successful; if the similarity is less than or equal to the first threshold but greater than the second threshold, then execute S1704.

[0182] S1704. The electronic device determines whether the face in the first face image is complete.

[0183] If the face in the first face image is complete, the face recognition fails; if the face in the first face image is incomplete, then S1705 is executed.

[0184] S1705. Electronic device detects the face orientation in the first face image.

[0185] S1705 can be referred to the relevant description in the above embodiments, and will not be repeated here.

[0186] S1706. The electronic device determines the missing region of the first face image and determines the offset width based on the missing region of the first face image.

[0187] S1706 can be referred to the relevant description in the above embodiments, and will not be repeated here.

[0188] S1707. The electronic device performs occlusion processing on the second template image according to the offset width to obtain the first template image.

[0189] S1708. The electronic device calculates the similarity between the first template image and the first face image.

[0190] If the similarity is greater than the third threshold, the face recognition is successful; if the similarity is less than or equal to the third threshold, the face recognition fails.

[0191] based on Figure 17 The proposed technical solution allows the electronic device to detect the face after acquiring the user's facial image and perform facial recognition based on various detection methods. Even when the user's face image is incomplete, the electronic device can still perform detection based on a template image of the incomplete face. Therefore, the technical solution of this application embodiment can not only recognize incomplete facial images but also maintain the recognition rate for complete facial images.

[0192] In some embodiments, please refer to Figure 18 This demonstrates the results of verifying the technical solutions of the embodiments of this application. Figure 18 It can be seen that, compared with other face recognition technologies, the technical solution of this application embodiment has a higher matching score, and can successfully recognize face images with missing mouths and other incomplete face images (such as those wearing hats).

[0193] The embodiments of the present application further provide a chip system, which can be applied to the electronic device or the management device. As shown in Figure 19 The chip system 1900 includes at least one processor 1901 and at least one interface circuit 1902. The processor 1901 and the interface circuit 1902 can be interconnected by a line. For example, the interface circuit 1902 can be used to receive signals from other devices (e.g., the memory of the electronic device or the sound box). For another example, the interface circuit 1902 can be used to send signals to other devices (e.g., the processor 1901 of the electronic device or the sound box). For example, the interface circuit 1902 can read the instructions stored in the memory and send the instructions to the processor 1901. Of course, the chip system can also include other discrete devices, which are not limited in the embodiments of the present application.

[0194] When the instructions are executed by the processor 1901, the electronic device can perform the steps of the electronic device in the above embodiments.

[0195] When the instructions are executed by the processor 1901, the electronic device can perform the steps of the electronic device in the above embodiments.

[0196] The embodiments of the present application further provide a computer storage medium, which includes computer instructions, when the computer instructions are run on the electronic device, the electronic device performs the functions or steps of the electronic device in the above method embodiments. Figure 6

[0197] The embodiments of the present application further provide a computer program product, when the computer program product is run on the computer, the computer performs the functions or steps of the electronic device in the above method embodiments.

[0198] The embodiments of the present application further provide a computer storage medium, which includes computer instructions, when the computer instructions are run on the electronic device, the electronic device performs the functions or steps of the electronic device in the above method embodiments.

[0199] The embodiments of the present application further provide a computer program product, when the computer program product is run on the computer, the computer performs the functions or steps of the electronic device in the above method embodiments.

[0200] ​Some embodiments of the present application provide an electronic device, which can include the above-mentioned touch screen, a memory and one or more processors. The touch screen, the memory and the processors are coupled. The memory is configured to store computer program codes including computer instructions. When the processors execute the computer instructions, the electronic device can perform each function or step performed by the mobile phone in the above-mentioned method embodiments. The structure of the electronic device can refer to the structure of the electronic device 500 shown in Figure 5

[0201] Some embodiments of the present application provide a display device, which can be applied to an electronic device including the above-mentioned touch screen. The device is configured to perform each function or step performed by the target electronic device in the above-mentioned method embodiments.

[0202] Some embodiments of the present application further provide a computer storage medium, which includes computer instructions, when the computer instructions are run on the above-mentioned electronic device, make the electronic device perform each function or step performed by the mobile phone in the above-mentioned method embodiments.

[0203] Some embodiments of the present application further provide a computer program product, when the computer program product is run on a computer, make the computer perform each function or step performed by the mobile phone in the above-mentioned method embodiments.

[0204] Through the above description of the embodiments, those skilled in the art can clearly understand that, for the convenience and brevity of description, only the above-mentioned division of each functional module is taken as an example for illustration, and in actual application, the above-mentioned functions can be completed by different functional modules according to needs, that is, the internal structure of the device is divided into different functional modules to complete all or part of the above-described functions.

[0205] In several embodiments provided in the present application, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the above-mentioned device embodiments are only schematic, for example, the division of the modules or units is only a logical function division, and actual implementation can have another division manner, for example, a plurality of units or components can be combined or integrated into another device, or some features can be ignored or not executed. In addition, the coupling or direct coupling or communication connection between the displayed or discussed each other can be through some interface, indirect coupling or communication connection between devices or units, which can be electrical, mechanical or other forms.

[0206] ​The units described as separate components may or may not be physically separate, and the components displayed as units may be a physical unit or multiple physical units, that is, may be located in one place, or also can be distributed to multiple different places. Part or all of the units can be selected according to actual needs to achieve the purpose of the embodiment scheme.

[0207] In addition, each functional unit in each embodiment of the present application can be integrated in one processing unit, or each unit can be physically present alone, or two or more units can be integrated in one unit. The integrated unit can be realized in the form of hardware or in the form of a software functional unit.

[0208] The integrated unit, if realized in the form of a software functional unit and sold or used as an independent product, can be stored in a readable storage medium. Based on such understanding, the technical scheme of the embodiments of the present application essentially or the part that contributes to the prior art or the whole or part of the technical scheme can be embodied in the form of a software product, which is stored in a storage medium and includes a plurality of instructions for causing an apparatus (which can be a single-chip microcomputer, a chip, etc.) or a processor to execute all or part of the steps of the method described in the embodiments of the present application. The foregoing storage medium includes: a U disk, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, and various program code storage media.

[0209] The above is only a specific embodiment of the present application, but the protection scope of the present application is not limited thereto, any change or replacement within the technical scope disclosed in the present application should be covered in the protection scope of the present application. Therefore, the protection scope of the present application should be subject to the protection scope of the claims.

Claims

1. A face recognition method, characterized in that, Applied to electronic devices, the method includes: In response to the first operation, the user's first facial image is captured; Obtain multiple key points from the first face image; In the case where the face in the first face image is incomplete, face recognition is performed on the first face image based on the first template image to obtain a face recognition result; the first template image is a template image with an incomplete face, and the incomplete face in the first face image includes the presence of abnormal key points among the plurality of key points, and the abnormal key points are not included in the area where the first face image is located. Based on the facial recognition results, a preset operation is performed.

2. The method according to claim 1, characterized in that, The method further includes: The first template image is obtained from the storage module. The first template image is obtained in advance based on the second template image, which is a template image of a complete face.

3. The method according to claim 1, characterized in that, The method further includes: The first template image is obtained based on a pre-stored second template image, where the second template image is a complete template image of a human face.

4. The method according to claim 2 or 3, characterized in that, The first template image is obtained in the following way: The second template image is occluded to obtain the first template image.

5. The method according to claim 2 or 3, characterized in that, In the case where the face in the first face image is incomplete, the method further includes: Determine the orientation of the face in the first face image; Based on the face orientation in the first face image, determine the missing region of the first face image; The first template image is obtained in the following way: The face portion corresponding to the missing area in the second template image is occluded to obtain the first template image.

6. The method according to claim 5, characterized in that, The step of determining the missing region of the first face image based on the face orientation in the first face image includes: Based on the face orientation in the first face image, the face region corresponding to the abnormal key point in the first face image is determined; the abnormal key point is a key point among multiple key points in the first face image that is not included in the area where the first face image is located. Based on the facial features corresponding to the abnormal key points, the missing regions of the first facial image are determined.

7. The method according to claim 6, characterized in that, The height of the obscured face in the first template image is the same as the height of the missing area.

8. The method according to claim 1, characterized in that, The step of performing face recognition on the first face image based on the first template image to obtain the face recognition result includes: The feature vector of the second template image is processed to obtain the feature vector of the first template image, and face recognition is performed on the first face image based on the feature vector of the first template image to obtain the face recognition result; the second template image is a template image of a complete face.

9. The method according to any one of claims 1-8, characterized in that, Before performing face recognition on the first face image based on the first template image, the method further includes: Face recognition is performed on the first face image based on the second template image; In the case where the face in the first face image is incomplete, face recognition is performed on the first face image based on the first template image to obtain a face recognition result, including: If the face recognition of the first face image based on the second template image fails, face recognition is performed on the first face image based on the first template image to obtain the face recognition result.

10. The method according to claim 9, characterized in that, The method further includes: If the similarity between the first face image and the second template image is less than or equal to a first threshold and greater than a second threshold, it is determined that the face recognition of the first face image based on the second template image failed; wherein, the first threshold is greater than the second threshold.

11. The method according to claim 10, characterized in that, If the similarity between the first face image and the first template image is greater than a third threshold, the face recognition result is considered successful; wherein the third threshold is greater than the first threshold.

12. An electronic device, characterized in that, The electronic device includes: a memory, a communication module, and one or more processors; the memory and the communication module are coupled to the processors; the communication module is used to communicate with other devices, and the memory is used to store data in the electronic device; the memory is also used to store computer program code, the computer program code including computer instructions, which, when executed by the processor, cause the electronic device to perform the method as described in any one of claims 1-11.

13. A chip system, characterized in that, The chip system is applied to an electronic device including a communication module and a memory; the chip system includes one or more interface circuits and one or more processors; the interface circuits and the processors are interconnected via lines; the interface circuits are used to receive signals from the memory and send the signals to the processors, the signals including computer instructions stored in the memory; when the processor executes the computer instructions, the electronic device performs the method as described in any one of claims 1-11.

14. A computer-readable storage medium, characterized in that, Includes computer instructions that, when executed on an electronic device, cause the electronic device to perform the method as described in any one of claims 1-11.

Images