Face recognition method and device

Abstract

The application provides a face recognition method and device, relates to face recognition technology, and can reduce the use requirement of the face recognition technology and improve the availability of the face recognition technology. The specific scheme is applied to an electronic device, and the method comprises the following steps: the electronic device displays a first interface, the first interface is used for setting a face image used for face recognition; a first image is collected through a camera, and the first image is displayed on the first interface; facial features in the first image are extracted, first facial features are obtained and saved; the first facial features are local face facial features, and the first facial features are used for face recognition; when a starting operation on a first application service is received, a second image is collected through the camera; facial features in the second image are extracted, second facial features are obtained, and the second facial features are compared with the first facial features; the second facial features are local face facial features; and if the second facial features match the first facial features, the first application service is started.

Description

Technical Field

[0001] This application relates to facial recognition technology, and more particularly to a facial recognition method and apparatus. Background Technology

[0002] Currently, facial recognition technology is increasingly being used to identify users. This allows for rapid verification of user identity and execution of corresponding actions without user intervention. For example, in the scenario of unlocking a mobile phone, the phone first captures a first image including the user's face; extracts and saves the first facial features from the first image; then, when the user initiates a preset application service (such as unlocking or payment), the camera captures a second image of the user; the second facial features are then extracted from the second image and compared with the first facial features. If the second facial features match the first facial features successfully, it indicates that the user's identity information is correct, and the aforementioned preset application service can be executed.

[0003] Generally, when a mobile phone captures the first image, it must include the user's complete face, so that the first facial features extracted from that first image include the user's complete facial features. Therefore, only when the second image captured by the mobile phone also includes the user's complete facial features can the second facial features extracted from the second image include the user's complete facial features. Thus, the second facial features can successfully match the first facial features, and user authentication can be passed. However, if the user's face is partially obscured by a mask, scarf, or other object when activating the aforementioned preset application service, user identity verification will fail. The above scheme has high requirements for the use of facial recognition technology, thus reducing its usability. Summary of the Invention

[0004] This application provides a face recognition method and apparatus, which can reduce the requirements for using face recognition technology and improve its usability.

[0005] To achieve the above technical objectives, this application adopts the following technical solution:

[0006] In a first aspect, this application provides a face recognition method applied to an electronic device, which includes a camera. In this face recognition method, the electronic device can display a first interface for setting a face image for face recognition. Then, the electronic device can capture a first image through the camera and display the first image on the first interface. The electronic device can extract facial features from the first image to obtain a first facial feature and save the first facial feature. The first facial feature is a local facial feature, and it can be used for face recognition.

[0007] In other words, the electronic device can receive a partial face image (i.e., the first image) set by the user for face recognition on the first interface, and the partial face image may include partial facial features.

[0008] Thus, when the electronic device receives a request to launch the first application service, it can capture a second image through its camera; extract facial features from the second image to obtain the second facial features. The image captured by the electronic device during face recognition (such as the second image) may be a complete face image, including complete facial features; or it may be a partial face image, including partial facial features.

[0009] In conventional technologies, electronic devices can only perform face recognition based on complete face images; that is, face recognition based on the second image can only succeed if the second image is a complete face image. However, in this application, the electronic device displays a partial face image for face recognition on its first interface. Therefore, when the second image is a partial face image and the second facial features are partial facial features, if the second facial features match the first facial features, face recognition can succeed, and the electronic device can initiate the first application service.

[0010] In summary, this method demonstrates that facial recognition can be successfully performed using partial facial images. For example, electronic devices can record partial facial features, enabling user identification even when the user's face is obscured. This approach lowers the requirements for using facial recognition technology and improves its usability.

[0011] In one possible implementation, the first interface includes a first option; wherein the first option is used to trigger the electronic device to set local facial features for face recognition. The electronic device acquires a first image via a camera, including: the electronic device acquiring the first image via the camera in response to a user's click on the first option; wherein the user's click on the first option triggers the electronic device to save the first facial features to a preset local feature set library. Saving the first facial features includes: the electronic device saving the first facial features to the preset local feature set library.

[0012] This design approach describes one implementation where an electronic device provides settings for localized facial features used for face recognition. The electronic device can offer a user a first option to set localized facial features, allowing the user to select the appropriate localized facial features for face recognition when the face is occluded, according to their needs.

[0013] In another possible implementation, the electronic device saves the first facial feature to a preset local feature set library, including: if the electronic device determines that the first facial feature is a local facial feature, then it saves the first facial feature to the preset local feature set library.

[0014] This design describes one way in which an electronic device can save a first facial feature: after acquiring local facial features, the electronic device can save the local facial features to a corresponding preset local feature set library.

[0015] In another possible implementation, the face recognition method further includes: if the first facial feature is a complete facial feature, the electronic device issues a first prompt message, which is used to instruct the user to enter a partial facial image.

[0016] In this design, when the electronic device is used to set a partial face image for face recognition on the first interface, if the extracted first facial feature is not a partial face feature, the user will be prompted that the first image currently captured is not a partial face image. This avoids the problem that the complete face feature is captured when the user triggers the electronic device to set a partial face feature for face recognition, which would prevent the user from using face recognition technology when the user's face is occluded.

[0017] In another possible implementation, the first interface includes a second option; wherein the second option is used to trigger the electronic device to set complete facial features for face recognition. The face recognition method further includes: in response to a user's click on the second option, the electronic device captures a third image through a camera and displays the third image on the first interface; the electronic device extracts facial features from the third image to obtain a third facial feature, and saves the third facial feature to a preset complete feature set library; wherein the third facial feature is a complete facial feature, and the third facial feature is used for face recognition. The user's click on the second option triggers the electronic device to save the third facial feature to the preset complete feature set library.

[0018] This design approach describes one implementation where an electronic device provides settings for complete facial features used for face recognition. The electronic device can provide a second option for the user to set complete facial features, allowing the user to customize the full facial features for face recognition when the face is not obscured, according to their own needs.

[0019] In another possible implementation, saving the first facial feature includes: if the electronic device determines that the first facial feature is a local facial feature, then saving the first facial feature to a preset local feature set library. The face recognition method further includes: if the first facial feature is a complete facial feature, then the electronic device saves the first facial feature to a preset complete feature set library.

[0020] In this design approach, after the electronic device extracts the first facial feature, it determines whether the first facial feature is a partial or complete facial feature. Then, it categorizes and stores the partial and complete facial features.

[0021] In another possible implementation, the above comparison of the second facial feature with the first facial feature includes: the electronic device comparing the second facial feature with the first facial feature in a preset local feature set library.

[0022] In this design, after the electronic device extracts the second facial feature, if the second facial feature is a local facial feature, it compares the second facial feature with the first facial feature in a preset local feature set library. The first facial feature in the preset local feature set library is a local facial feature, meaning the electronic device verifies user identity information even when the user's face is occluded.

[0023] In another possible implementation, the electronic device has multiple security levels pre-set for application services. Specifically, the action of activating the first application service if the second facial feature matches the first facial feature includes: if the second facial feature matches the first facial feature and the security level of the first application service is lower than (or not higher than) the security level of the first facial feature, then the electronic device activates the first application service.

[0024] In this design approach, considering that the more facial features used, the higher the accuracy of verifying user identity information, the electronic device can set a corresponding security level for the first facial feature (i.e., a local facial feature). Secondly, since different application services within the electronic device involve varying degrees of information privacy, their security levels may also differ. Therefore, if the security level of the first application service is lower (or not higher) than the security level of the first facial feature, it means that the application service accessible to the user in the second image (i.e., the user whose second facial feature matches the first facial feature) has a higher security level than the first application service, and only then will the electronic device activate the first application service. In this way, the usability of the facial recognition technology provided by the electronic device can be improved while ensuring its security.

[0025] In another possible implementation, the facial recognition method further includes: if the second facial feature matches the first facial feature, and the security level of the first application service is higher than or equal to the security level of the first facial feature, then the electronic device issues a second prompt message. The second prompt message is used to prompt the user to record a complete facial image or to launch the first application service through a preset authentication method; the preset authentication method includes password authentication and / or fingerprint authentication.

[0026] In this design approach, based on the above analysis that "if the security level of the first application service of the electronic device is lower than (or not higher than) the security level of the first facial feature, it means that the security level of the application service that the user in the second image (i.e., the user whose second facial feature matches the first facial feature) can access is higher than that of the first application service," it can be seen that if the security level of the first application service of the electronic device is higher than that of the first facial feature, it means that the security level of the application service that the user in the second image (i.e., the user whose second facial feature matches the first facial feature) can access is lower than (or not higher than) that of the first application service. Therefore, the electronic device does not start the first application service, which can ensure the security of the facial recognition technology provided by the electronic device.

[0027] In another possible implementation, the security level of the first facial feature is a preset first security level; or, the security level of the first facial feature is determined based on the degree of matching between the first facial feature and the user's complete facial features.

[0028] This design describes two methods for setting the security level of the first facial feature. Considering that the more facial features used, the higher the accuracy of verifying user identity information, the electronic device can determine the security level of the first facial feature based on the degree of matching between the first facial feature (i.e., a partial facial feature) and the user's complete facial features. For example, the higher the degree of matching between the first facial feature (i.e., a partial facial feature) and the user's complete facial features, the higher the security level of the first facial feature.

[0029] Secondly, this application also provides an electronic device, which includes: a camera, a display module, a data acquisition module, a feature processing module, and an execution module.

[0030] The system includes a display module for displaying a first interface, which is used to set a facial image for face recognition; an acquisition module for acquiring a first image via a camera; a feature processing module for extracting facial features from the first image to obtain a first facial feature and saving the first facial feature; wherein the first facial feature is a local facial feature used for face recognition; an acquisition module for acquiring a second image via a camera when the electronic device receives a startup operation for a first application service; a feature processing module for extracting facial features from the second image to obtain a second facial feature and comparing the second facial feature with the first facial feature; wherein the second facial feature is a local facial feature; and an execution module for starting the first application service if the second facial feature matches the first facial feature.

[0031] In one possible implementation, the first interface includes a first option; wherein the first option is used to trigger the electronic device to set local facial features for face recognition. The aforementioned acquisition module, used to acquire a first image via a camera, includes: an acquisition module specifically used to acquire the first image via a camera in response to a user's click operation on the first option. The user's click operation on the first option triggers the electronic device to save the first facial features to a preset local feature set library. The aforementioned feature processing module, used to save the first facial features, includes: a feature processing module specifically used to save the first facial features to a preset local feature set library.

[0032] In another possible implementation, the feature processing module described above is used to save the first facial feature to a preset local feature set library, including: the feature processing module is specifically used to determine that the first facial feature is a local facial feature, and then save the first facial feature to the preset local feature set library.

[0033] In another possible implementation, the execution module is further configured to issue a first prompt message if the first facial feature is a complete human face feature, the first prompt message being used to instruct the user to enter a partial human face image.

[0034] In another possible implementation, the first interface includes a second option; wherein the second option is used to trigger the electronic device to set complete facial features for face recognition. The acquisition module is further configured to acquire a third image via a camera in response to the user's click on the second option; the display module is further configured to display the third image on the first interface; and the feature processing module is further configured to extract facial features from the third image to obtain third facial features, and save the third facial features to a preset complete feature set library.

[0035] The third facial feature is a complete set of facial features used for facial recognition. The user's click on the second option triggers the electronic device to save the third facial feature to a pre-defined complete feature set library.

[0036] In another possible implementation, the feature processing module is specifically used to: if the first facial feature is a local facial feature, then save the first facial feature to a preset local feature set library; if the first facial feature is a complete facial feature, then save the first facial feature to a preset complete feature set library.

[0037] In another possible implementation, the feature processing module described above is used to compare the second facial features with the first facial features, including: a feature processing module specifically used to compare the second facial features with the first facial features in a preset local feature set library.

[0038] In another possible implementation, the electronic device has multiple application service security levels pre-set. The aforementioned execution module, configured to activate the first application service if the second facial feature matches the first facial feature, includes: the execution module specifically configured to activate the first application service if the second facial feature matches the first facial feature and the security level of the first application service is lower than (or not higher than) the security level of the first facial feature.

[0039] In another possible implementation, the execution module is further configured to issue a second prompt message if the second facial feature matches the first facial feature and the security level of the first application service is higher than or equal to the security level of the first facial feature. The second prompt message prompts the user to enter a complete facial image or to launch the first application service through a preset authentication method; the preset authentication method includes password authentication and / or fingerprint authentication.

[0040] In another possible implementation, the security level of the first facial feature is a preset first security level; or, the security level of the first facial feature is determined based on the degree of matching between the first facial feature and the user's complete facial features.

[0041] Thirdly, this application also provides an electronic device, which includes a processor and a memory. The memory stores computer program code, including computer instructions; the processor executes the computer instructions, causing the electronic device to perform a face recognition method as described in the first aspect and any possible implementation thereof.

[0042] Fourthly, this application also provides a computer-readable storage medium storing computer instructions that, when executed on an electronic device, cause the electronic device to perform a face recognition method as described in the first aspect and any possible implementation thereof.

[0043] Fifthly, this application also provides a computer program product including one or more instructions that can run on an electronic device to cause the electronic device to perform a face recognition method as described in the first aspect and any possible implementation thereof.

[0044] The technical effects of the second aspect of this application and any possible design method thereof, as well as the third, fourth and fifth aspects, can be found in the technical effects of the different design methods in the first aspect above, and will not be repeated here. Attached Figure Description

[0045] Figure 1 A flowchart of a face recognition method provided by conventional technology;

[0046] Figure 2 A schematic diagram of an interface for an electronic device to capture facial images using conventional technology. Figure 1 ;

[0047] Figure 3 A schematic diagram of an interface for facial recognition in an electronic device using conventional technology;

[0048] Figure 4 A schematic diagram of an interface for an electronic device to capture facial images using conventional technology. Figure 2 ;

[0049] Figure 5 A schematic diagram of the hardware structure of an electronic device provided in an embodiment of this application;

[0050] Figure 6 A flowchart of a face recognition method provided in this application embodiment Figure 1 ;

[0051] Figure 7 A schematic diagram of an interface for an electronic device to capture facial images, provided as an embodiment of this application;

[0052] Figure 8 A schematic diagram of the interface of an electronic device waiting to acquire a partial face image, provided as an embodiment of this application;

[0053] Figure 9 A schematic diagram of the interface for an electronic device to acquire partial facial images, provided in an embodiment of this application;

[0054] Figure 10This is a schematic diagram of the interface of an electronic device when data acquisition ends, provided in an embodiment of this application.

[0055] Figure 11 This application provides an embodiment of an interface for an electronic device to capture facial images when starting an application service. Figure 1 ;

[0056] Figure 12 This is a schematic diagram of the interface of an electronic device that performs face recognition on a partial face image, provided in an embodiment of this application.

[0057] Figure 13 A flowchart of a face recognition method provided in this application embodiment Figure 2 ;

[0058] Figure 14 A schematic diagram of the interface of an electronic device waiting to collect partial or complete facial images, provided for an embodiment of this application;

[0059] Figure 15 A schematic diagram of the interface for an electronic device to capture a complete face image, provided as an embodiment of this application;

[0060] Figure 16 This application provides an embodiment of an interface for an electronic device to capture facial images when starting an application service. Figure 2 ;

[0061] Figure 17 A schematic diagram of the interface of an electronic device performing face recognition on a complete face image, provided in an embodiment of this application;

[0062] Figure 18 A flowchart of a face recognition method provided in this application embodiment Figure 3 ;

[0063] Figure 19 A schematic diagram of the interface of an electronic device waiting for automatic acquisition of partial or complete facial images, provided in an embodiment of this application;

[0064] Figure 20 A schematic diagram of the interface for an electronic device to perform automatic data acquisition, provided in an embodiment of this application;

[0065] Figure 21 This application provides an embodiment of an interface for an electronic device to capture facial images when starting an application service. Figure 3 ;

[0066] Figure 22 This is a schematic diagram of the interface of an electronic device that performs face recognition on partial or complete face images, as provided in an embodiment of this application.

[0067] Figure 23 This is a schematic diagram of the module composition of an electronic device provided in an embodiment of this application. Detailed Implementation

[0068] Hereinafter, the terms "first," "second," etc., are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of indicated technical features. Therefore, a feature defined with "first," "second," etc., may explicitly or implicitly include one or more of that feature. In the description of this embodiment, unless otherwise stated, "a plurality of" means two or more.

[0069] Currently, electronic devices are increasingly using facial recognition technology to identify users. Specifically, the electronic device first captures images or videos including the user's face using a camera or video camera; then, it detects faces in the captured images or videos and compares the detected faces with a facial database to identify the user. Because facial recognition technology can quickly confirm user identity without user intervention in scenarios where electronic devices have image capture capabilities, it provides a fast and convenient authentication method for electronic devices.

[0070] In one approach, to enhance the security of facial recognition technology, electronic devices only extract and save the user's complete facial features. When a user activates a preset application service on the electronic device, the user's complete facial features are used to verify their identity. If the user's face is obscured, the complete facial features cannot be extracted, thus failing the identity verification. The preset application service could be for tasks such as unlocking the display, making payments, or viewing multimedia files.

[0071] Please refer to Figure 1 The face recognition method provided by the above-described scheme includes S101-S107.

[0072] S101. The electronic device displays a face registration interface, which is used to set a complete face image for face recognition.

[0073] S102. The electronic device captures a first image through a camera and displays the first image on the face recognition interface.

[0074] S103. The electronic device extracts facial features from the first image to obtain the first facial features. When it is determined that the first facial features are complete facial features, the first face is saved to the face feature set library.

[0075] S104. When the electronic device receives a startup operation for a preset application service, the electronic device captures a second image through the camera.

[0076] S105. The electronic device extracts facial features from the second image to obtain second facial features, and compares the second facial features with the first facial features.

[0077] S106. If the second facial feature is a complete facial feature and the second facial feature matches the first facial feature, the electronic device starts the preset application service.

[0078] S107. If the second facial feature is not a complete facial feature, the electronic device will not activate the preset application service and will issue a prompt message for re-entry.

[0079] For example, taking a mobile phone as the electronic device and the preset application service as screen unlocking, the mobile phone 200 displays a face registration interface 210, such as... Figure 2 As shown in (a) above, the face registration interface 210 is the interface for the mobile phone 200 to capture face images for face recognition. The face registration interface 210 displays an image preview area 211 and registration prompt information 212. The image preview area 211 displays the image captured by the mobile phone 200 through the camera. The registration prompt information 212 may include "Ensure your face is not obstructed".

[0080] While displaying the face registration interface 210, the mobile phone 200 captures the user's first image and displays it in the image preview area 211. Then, the mobile phone 200 determines whether the first image includes the user's complete face. If the user is wearing a mask covering part of their face, the first image does not include the user's complete face (or in other words, the first image includes a portion of the user's face), and image capture fails. The mobile phone 200 displays a failure message 213 on the face registration interface 210. The failure message 213 may include "No face detected" and "Ensure face is unobstructed," such as... Figure 2 As shown in (b) of the diagram. If the first image includes the user's complete face, the mobile phone 200 extracts the first facial feature from the first image and saves the first facial feature to the face feature set library; the first facial feature is the user's complete facial feature. The mobile phone 200 also displays a success message 214 on the face registration interface 210, the content of which can be "Registration successful", such as... Figure 2 As shown in (c) in the figure.

[0081] Furthermore, when the phone 200 is black and the screen is locked, it receives the user's screen-on operation and displays the lock screen interface 220, such as... Figure 3As shown in (a) above. The mobile phone 200 also captures a second image of the user and extracts a second facial feature from it. If the second image includes a partial image of the user's face, then the second facial feature is the user's partial facial feature. If the mobile phone 200 determines that the second facial feature does not match the first facial feature and cannot pass the user's identity verification, then the mobile phone 200 does not initiate the display unlock and displays a re-entry prompt on the lock screen 220. The re-entry prompt reads, "Click here or swipe the screen to try again, ensuring your face is not obstructed," as shown below. Figure 3 As shown in (b) of the diagram.

[0082] For example, please refer to Figure 4 While displaying another face registration interface 230, the mobile phone 200 can capture the user's first image and display it in the image preview area 231 of the face registration interface 230. Then, when the mobile phone 200 determines that the first image includes a portion of the user's face, it indicates that the image capture has failed and can display a failure message 232 on the face registration interface 230. The content of the failure message 232 may include "Face is obscured" and "Please try to remove the object obscuring the face".

[0083] Understandably, since mobile phones only capture the user's complete face and then use these features to verify identity, verification fails when the user's face is obscured; in other words, facial recognition technology cannot be used when the user's face is occluded. To address this issue, the phone can also save a second image, including a partial view of the user's face, when the user fails verification. Then, after saving multiple second images containing partial facial features, the phone extracts and saves these partial facial features. Consequently, even when the user's face is obscured, these partial facial features can be used to verify identity. Thus, users can pass identity verification even when their faces are obscured.

[0084] However, the aforementioned scheme only extracts the user's partial facial features from a second image, including parts of the user's face, after the user has experienced multiple verification failures due to facial occlusion. This results in the user experiencing multiple verification failures when their face is occluded, meaning facial recognition technology cannot function properly when the user's face is obscured. Secondly, if the size of the obscured area varies during multiple verification attempts, the electronic device needs to collect more second images, including parts of the user's face, to ensure the stability of the partial facial features used for verification. In other words, the user experiences more verification failures when their face is occluded, and the number of failures varies from user to user. In conclusion, facial recognition technology is highly unstable when the user's face is occluded, reducing the user experience.

[0085] In another approach, electronic devices use iris recognition technology to verify user identity. Iris recognition technology identifies individuals based on the iris of the eye. However, because iris recognition technology requires specific iris image acquisition devices, it suffers from high hardware costs, limiting its widespread use.

[0086] This application provides a face recognition method that reduces the requirements for using face recognition technology, allowing it to function even when the user's face is obscured, thus improving its usability. It also enhances the stability of face recognition when the user's face is obscured. Furthermore, the face recognition method provided in this application can employ either two-dimensional (2D) or three-dimensional (3D) face recognition technology, avoiding the problem of excessively high hardware costs that limit its widespread application.

[0087] It should be noted that the electronic device in the embodiments of this application can be any device including a camera, such as a mobile phone, tablet computer, desktop computer, handheld computer, laptop computer, in-vehicle device, ultra-mobile personal computer (UMPC), netbook, personal digital assistant (PDA), augmented reality device, virtual reality device, etc. The embodiments of this application do not impose any special restrictions on the specific form of the electronic device.

[0088] It should be noted that the face recognition method provided in this application embodiment can be applied to the aforementioned electronic device. The executing entity of the face recognition method provided in this application embodiment can also be a face recognition device, which can be an electronic device. This face recognition device can also be an APP installed with face recognition functionality; or, the face recognition device can also be a central processing unit (CPU) in the electronic device; or, a control module in the electronic device for executing the face recognition method.

[0089] The following section uses a mobile phone as an example to introduce the hardware structure of electronic devices. Figure 5 As shown, the electronic device 300 may include: a processor 310, an external memory interface 320, an internal memory 321, a universal serial bus (USB) interface 330, a charging management module 340, a power management module 341, a battery 342, an antenna 1, an antenna 2, a mobile communication module 350, a wireless communication module 360, an audio module 370, a speaker 370A, a receiver 370B, a microphone 370C, a headphone jack 370D, a sensor module 380, buttons 390, a motor 391, an indicator 392, a camera 393 (which may include cameras 1-N), a display screen 394 (such as a touch screen), and a subscriber identification module (SIM) card interface 395 (which may include SIM card interfaces 1-N), etc.

[0090] The aforementioned sensor module 380 may include sensors such as orientation sensors, pressure sensors, gyroscope sensors, barometric pressure sensors, magnetic sensors, accelerometers, distance sensors, proximity sensors, fingerprint sensors, temperature sensors, touch sensors, ambient light sensors, and bone conduction sensors.

[0091] It is understood that the structure illustrated in this embodiment does not constitute a specific limitation on the electronic device 300. In other embodiments, the electronic device 300 may include more or fewer components than illustrated, or combine some components, or split some components, or have different component arrangements. The illustrated components may be implemented in hardware, software, or a combination of software and hardware.

[0092] Processor 310 may include one or more processing units, such as an application processor (AP), a modem processor, a graphics processing unit (GPU), a controller, memory, a video codec, a digital signal processor (DSP), a baseband processor, and / or a neural network processing unit (NPU). These different processing units may be independent devices or integrated into one or more processors.

[0093] The controller can be the nerve center and command center of the electronic device 300. The controller can generate operation control signals according to the instruction opcode and timing signals to complete the control of instruction fetching and execution.

[0094] The processor 310 may also include a memory for storing instructions and data. In some embodiments, the memory in the processor 310 is a cache memory. This memory can store instructions or data that the processor 310 has just used or that are used repeatedly. If the processor 310 needs to use the instruction or data again, it can retrieve it directly from the memory. This avoids repeated accesses, reduces the waiting time of the processor 310, and thus improves the efficiency of the system.

[0095] In some embodiments, the processor 310 may include one or more interfaces. It is understood that the interface connection relationships between the modules illustrated in this embodiment are merely illustrative and do not constitute a structural limitation of the electronic device 300. In other embodiments, the electronic device 300 may also employ different interface connection methods or combinations of multiple interface connection methods as described in the above embodiments.

[0096] The charging management module 340 receives charging input (wired charging input and / or wireless charging input) from a charger to charge the battery 342. The charger can be either a wireless charger or a wired charger. While charging the battery 342, the charging management module 340 can also supply power to the electronic device via the power management module 341.

[0097] The power management module 341 connects the battery 342, the charging management module 340, and the processor 310. The power management module 341 receives input from the battery 342 and / or the charging management module 340, and supplies power to the processor 310, internal memory 321, external memory, display screen 394, camera 393, and wireless communication module 360, etc. In some embodiments, the power management module 341 may also be located within the processor 310. In other embodiments, the power management module 341 and the charging management module 340 may also be located in the same device.

[0098] The wireless communication function of electronic device 300 can be implemented through antenna 1, antenna 2, mobile communication module 350, wireless communication module 360, modem processor, and baseband processor. Antenna 1 and antenna 2 are used to transmit and receive electromagnetic wave signals. In some embodiments, antenna 1 of electronic device 300 is coupled to mobile communication module 350, and antenna 2 is coupled to wireless communication module 360, enabling electronic device 300 to communicate with networks and other devices through wireless communication technology. The wireless communication technology may include Global Navigation Satellite System (GNSS), wireless local area networks (WLAN) (such as Wi-Fi network) technology, etc. The GNSS may include Global Positioning System (GPS), Global Navigation Satellite System (GLONASS), BeiDou Navigation Satellite System (BDS), Quasi-Zenith Satellite System (QZSS), and / or satellite based augmentation systems (SBAS), etc. For example, electronic device 300 can obtain real-time location information through positioning technologies such as GPS, BDS, or SBAS.

[0099] The mobile communication module 350 can provide wireless communication solutions, including 2G / 3G / 4G / 5G, for use on the electronic device 300. For example, the electronic device 300 can send emails to a server via the mobile communication module 350.

[0100] The wireless communication module 360 ​​can provide solutions for wireless communication applications on the electronic device 300, including WLAN (such as Wi-Fi networks), Bluetooth, GNSS, Near Field Communication (NFC), Infrared (IR), and Frequency Modulation (FM). For example, the electronic device 300 can obtain its real-time location information through GNSS positioning technology.

[0101] Electronic device 300 implements display functions through a GPU, display screen 394, and application processor. Electronic device 300 implements shooting functions through an ISP, camera 393, video codec, GPU, display screen 394, and application processor. External memory interface 320 can be used to connect an external memory card, such as a Micro SD card, to expand the storage capacity of electronic device 300. Internal memory 321 can be used to store computer executable program code, which includes instructions. Processor 310 executes various functional applications and data processing of electronic device 300 by running the instructions stored in internal memory 321. Electronic device 300 implements audio functions through an audio module 370, speaker 370A, receiver 370B, microphone 370C, headphone jack 370D, and application processor.

[0102] It should be noted that the methods in the following embodiments can all be implemented in the electronic device 300 having the above-described hardware structure.

[0103] In this embodiment, considering that the electronic device may verify user identity information for users whose faces are occluded, in addition to verifying user identity information for users whose faces are occluded, the electronic device, in order to support user identity verification even when the user's face is occluded, pre-captures an image including a portion of the user's face before starting the verification process. This image including the portion of the user's face is used for face recognition. Please refer to... Figure 6 This is a flowchart illustrating a face recognition method provided in an embodiment of this application. The face recognition method is applied in an electronic device, which includes a camera. Figure 6 As shown, the face recognition method may include S401-S406.

[0104] S401. The electronic device displays a first interface, which is used to set the face image for face recognition.

[0105] The electronic device can receive a user's command to set a facial image; in response to this command, a first interface is displayed. Thus, the user can set or register a facial image for facial recognition on the electronic device through the first interface. The first interface is used to set or register a facial image for facial recognition.

[0106] In this embodiment, the electronic device can provide users with multiple authentication methods, which may include at least one of facial recognition, password authentication, and fingerprint authentication. The user's operation of setting a facial image can be a user's selection of facial recognition authentication.

[0107] For example, let's take a mobile phone as an example. Figure 7 As shown in (a), in response to a user's security authentication settings, mobile phone 300 can display a security settings interface 510. The security settings interface 510 may include a face authentication option 511, a password authentication option 512, a fingerprint authentication option 513, and more settings options 514. The display screen of mobile phone 300 is a touchscreen, and mobile phone 300 can receive a user's click on the face authentication option 511 (i.e., the user's selection of face authentication). Then, mobile phone 300 can jump from the security settings interface 510 to the first interface 520. Figure 7 As shown in (b), the first interface 520 may include an image preview area 521 and a first input prompt message 522. The mobile phone 300 may display an image captured by the camera within the image preview area 521. The first input prompt message 522 is used to instruct the user to input a partial facial image; for example, the content of the first input prompt message 522 may be "Ensure proper facial occlusion (e.g., wearing a mask)".

[0108] Normal facial occlusion refers to obscuring only part of a user's facial features. These features can include eyebrows, eyes, nose, and mouth. For example, when a user is wearing a mask, normal facial occlusion might mean that the mask only covers the user's nose and mouth, but not their eyebrows and eyes.

[0109] In this embodiment of the application, in order to support the verification of user identity information when the user's face is occluded and / or the face is not occluded, the electronic device can provide the user with the option to set a partial face image and / or a complete face image.

[0110] Specifically, the first interface may include a first option and / or a second option. The first option triggers the electronic device to set partial facial features for face recognition. The second option triggers the electronic device to set complete facial features for face recognition.

[0111] Among them, a partial face image can refer to an image that includes the facial area around the eyes, while a complete face image can refer to an image that includes the entire facial area.

[0112] For example, before displaying the first interface, the electronic device may first display at least one of the first and second options in a pop-up window; then, after the user clicks the first or second option, the pop-up window is closed and the first interface is displayed.

[0113] For example, when mobile phone 300 displays the security settings interface 510, upon receiving a user's click on the face authentication option 511, a pop-up window 523 may first appear. Pop-up window 523 may include a first option 524 and a cancel option 525. The cancel option 525 is used to trigger the electronic device to capture a complete face image for face recognition. When mobile phone 300 receives a user's click on the first option 524 or the cancel option 525, it closes the pop-up window 523. Figure 8 As shown, the name of the first option 524 can be "Register Local Face".

[0114] S402. The electronic device captures a first image through a camera and displays the first image on a first interface.

[0115] The electronic device can activate its camera at the same time as it begins displaying the first interface, so as to capture a first image including the user and display the captured first image in real time on the first interface. The first image may include one or more images.

[0116] In this embodiment of the application, when the first interface includes a first option, the electronic device can respond to the user's click operation on the first option in the first interface and capture a first image through the camera. Wherein, the electronic device receiving the user's click operation on the first option indicates that a partial facial image of the user has been captured; that is, the first image is a partial facial image.

[0117] For example, when the mobile phone 300 receives a user's click on the first option 524, it closes the pop-up window 523; then, it displays the first interface 520 and starts the camera to capture the first image. Figure 9 As shown, the mobile phone 300 displays the captured first image in the image preview area 521 of the first interface 520.

[0118] S403. The electronic device extracts facial features from the first image to obtain first facial features and saves the first facial features; wherein, the first facial features are local facial features and the first facial features are used for face recognition.

[0119] While acquiring the first image, the electronic device extracts facial features from the first image to obtain the first facial features. The electronic device can then save the first facial features to a preset local feature set library or a preset complete feature set library. The preset local feature set library stores local facial features; the preset complete feature set library stores complete facial features.

[0120] In this embodiment, after obtaining the first facial feature, the electronic device determines whether the first facial feature is a local facial feature. If it is determined that the first facial feature is a local facial feature, the first facial feature can be saved to a preset local feature set library. If the electronic device determines that the first facial feature is a complete facial feature, the first facial feature can be saved to a preset complete feature set library.

[0121] When the electronic device displays a first option on a first interface (i.e., the first interface includes a first option), if it receives a user's click on the first option and determines that the first facial feature is a partial facial feature, it saves the first facial feature to a preset local feature set library. If it receives a user's click on the first option and determines that the first facial feature is a complete facial feature, the electronic device issues a first prompt message. The user's click on the first option triggers the electronic device to save the first facial feature to the preset local feature set library. The first prompt message instructs the user to input a partial facial image.

[0122] Furthermore, the electronic device issues a first prompt to alert the user to cover a portion of their face; then, the electronic device can continue to capture new first images through the camera until local facial features are extracted from the new first images.

[0123] For example, such as Figure 10 As shown in (a), after receiving a user's click on the first option 524, the mobile phone 300 displays the captured first image in the image preview area 521 of the first interface 520. If the first image includes a portion of the user's face, the first facial feature extracted by the mobile phone 300 from the first image is a local facial feature. Furthermore, the mobile phone 300 can save this first facial feature to a preset local feature set library and display a first success message on the first interface 520. The first success message is used to notify the user that the local facial feature has been successfully recorded; for example, the content of the first success message could be "Local facial feature recording successful".

[0124] For example, such as Figure 10As shown in (b), after receiving a user's click on the first option 524, the mobile phone 300 displays the captured first image in the image preview area 521 of the first interface 520. If the first image includes the user's complete face, the first facial feature extracted by the mobile phone 300 from the first image is a complete human face feature. Furthermore, the mobile phone 300 can display a first prompt message 526, for example, the content of the first prompt message 526 can be "Entry failed, please ensure proper face covering (e.g., wearing a mask)".

[0125] S404. When the electronic device receives a startup operation for the first application service, the electronic device captures a second image through the camera.

[0126] Electronic devices can provide users with multiple application services; these services may include functions provided by the electronic device itself (such as viewing pictures or unlocking), or functions provided by applications installed on the device (such as payment or money transfer). The first application service can be one of these services that requires successful user authentication to activate. When the electronic device uses facial recognition, upon receiving an activation command for the first application service, the device activates its camera to capture a second image. This second image may include one or more images.

[0127] For example, taking unlocking as an example with the first application service, after saving the first facial feature to a preset local feature set library, the mobile phone 300 receives the user's screen lock operation and locks the display. Then, the mobile phone 300 receives the user's screen light-up operation, which is the initiation operation for unlocking. Figure 11 As shown, in response to the screen-on operation, the mobile phone 300 displays the lock screen interface 530 and captures a second image through the camera.

[0128] S405. The electronic device extracts facial features from the second image to obtain second facial features, and compares the second facial features with the first facial features; wherein, the second facial features are local facial features.

[0129] While acquiring the second image, the electronic device extracts facial features from the second image to obtain the second facial features. Then, the electronic device can calculate the matching degree between the second facial features and the first facial features in a preset local feature set library. The matching degree between the second facial features and the first facial features in the preset local feature set library is the first matching degree. Then, based on whether the first matching degree is greater than a first preset threshold, it is determined whether the second facial features match the first facial features in the preset local feature set library.

[0130] S406. If the second facial feature matches the first facial feature, the electronic device starts the first application service.

[0131] When the electronic device determines that the second facial feature matches the first facial feature in a preset local feature set library when the first matching degree is greater than a first preset threshold, it then initiates the first application service. If the electronic device determines that the second facial feature does not match the first facial feature in the preset local feature set library when the first matching degree is less than or equal to the first preset threshold, it does not initiate the first application service; it may also issue a third prompt message. The third prompt message is used to inform the user that facial verification has failed or to prompt the user to initiate the first application service through a preset authentication method. The preset authentication methods include password authentication and / or fingerprint authentication.

[0132] For example, mobile phone 300 displays as follows Figure 11 While the lock screen interface is displayed, if the second facial feature matches the first facial feature in the preset local feature set library, the phone 300 initiates unlocking and jumps from the lock screen interface 530 to the main interface 540 before the phone 300 enters the lock screen. For example... Figure 12 As shown in (a), the content of the main interface 540 may include the following applications (APP): clock, gallery, email, camera, contacts, and phone.

[0133] For example, mobile phone 300 displays as follows Figure 11 While displaying the lock screen interface, if the second facial feature does not match the first facial feature in the preset local feature set library, the phone 300 will not initiate unlocking and will display a third prompt message on the lock screen interface 530. For example... Figure 12 As shown in (b), the content of the third prompt message may include "Click here or swipe the screen to try again".

[0134] Understandably, the electronic device displays a first interface. This first interface is used to set a facial image for face recognition, meaning the user can set their facial image for face recognition through this interface. Then, the electronic device captures the first image using a camera and extracts a first facial feature from it. This first facial feature is a localized facial feature, and the electronic device also saves this first facial feature, indicating that the user can set a localized facial feature through the first interface. Then, when the electronic device receives a request to launch a first application service, it captures a second image and extracts a second facial feature from it. If the second facial feature matches the first facial feature (i.e., the saved localized facial feature), the electronic device launches the first application service. Therefore, by displaying the first interface, allowing the user to set a localized facial feature for face recognition, rather than obtaining the feature after multiple failed user identity verifications due to facial occlusion, the stability of face recognition technology is improved when the user's face is occluded.

[0135] Secondly, the second facial feature is a local facial feature, and when the second facial feature matches the first facial feature, the electronic device starts the first application service, which means that facial recognition technology can be used even when the user's face is occluded, thus improving the usability of the facial recognition technology provided by the electronic device.

[0136] In this embodiment, since different application services in an electronic device involve information with varying degrees of privacy, the electronic device can set different security levels for different application services. For example, the security level for payment is higher than the security level for unlocking. Secondly, considering that the more facial features used, the higher the accuracy of verifying user identity information, the electronic device can also set different security levels for partial facial features and complete facial features. For example, the security level corresponding to partial facial features is lower than (or not higher than) the security level corresponding to complete facial features.

[0137] Specifically, electronic devices can have multiple security levels preset for application services; electronic devices can also have a first security level preset for local facial features.

[0138] It should be noted that the electronic device sets different security levels for partial and complete facial features. Specifically, this could involve setting different security levels for a preset partial feature set library and a preset complete feature set library. After receiving a user's click on the aforementioned first option, the electronic device obtains a first facial feature. If the first facial feature is a complete facial feature, and the electronic device saves it to the preset partial feature set library, then the electronic device obtains a second facial feature that is also a complete facial feature. It can then determine that the second facial feature matches the first facial feature in the preset partial feature set library. Furthermore, if the second facial feature matches the first facial feature in the preset partial feature set library, the electronic device determines that the security level of the second facial feature is equal to the security level of the preset partial feature set library (i.e., the security level of the partial facial feature). However, if the second facial feature is a complete facial feature, and the security level of a complete facial feature should be higher than the security level of the preset partial feature set library, then the electronic device has determined an incorrect security level for the second facial feature.

[0139] Secondly, if the first facial feature is a complete human face feature, and the electronic device saves the first facial feature to a preset local feature set library, while the electronic device also saves a complete human face feature in a preset complete feature set library, then the electronic device determines that the second facial feature matches both the first facial feature in the preset local feature set library and the complete facial feature in the preset complete feature set library. Therefore, it cannot determine the security level of the second facial feature. In other words, if the electronic device receives a user's click on the first option and determines that the first facial feature is a complete human face feature, saving the first facial feature to the preset local feature set library will lead to a security level confusion issue.

[0140] In this embodiment, after the electronic device determines that the second facial feature matches the first facial feature, it must also determine that the security level of the first facial feature is higher than the security level of the first application service before starting the first application service. Specifically, the electronic device determines whether the security level of the first application service is lower than (or not higher than) the security level of the first facial feature. If the second facial feature matches the first facial feature, and the security level of the first application service is lower than (or not higher than) the security level of the first facial feature, then the electronic device starts the first application service. If the second facial feature matches the first facial feature, and the security level of the first application service is higher than or equal to the security level of the first facial feature, then the electronic device issues a second prompt message.

[0141] The second prompt message is used to prompt the user to enter a complete facial image or to launch the first application service through a preset authentication method; the preset authentication method includes password authentication and / or fingerprint authentication. When the first facial feature is a partial facial feature, the security level of the first facial feature is the aforementioned preset first security level.

[0142] In this embodiment of the application, in addition to using the first security level as the security level of the first facial feature, the electronic device can also determine the security level of the first facial feature based on the matching degree between the first facial feature and the user's complete facial features.

[0143] When an electronic device is able to acquire a user's complete facial features, it calculates the matching degree between a first facial feature and the user's complete facial features; the matching degree between the first facial feature and the user's complete facial features is called a second matching degree. Based on the second matching degree and the first correspondence, the security level of the first facial feature is determined.

[0144] The first correspondence may include at least one security level that corresponds one-to-one with at least one matching degree interval. When the second matching degree belongs to a certain matching degree interval in the first correspondence, the security level of the first facial feature is equal to the security level corresponding to that certain matching degree interval.

[0145] In this embodiment, the electronic device can pre-capture images including a portion of the user's face, and also pre-capture images including the user's complete face. The image including the portion of the user's face is used for face recognition of users whose faces are occluded; the image including the complete face is used for face recognition of users whose faces are not occluded. Specifically, the first interface may include the aforementioned second option, which triggers the electronic device to set complete facial features for face recognition. In response to the user's click on the second option, the electronic device captures a third image through a camera and displays the third image on the first interface. Then, the electronic device extracts facial features from the third image to obtain third facial features and saves the third facial features to a preset complete feature set library.

[0146] The third facial feature is a complete set of facial features used for facial recognition. The user's click on the second option triggers the electronic device to save the third facial feature to a pre-defined complete feature set library.

[0147] Please refer to Figure 13 This application embodiment also provides a face recognition method, in which S401 may include S601. S402 may include S602. S403 may include S603-S605. S405 may include S607. S406 may include S608. After S601, the face recognition method may include S609-S616.

[0148] S601. The electronic device displays a first interface, which includes a first option and a second option.

[0149] In this process, the electronic device receives an operation from the user to set a facial image. In response to this operation, the electronic device can display a first interface including a first option and a second option; then, the electronic device receives a click operation from the user on either the first or second option, but does not display either option.

[0150] S602. When the electronic device receives a user's click operation on the first option in the first interface, it captures the first image through the camera and displays the first image on the first interface.

[0151] The electronic device receives a click from the user on the first option, hides both the first and second options, and activates the camera to capture a first image. The electronic device also displays the captured first image in real-time on the first interface.

[0152] S603. The electronic device extracts facial features from the first image to obtain the first facial features.

[0153] In this process, the electronic device captures the first image and extracts facial features from the first image to obtain the first facial features.

[0154] S604. Electronic devices determine whether the first facial feature is a local facial feature.

[0155] If the electronic device determines that the first facial feature is a local facial feature, it executes S605, and then executes S404-S406. If the electronic device determines that the first facial feature is not a local facial feature, it does not execute S404-S406, but executes S606.

[0156] S605. The electronic device saves the first facial feature to a preset local feature set library.

[0157] When an electronic device determines that the first facial feature is a local facial feature, it saves the first facial feature to a preset local feature set library. The preset local feature set library is a database used to store local facial features.

[0158] S606. The electronic device issues a first prompt message, which is used to instruct the user to record a partial face image.

[0159] If the first facial feature is not a local facial feature, the electronic device will issue a first prompt message to indicate to the user that the local facial feature acquisition failed and that the local facial image needs to be re-entered. The first prompt message can be text or voice.

[0160] S607. The electronic device extracts facial features from the second image to obtain second facial features, and compares the second facial features with the first facial features in the preset local feature set library; the second facial features are local facial features.

[0161] After obtaining the second facial feature, the electronic device can directly compare the second facial feature with the first facial feature in a preset local feature set library. If the second facial feature matches the first facial feature in the preset local feature set library, the electronic device starts the first application service; if the second facial feature does not match the first facial feature in the preset local feature set library, the electronic device compares the second facial feature with the facial features in the aforementioned preset complete feature set library. If the second facial feature matches the facial features in the aforementioned preset complete feature set library, the electronic device starts the first application service; if the second facial feature does not match the facial features in the aforementioned preset complete feature set library, the electronic device does not start the first application service.

[0162] Alternatively, after obtaining the second facial feature, the electronic device determines whether the second facial feature is a local facial feature; if it is determined that the second facial feature is a local facial feature, it compares the second facial feature with the first facial feature in the preset local feature set library; if it is determined that the second facial feature is a complete facial feature, it compares the second facial feature with the facial features in the aforementioned preset complete feature set library.

[0163] S608. If the second facial feature matches the first facial feature in the preset local feature set library, the electronic device starts the first application service.

[0164] If the second facial feature matches the first facial feature in the preset local feature set library, it indicates that the local face image recognition is successful, and the electronic device starts the first application service.

[0165] S609. In response to the user's click on the second option, the electronic device captures a third image through the camera and displays the third image on the first interface.

[0166] The electronic device receives a click from the user on the second option, hides both the first and second options, and activates the camera to capture a third image. The device also displays the captured third image in real-time on the first interface.

[0167] S610: The electronic device extracts facial features from the third image to obtain the third facial features.

[0168] In this process, the electronic device extracts facial features from the third image while acquiring the third image, thus obtaining the third facial features.

[0169] S611. Electronic devices determine whether a third facial feature is a complete human face feature.

[0170] If the electronic device determines that the third facial feature is not a complete human face feature, proceed to step S612. If the electronic device determines that the third facial feature is a complete human face feature, proceed to steps S613-S616.

[0171] S612, The electronic device issues a fourth prompt message, which is used to instruct the user to enter a complete face image.

[0172] If the third facial feature is not a complete facial feature, the electronic device will issue a fourth prompt message to indicate that the acquisition of the complete partial facial feature has failed and a complete facial image needs to be re-entered. The fourth prompt message can be text or voice.

[0173] S613, The electronic device saves the third facial features to a preset complete feature set library.

[0174] When an electronic device determines that a third facial feature is a complete human face feature, it saves the third facial feature to a preset complete feature set library. The preset complete feature set library is a database used to store complete human face features.

[0175] S614. When the electronic device receives a startup operation for the first application service, the electronic device captures a fourth image through the camera.

[0176] When an electronic device uses facial recognition, upon receiving a request to launch the first application service, it activates its camera to capture a fourth image. This fourth image may include one or more images.

[0177] S615. The electronic device extracts facial features from the fourth image to obtain the fourth facial feature, and compares the fourth facial feature with the third facial feature in the preset complete feature set library; the fourth facial feature is a complete human face feature.

[0178] While acquiring the fourth image, the electronic device extracts facial features from the fourth image to obtain the fourth facial feature. Then, the electronic device calculates the matching degree between the fourth facial feature and the third facial feature in a preset complete feature set library; this matching degree is called the third matching degree. The electronic device then determines whether the fourth facial feature matches the third facial feature in the preset complete feature set library based on whether the third matching degree is greater than a third preset threshold.

[0179] S616. If the fourth facial feature matches the third facial feature in the preset complete feature set library, the electronic device starts the first application service.

[0180] If the electronic device determines that the fourth facial feature matches the third facial feature in the preset complete feature set library when the third matching degree is greater than the third preset threshold, then it starts the first application service. If the electronic device determines that the fourth facial feature does not match the third facial feature in the preset complete feature set library when the third matching degree is less than or equal to the third preset threshold, then it does not start the first application service; it may also issue a fifth prompt message. The fifth prompt message is used to inform the user that facial verification failed or to prompt the user to start the first application service through a preset authentication method. The preset authentication methods include password authentication and / or fingerprint authentication.

[0181] For example, such as Figure 14As shown, before displaying the first interface 710, the mobile phone 300 can display a pop-up window 720, which includes a first option 721 and a second option 722. The first option 721 can be named "Register Partial Face," and the second option 722 can be named "Register Full Face." The first option 721 is used to trigger the electronic device to set a partial face image for face recognition. The second option 722 is used to trigger the electronic device to set a full face image for face recognition.

[0182] Then, when the phone 300 receives the user's click on the second option 722, it closes the pop-up window 720. The phone 300 then displays the first interface 710 and activates the camera to capture a third image. The phone 300 displays the captured third image in the image preview area 711 of the first interface 710. Figure 15 As shown in (a), the mobile phone 300 displays a second input prompt 712 on the first interface 710. The second input prompt 712 is used to instruct the user to input a complete face image. For example, the content of the second input prompt 712 includes "Input face" and "Ensure face is not obscured".

[0183] Secondly, after receiving the user's click on the second option 722, the mobile phone 300 displays the captured third image in the image preview area 711 of the first interface 710. If the third image includes the user's complete face, the third facial feature extracted by the mobile phone 300 from the third image is a complete facial feature. Furthermore, the mobile phone 300 can save this third facial feature to a preset complete feature set library and display a second success message on the first interface 710. The second success message is used to notify the user that the complete facial feature has been successfully entered; for example, the content of the second success message could be "Complete facial feature entered successfully," such as... Figure 15 As shown in (b) of the diagram.

[0184] In addition, such as Figure 15 As shown in (c), if the third image is an image that includes a portion of the user's face, then the third facial feature extracted by the mobile phone 300 from the third image is a local facial feature. Furthermore, the mobile phone 300 can display a fourth prompt message 713, the content of which can be "Enrollment failed, ensure face is not obscured".

[0185] Furthermore, taking unlocking as an example using the first application service, after saving the third facial feature to a preset complete feature set library, phone 300 receives the user's screen lock operation, locking the phone 300's display. Then, phone 300 receives the user's screen-on operation, which initiates the unlocking process. For example... Figure 16As shown, in response to the screen-on operation, mobile phone 300 displays the lock screen interface 730 and captures a fourth image through the camera. Mobile phone 300 extracts a fourth facial feature from the fourth image.

[0186] Finally, if the fourth facial feature matches the third facial feature in the preset complete feature set library, the phone 300 initiates unlocking, jumping from the lock screen interface 730 to the main interface 740 before the phone 300 enters the lock screen, as shown below. Figure 17 As shown in (a) above. If the fourth facial feature does not match the third facial feature in the preset complete feature set library, the phone 300 will not initiate unlocking and will display the fifth prompt message on the lock screen 730, such as... Figure 17 As shown in (b) above. The content of the fifth prompt message may include "Click here or swipe the screen to try again".

[0187] In this embodiment, to support user identity verification even when the face is occluded and / or unoccluded, the electronic device, in addition to providing the user with options to set a partial face image and / or a complete face image, can also determine whether the captured image is a partial or complete face image, and then save the partial or complete facial features based on the determination result. Thus, without user intervention, partial or complete facial features can be automatically acquired and saved. Specifically, please refer to... Figure 18 This application embodiment also provides a face recognition method, which may include S801-S806 after S402 and before S404. After S404, the face recognition method may include S807-S809.

[0188] S801. The electronic device extracts facial features from the first image to obtain the first facial features.

[0189] S802. Electronic devices determine whether the first facial feature is a local facial feature.

[0190] If the electronic device determines that the first facial feature is a local facial feature, it executes S803. If the electronic device determines that the first facial feature is not a local facial feature, it executes S804-S806.

[0191] S803, The electronic device saves the first facial feature to a preset local feature set library.

[0192] It should be noted that the specific processes of S801-S803 can be found in the detailed description of S603-S605 above, and will not be repeated here in the embodiments of this application.

[0193] S804. Electronic devices determine whether the first facial feature is a complete human face feature.

[0194] If the electronic device determines that the first facial feature is not a complete human face feature, it executes S805. If the electronic device determines that the first facial feature is a complete human face feature, it executes S806.

[0195] S805, The electronic device issues a sixth prompt message, which is used to indicate that the user's face registration has failed.

[0196] If the first facial feature is neither a partial nor a complete facial feature, the electronic device will issue a sixth prompt message to indicate that the facial feature acquisition has failed and the user needs to re-enter the facial image. This sixth prompt message can be text or voice.

[0197] S806. The electronic device saves the first facial feature to a preset complete feature set library.

[0198] When the electronic device determines that the first facial feature is a complete human face feature, it saves the first facial feature to the aforementioned preset complete feature set library. This preset complete feature set library is a database used to store complete human face features.

[0199] S807. The electronic device extracts facial features from the second image to obtain the second facial features.

[0200] S808, the electronic device compares the second facial feature with the first facial feature in the preset local feature set library, or compares the second facial feature with the first facial feature in the preset complete feature set library.

[0201] The second facial feature may be a local facial feature or a complete facial feature; therefore, the electronic device can directly compare the second facial feature with the first facial feature in the preset local feature set library, or compare the second facial feature with the first facial feature in the preset complete feature set library.

[0202] S809. If the second facial feature matches the first facial feature in the preset local feature set library, or if the second facial feature matches the first facial feature in the preset complete feature set library, then the electronic device starts the first application service.

[0203] For example, taking a mobile phone as an electronic device, such as Figure 19As shown in (a), in response to a user's security authentication settings, mobile phone 300 can display a security settings interface 900. The security settings interface 900 may include a face authentication option 901, a password authentication option 902, a fingerprint authentication option 903, and more settings options 904. Mobile phone 300 has a touchscreen display, and it can receive a user's click on the face authentication option 901 (i.e., the user's selection of face authentication), navigating from the security settings interface 900 to the first interface 910. Mobile phone 300 also captures a first image via a camera. Figure 19 As shown in (b), the mobile phone 300 can display an image preview area 911 and a third input prompt message 912 on the first interface 910. The image preview area 912 displays the first image captured by the mobile phone 300 through its camera. The third input prompt message 912 instructs the user to input a facial image; for example, the content of the third input prompt message 912 may include "Input face" and "Ensure your face is displayed in the recognition area".

[0204] Then, the mobile phone 300 extracts the first facial feature from the first image. If the first image includes a portion of the user's face, indicating that the first facial feature is a local facial feature, the mobile phone 300 saves the first facial feature to a preset local feature set library and displays a first success message on the first interface 910. Figure 20 As shown in (a), the content of the first success message can be "Partial facial feature recording successful". If the first image is an image including the user's complete face, indicating that the first facial feature is a complete human face feature, then the mobile phone 300 saves the first facial feature to a preset complete feature set library and displays the second success message on the first interface 910. Figure 20 As shown in (b) above, the content of the second success message can be "Complete facial feature entry successful".

[0205] Additionally, if the first image is an image that does not include a partial or complete view of the user's face, indicating that the first facial feature is neither a partial nor a complete facial feature, then the mobile phone 300 can display the sixth prompt message 913. For example... Figure 20 As shown in (c), the content of the sixth prompt message 913 can be "User face registration failed. Ensure the face is within the camera's field of view."

[0206] Furthermore, taking unlocking as an example using the first application service, after saving the first facial features to a preset local feature set library or a preset complete feature set library, the phone 300 receives the user's screen lock operation, thus locking the phone 300's display screen. Then, the phone 300 receives the user's screen-on operation, which initiates the unlocking process. For example... Figure 21As shown, in response to the screen-on operation, the mobile phone 300 displays the lock screen interface 920 and captures a second image through the camera, extracting the second facial features from the second image.

[0207] Finally, if the second facial feature matches the first facial feature in a preset local feature set library or a preset complete feature set library, then the phone 300 initiates unlocking, jumping from the lock screen interface 920 to the main interface 930 before the phone 300 enters the lock screen. Figure 22 As shown in (a) above. If the second facial feature does not match the first facial feature in the preset local feature set library or the preset complete feature set library, the phone 300 will not initiate unlocking and will display the fifth prompt message on the lock screen 920, such as... Figure 22 As shown in (b) above. The content of the fifth prompt message may include "Click here or swipe the screen to try again".

[0208] In this embodiment of the application, the electronic device may use 2D face recognition technology or 3D face recognition technology to acquire images and extract facial features.

[0209] For example, taking a first image and 2D face recognition technology as examples, the specific process of an electronic device acquiring a first image and extracting a first facial feature from the first image using 2D face recognition technology may include: acquiring a first image through a regular camera; the first image is a two-dimensional three-channel color (Red Green Blue, RGB) image; preprocessing the first image (e.g., filtering, brightness adjustment, noise reduction, etc.) to obtain a preprocessed first image; using a face detection algorithm (e.g., Haar wavelet method, multi-task cascaded convolutional networks (MTCNN)) to perform face detection on the preprocessed first image and determine the face region; and using a preset feature extraction model to extract facial features from the face region to obtain a first facial feature, which is a 2D face feature.

[0210] Here, a standard camera refers to a camera that captures two-dimensional images. The preset feature extraction model can refer to a neural network model trained using sample images; the sample images can include images of the user's complete face and / or images of parts of the user's face.

[0211] It should be noted that the specific process of the electronic device using 2D face recognition technology to collect and extract facial features from any of the second, third, and fourth images can be found in the detailed description above of the electronic device using 2D face recognition technology to collect the first image and extract the first facial features from the first image. This embodiment will not repeat the details here. Furthermore, the specific process described above for the electronic device using 2D face recognition technology to collect the first image and extract the first facial features from the first image is merely an example; this embodiment does not limit the specific process of the electronic device using 2D face recognition technology to collect the first image and extract the first facial features from the first image.

[0212] For example, taking a first image and 3D face recognition technology as an example, when an electronic device uses 3D face recognition technology, it acquires a first image through a specific camera; this first image is an RGB-D image, which includes an RGB image and a depth image. Then, preprocessing, 3D reconstruction, and face feature extraction are sequentially performed on the first image to obtain a first facial feature. This first facial feature is a 3D face feature, or it may be a 2D face feature and a depth feature.

[0213] Based on the different imaging principles of cameras, the specific cameras used in 3D face recognition technology can be divided into: 3D structured light, time-of-flight (TOF) technology, and binocular stereo vision.

[0214] It should be noted that the specific process of the electronic device using 3D face recognition technology to collect and extract facial features from any of the second, third, and fourth images can be found in the detailed description above of the electronic device using 3D face recognition technology to collect the first image and extract the first facial features from the first image. This embodiment will not repeat the details here. Furthermore, the specific process described above for the electronic device using 3D face recognition technology to collect the first image and extract the first facial features from the first image is merely an example; this embodiment does not limit the specific process of the electronic device using 3D face recognition technology to collect the first image and extract the first facial features from the first image.

[0215] It is understood that the above methods can be implemented by a face recognition device. To achieve the above functions, a face recognition device includes hardware structures and / or software modules corresponding to the execution of each function. Those skilled in the art should readily recognize that, based on the units and algorithm steps of the examples described in conjunction with the embodiments disclosed herein, the embodiments of this application can be implemented in hardware or a combination of hardware and computer software. Whether a function is executed in hardware or by computer software driving hardware depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of the embodiments of this application.

[0216] This application embodiment can divide the above-mentioned face recognition device and the like into functional modules according to the above method examples. For example, each function can be divided into a separate functional module, or two or more functions can be integrated into one processing module. The integrated module can be implemented in hardware or as a software functional module. It should be noted that the module division in this application embodiment is illustrative and only represents one logical functional division. In actual implementation, there may be other division methods.

[0217] When dividing each function into modules according to its corresponding function. Figure 23 A possible structural diagram of the electronic device involved in the above embodiments is shown. The electronic device 1000 includes: a camera 1001, a display module 1002, a data acquisition module 1003, a feature processing module 1004, and an execution module 1005.

[0218] The system includes a display module 1002 for displaying a first interface, which is used to set a facial image for face recognition; an acquisition module 1003 for acquiring a first image via a camera 1001; and a feature processing module 1004 for extracting facial features from the first image to obtain a first facial feature and saving the first facial feature, wherein the first facial feature is a local facial feature used for face recognition; the acquisition module 1003 is also used to acquire a second image via the camera 1001 when the electronic device receives a startup operation for the first application service; the feature processing module 1004 is also used to extract facial features from the second image to obtain a second facial feature and compare the second facial feature with the first facial feature, wherein the second facial feature is a local facial feature; and an execution module 1005 is used to start the first application service if the second facial feature matches the first facial feature.

[0219] In one possible implementation, the first interface includes a first option; wherein the first option is used to trigger the electronic device to set local facial features for face recognition. The aforementioned acquisition module 1003 is used to acquire a first image via the camera 1001, including: the acquisition module 1003, specifically used to acquire the first image via the camera 1001 in response to a user's click operation on the first option. The user's click operation on the first option triggers the electronic device to save the first facial features to a preset local feature set library. The aforementioned feature processing module 1004 is used to save the first facial features, including: the feature processing module 1004, specifically used to save the first facial features to a preset local feature set library.

[0220] In another possible implementation, the feature processing module 1004 is used to save the first facial feature to a preset local feature set library, including: the feature processing module 1004 is specifically used to determine that the first facial feature is a local facial feature, and then save the first facial feature to the preset local feature set library.

[0221] In another possible implementation, the execution module 1005 is further configured to issue a first prompt message if the first facial feature is a complete human face feature, the first prompt message being used to instruct the user to enter a partial human face image.

[0222] In another possible implementation, the first interface includes a second option; wherein the second option is used to trigger the electronic device to set complete facial features for face recognition. The acquisition module 1003 is also used to acquire a third image through the camera 1001 in response to the user's click operation on the second option; the display module 1002 is also used to display the third image on the first interface; the feature processing module 1004 is also used to extract facial features from the third image to obtain third facial features, and save the third facial features to a preset complete feature set library.

[0223] The third facial feature is a complete set of facial features used for facial recognition. The user's click on the second option triggers the electronic device to save the third facial feature to a pre-defined complete feature set library.

[0224] In another possible implementation, the feature processing module 1004 is specifically used to: if the first facial feature is a local facial feature, then save the first facial feature to a preset local feature set library; if the first facial feature is a complete facial feature, then save the first facial feature to a preset complete feature set library.

[0225] In another possible implementation, the feature processing module 1004 is used to compare the second facial features with the first facial features, including: the feature processing module 1004 is specifically used to compare the second facial features with the first facial features in a preset local feature set library.

[0226] In another possible implementation, the electronic device has multiple application service security levels pre-set. Specifically, the execution module 1005 is configured to activate the first application service if the second facial feature matches the first facial feature. This includes: the execution module 1005 specifically activates the first application service if the second facial feature matches the first facial feature and the security level of the first application service is lower than (or not higher than) the security level of the first facial feature.

[0227] In another possible implementation, the execution module 1005 is further configured to issue a second prompt message if the second facial feature matches the first facial feature and the security level of the first application service is higher than or equal to the security level of the first facial feature. The second prompt message prompts the user to enter a complete facial image or to launch the first application service through a preset authentication method; the preset authentication method includes password authentication and / or fingerprint authentication.

[0228] In another possible implementation, the security level of the first facial feature is a preset first security level; or, the security level of the first facial feature is determined based on the degree of matching between the first facial feature and the user's complete facial features.

[0229] This application also provides an electronic device, which may include a processor and a memory. The memory stores computer program code, including computer instructions; the processor executes the computer instructions, causing the electronic device to perform the various functions or steps performed by the mobile phone 300 or electronic device 1000 in the above method embodiments. Other hardware structures of the electronic device can be referenced from the above embodiments. Figure 5 The detailed description of the mobile phone 300 shown is not repeated here in this embodiment of the application.

[0230] This application also provides a computer-readable storage medium storing computer instructions, which are used in an electronic device (such as...). Figure 5 , Figures 7-12 , Figures 14-17 and Figures 19-22 The mobile phone 300 shown in any of the attached figures, or Figure 23When operated on the illustrated electronic device 1000, the electronic device performs the various functions or steps performed by the mobile phone 300 or the electronic device 1000 in the above method embodiments. For example, the computer-readable storage medium may be a read-only memory (ROM), a random access memory (RAM), a compact disc read-only memory (CD-ROM), magnetic tape, floppy disk, and optical data storage device, etc.

[0231] This application also provides a computer program product, including one or more instructions, which can be used in electronic devices (such as...). Figure 5 , Figures 7-12 , Figures 14-17 and Figures 19-22 The mobile phone 300 shown in any of the attached figures, or Figure 23 The electronic device 1000 shown is operated on, causing the electronic device to perform various functions or steps performed by the mobile phone 300 or the electronic device 1000 in the above method embodiment.

[0232] Through the above description of the embodiments, those skilled in the art can clearly understand that, for the sake of convenience and brevity, only the division of the above functional modules is used as an example. In actual applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above.

[0233] In the several embodiments provided in this application, it should be understood that the disclosed apparatus and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of modules or units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another apparatus, or some features may be ignored or not executed. Furthermore, the mutual coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between apparatuses or units may be electrical, mechanical, or other forms.

[0234] The units described as separate components may or may not be physically separate. A component shown as a unit can be one or more physical units; that is, it can be located in one place or distributed in multiple different locations. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0235] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0236] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a readable storage medium. Based on this understanding, the technical solutions of the embodiments of this application, essentially, or the parts that contribute to the prior art, or all or part of the technical solutions, can be embodied in the form of a software product. This software product is stored in a storage medium and includes several instructions to cause a device (which may be a microcontroller, chip, etc.) or processor to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, ROM, RAM, magnetic disks, or optical disks.

[0237] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions within the technical scope disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A face recognition method, characterized in that, Applied to an electronic device, the electronic device including a camera, the method includes: The electronic device displays a first interface, which is used to set a facial image for facial recognition. The electronic device captures a first image through the camera and displays the first image on the first interface; The electronic device extracts facial features from the first image to obtain a first facial feature and saves the first facial feature; wherein, the first facial feature is a local facial feature and is used for face recognition. When the electronic device receives a startup operation for the first application service, the electronic device captures a second image through the camera; The electronic device extracts facial features from the second image to obtain second facial features, and compares the second facial features with the first facial features; wherein, the second facial features are local facial features. The electronic device has multiple application service security levels pre-set; If the second facial feature matches the first facial feature, and the security level of the first application service is lower than the security level of the first facial feature, then the electronic device starts the first application service; wherein, the higher the degree of matching between the first facial feature and the user's complete facial features, the higher the security level of the first facial feature.

2. The method according to claim 1, characterized in that, The first interface includes a first option; wherein the first option is used to trigger the electronic device to set local facial features for face recognition; The electronic device captures a first image via the camera, including: The electronic device responds to the user's click on the first option by capturing the first image through the camera; wherein, the user's click on the first option triggers the electronic device to save the first facial feature to a preset local feature set library; The process of saving the first facial feature includes: The electronic device saves the first facial feature to the preset local feature set library.

3. The method according to claim 2, characterized in that, The electronic device saves the first facial feature to the preset local feature set library, including: If the electronic device determines that the first facial feature is a local facial feature, it saves the first facial feature to the preset local feature set library.

4. The method according to claim 3, characterized in that, The method further includes: If the first facial feature is a complete human face feature, the electronic device issues a first prompt message, which is used to instruct the user to enter a partial facial image.

5. The method according to any one of claims 1-4, characterized in that, The first interface includes a second option; wherein the second option is used to trigger the electronic device to set complete facial features for face recognition; The method further includes: In response to the user's click on the second option, the electronic device captures a third image through the camera and displays the third image on the first interface; The electronic device extracts facial features from the third image to obtain a third facial feature, and saves the third facial feature to a preset complete feature set library; wherein, the third facial feature is a complete human face feature, and the third facial feature is used for face recognition; The user's click on the second option triggers the electronic device to save the third facial feature to the preset complete feature set library.

6. The method according to claim 1, characterized in that, The process of saving the first facial feature includes: If the electronic device determines that the first facial feature is a local facial feature, it saves the first facial feature to a preset local feature set library. The method further includes: If the first facial feature is a complete human face feature, the electronic device will save the first facial feature to a preset complete feature set library.

7. The method according to any one of claims 2-4 and 6, characterized in that, The comparison of the second facial feature with the first facial feature includes: The electronic device compares the second facial feature with the first facial feature in the preset local feature set library.

8. The method according to claim 1, characterized in that, The method further includes: If the second facial feature matches the first facial feature, and the security level of the first application service is higher than or equal to the security level of the first facial feature, then the electronic device issues a second prompt message. The second prompt message is used to prompt the user to enter a complete facial image or to start the first application service through a preset authentication method; the preset authentication method includes password authentication and / or fingerprint authentication.

9. The method according to claim 1 or 8, characterized in that, The security level of the first facial feature is a preset first security level; Alternatively, the security level of the first facial feature is determined based on the degree of matching between the first facial feature and the user's complete facial features.

10. An electronic device, characterized in that, The electronic device includes: a camera, a display module, a data acquisition module, a feature processing module, and an execution module; The display module is used to display a first interface, which is used to set a face image for face recognition. The acquisition module is used to acquire a first image through the camera; The display module is further configured to display the first image on the first interface; The feature processing module is used to extract facial features from the first image to obtain a first facial feature and save the first facial feature; wherein, the first facial feature is a local facial feature and the first facial feature is used for face recognition. The acquisition module is also used to acquire a second image through the camera when the electronic device receives a startup operation for the first application service; The feature processing module is further configured to extract facial features from the second image to obtain second facial features, and compare the second facial features with the first facial features; wherein, the second facial features are local facial features. The electronic device has multiple application service security levels pre-set; the execution module is used to start the first application service if the second facial feature matches the first facial feature and the security level of the first application service is lower than the security level of the first facial feature; wherein, the higher the matching degree between the first facial feature and the user's complete facial features, the higher the security level of the first facial feature.

11. The electronic device according to claim 10, characterized in that, The first interface includes a first option; wherein the first option is used to trigger the electronic device to set local facial features for face recognition; The acquisition module is used to acquire a first image through the camera, including: The acquisition module is specifically used to acquire the first image through the camera in response to the user's click operation on the first option; wherein, the user's click operation on the first option is used to trigger the electronic device to save the first facial features to a preset local feature set library; The feature processing module is used to save the first facial feature, including: The feature processing module is specifically used to save the first facial feature to the preset local feature set library.

12. The electronic device according to claim 11, characterized in that, The feature processing module is used to save the first facial feature to the preset local feature set library, including: The feature processing module is specifically used to determine that the first facial feature is a local facial feature, and then save the first facial feature to the preset local feature set library.

13. The electronic device according to claim 12, characterized in that, The execution module is further configured to issue a first prompt message if the first facial feature is a complete human face feature, the first prompt message being used to instruct the user to input a partial human face image.

14. The electronic device according to any one of claims 10-13, characterized in that, The first interface includes a second option; wherein the second option is used to trigger the electronic device to set complete facial features for face recognition; The acquisition module is also used to acquire a third image through the camera in response to the user's click operation on the second option; The display module is also used to display the third image on the first interface; The feature processing module is further configured to extract facial features from the third image to obtain a third facial feature, and save the third facial feature to a preset complete feature set library; wherein, the third facial feature is a complete human face feature, and the third facial feature is used for face recognition; The user's click on the second option triggers the electronic device to save the third facial feature to the preset complete feature set library.

15. The electronic device according to claim 10, characterized in that, The feature processing module is specifically used for: If the first facial feature is determined to be a local facial feature, then the first facial feature is saved to a preset local feature set library; If the first facial feature is a complete human face feature, then the first facial feature is saved to a preset complete feature set library.

16. The electronic device according to any one of claims 11-13 and 15, characterized in that, The feature processing module is used to compare the second facial feature with the first facial feature, including: The feature processing module is specifically used to compare the second facial feature with the first facial feature in the preset local feature set library.

17. The electronic device according to claim 10, characterized in that, The execution module is further configured to issue a second prompt message if the second facial feature matches the first facial feature and the security level of the first application service is higher than or equal to the security level of the first facial feature. The second prompt message is used to prompt the user to enter a complete facial image or to start the first application service through a preset authentication method; the preset authentication method includes password authentication and / or fingerprint authentication.

18. The electronic device according to claim 10 or 17, characterized in that, The security level of the first facial feature is a preset first security level; Alternatively, the security level of the first facial feature is determined based on the degree of matching between the first facial feature and the user's complete facial features.

19. An electronic device, characterized in that, The electronic device includes: a processor and a memory; The memory is used to store computer program code, which includes computer instructions; the processor is used to execute the computer instructions, causing the electronic device to perform the method as described in any one of claims 1-9.

20. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer instructions that, when executed on an electronic device, cause the electronic device to perform the method as described in any one of claims 1-9.

Images