Facial recognition methods and related equipment

Abstract

This application provides a face recognition method and related equipment. The method includes: determining whether the current environment is a low-light environment based on the illuminance collected by an ambient light sensor during face recognition; if the current environment is determined to be a low-light environment, enhancing the image to be recognized captured by the imaging device; performing a similarity check between the facial features of the enhanced image and the facial features of a preset facial template image; if the similarity check passes, the user identity verification is confirmed to be successful; if the above check fails, the facial template image is degraded; performing a similarity check between the facial features of the image to be recognized and the facial features of the degraded image; if the similarity check passes, the user identity verification is confirmed to be successful; if both checks fail, the user identity verification is confirmed to be unsuccessful. This application can improve the accuracy of face recognition in low-light environments through low-light image enhancement strategies and template image degradation strategies.

Description

Technical Field

[0001] This application belongs to the field of terminal devices, and in particular relates to a face recognition method and related equipment. Background Technology

[0002] In the field of computer vision, facial recognition technology is a technique for authenticating users in an image based on facial features. Facial recognition technology has been applied in various scenarios. For example, it involves using the camera of a terminal device (such as a mobile phone, tablet, or security access control system) to capture real-time images of users, and then performing facial recognition on the captured images to authenticate the user. This allows users to easily unlock terminal devices or access specific functional interfaces using their faces, improving the user experience.

[0003] However, in certain environments (e.g., low-light scenes), the image quality captured by conventional shooting devices is poor, leading to reduced accuracy and the inability to unlock the device due to incorrect recognition, resulting in a poor user experience when using related technologies for facial recognition. Although additional auxiliary lighting devices can be added to terminal devices for facial recognition, this increases hardware costs. Summary of the Invention

[0004] In view of the above, it is necessary to provide a face recognition method and related equipment that can solve the problems of reduced face recognition accuracy and user experience caused by poor image quality captured by conventional shooting devices in specific environments.

[0005] In a first aspect, this application provides a face recognition method applied to a terminal device, the terminal device including a shooting device and an ambient light sensor, the method comprising: responding to a user operation that triggers face recognition, using the shooting device to acquire an image to be recognized, and using the ambient light sensor to obtain the illuminance of the current environment; if the current environment is determined to be a dark environment based on the illuminance, inputting the image to be recognized into a pre-trained image enhancement model to obtain an enhanced image of the image to be recognized; extracting a first face feature from the enhanced image using a preset face feature extraction model; if a first similarity between the first face feature and a second face feature in a preset face template image is greater than or equal to a preset similarity threshold, determining that the user's identity verification has passed.

[0006] Through the above technical solution, when performing face recognition using a terminal device without a supplementary light camera, the system can determine whether the current environment is dark based on the illuminance collected by the ambient light sensor of the terminal device, thereby determining whether the image to be recognized captured by the shooting device is a dark image; image enhancement is performed on the dark image captured in the dark environment to obtain an enhanced image with significantly reduced noise, appropriate brightness restoration, and no obvious damage to facial features; by performing feature matching between the facial features in the enhanced image and the facial features in the preset facial template image, the accuracy of face recognition in dark environments can be improved.

[0007] In one possible implementation, the method further includes: if the first similarity is less than the similarity threshold, performing image degradation processing on the face template image to obtain a degraded image of the face template image; extracting a third face feature from the image to be identified using the face feature extraction model, and extracting a fourth face feature from the degraded image using the face feature extraction model; if the second similarity between the third face feature and the four face features is greater than or equal to the similarity threshold, determining that the user's identity verification has passed; or, if the second similarity is less than the similarity threshold, determining that the user's identity verification has failed.

[0008] Using the above technical solution, if the face feature matching result using the enhanced image fails, the template image can be further degraded by adding noise from the low-light image to it. The degraded image is then used to perform a second matching attempt with the low-light image; that is, the face features of the unenhanced low-light image and the degraded template image are compared. By introducing a face template image degradation strategy, the false rejection rate (FRR) of face recognition in low-light scenes can be further reduced, improving the accuracy of face recognition and user experience.

[0009] In one possible implementation, the method further includes: using the ambient light sensor to collect multiple continuous illuminance values ​​of the current environment; determining the average value of the multiple continuous illuminance values ​​based on a moving average algorithm; and determining that the current environment is a dark environment if the average value is less than a preset illuminance threshold.

[0010] The above technical solution can determine the average value of multiple consecutive illuminance values ​​as the current ambient illuminance using a moving average algorithm. This avoids misjudging whether the current environment is a dark environment due to instantaneous increases or decreases in illuminance or fluctuations in ambient light sensors, thereby improving the accuracy of dark environment judgment.

[0011] In one possible implementation, the step of extracting the first facial feature from the enhanced image using a preset facial feature extraction model includes: determining whether a face to be identified exists in the enhanced image based on a preset facial detection model; if a face to be identified exists in the enhanced image, cropping a facial region image from the enhanced image; and extracting the first facial feature from the facial region image using the facial feature extraction model.

[0012] The above technical solution allows for the determination of whether a face exists in the enhanced image and the region where a face exists before performing facial feature recognition on the image to be enhanced. This enables targeted facial feature recognition in the facial region image, eliminating the need for the facial feature extraction model to consider features in regions that do not contain faces, reducing model computational power, and improving the efficiency and accuracy of facial feature extraction.

[0013] In one possible implementation, the step of extracting the first facial feature from the enhanced image using a preset facial feature extraction model further includes: if there is no face to be identified in the enhanced image, determining that the user's identity verification has failed.

[0014] By using the above technical solution, when it is determined that the enhanced image does not contain the face to be identified, there is no need to perform other unnecessary steps, such as calling the face feature extraction model to extract face features, thereby improving the efficiency of face recognition.

[0015] In one possible implementation, the method further includes: when the current environment is determined to be a non-dark environment based on the illumination, determining whether a face to be identified exists in the image to be identified based on a preset face detection model; if a face to be identified exists in the image to be identified, extracting a face region image from the image to be identified; extracting a fifth face feature from the face region image using the face feature extraction model; if the third similarity between the fifth face feature and the second face feature is determined to be greater than or equal to the similarity threshold, determining that the user's identity verification has passed; or, if the third similarity is determined to be less than the similarity threshold, determining that the user's identity verification has failed.

[0016] The above technical solution allows us to determine that the captured image is not a dark image when the current environment is not dark, thus eliminating the need for image enhancement or other processing. We can directly perform face recognition on the image and crop the face region from it. Then, we extract face features from the cropped face region and match them with the face features in the face template image to determine the face recognition result. This enables efficient and accurate face recognition even in non-dark environments.

[0017] In one possible implementation, the image enhancement model includes a first model and a second model. The step of inputting the image to be recognized into the pre-trained image enhancement model to obtain an enhanced image of the image to be recognized includes: using the first model to denoise the image to be recognized and outputting the denoised image to be recognized; using the second model to enhance the brightness of the denoised image to be recognized and using the output brightness-enhanced image as the enhanced image.

[0018] The above technical solution allows for the use of a first model to remove noise from a low-light image, followed by the use of a second model to enhance the brightness of the denoised low-light image, resulting in an enhanced image with significantly reduced noise, appropriate brightness restoration, and no significant damage to facial features.

[0019] In one possible implementation, the training method of the image enhancement model includes: acquiring multiple sample images, the multiple sample images including multiple first images and multiple second images, wherein the image noise of any first image is greater than the image noise of any second image; constructing a training set and a test set based on the multiple sample images; and using the training set to perform at least one iterative update on a preset initial model until the image enhancement model that meets the preset requirements is obtained.

[0020] The above technical solution addresses the objective limitation of the difficulty in collecting real paired datasets by employing a non-paired training strategy. This means that high-quality images corresponding to real low-light, low-quality images are not required as ground truth labels. In this approach, the first image represents a noisy, low-light, low-quality image, while the second image represents a high-quality image. There is no direct correspondence between the first and second images, thus reducing the difficulty of obtaining sample images for training the image enhancement model.

[0021] In one possible implementation, the preset requirement includes a combination of one or more of the following requirements: the loss function corresponding to the initial model reaches the convergence condition; the test results obtained by testing the initial model using the test set indicate that the model performance of the initial model meets expectations; and the number of the at least one iteration update reaches a preset iteration number threshold.

[0022] The above technical solutions can be used to determine whether the trained model meets the preset requirements in multiple ways, thereby improving the model's performance.

[0023] In one possible implementation, the initial model includes a first initial model, and any one of the at least one iterative updates includes: inputting any first image from the training set into the first initial model; using the first initial model to denoise the any first image to obtain a first denoised image corresponding to the any first image, and a first difference image between the any first image and the first denoised image; determining a first loss value corresponding to the first difference image using a preset first loss function, the first loss value being used to indicate the noise level in the first difference image; if the first loss value is less than or equal to a preset first threshold, using another first image to perform the next update on the first initial model.

[0024] In one possible implementation, any one of the at least one iterative updates further includes: if the first loss value is greater than the first threshold, fusing the first difference image with any second image in the training set to obtain a fused image corresponding to the second image; inputting the fused image into the first initial model, using the first initial model to denoise the fused image to obtain a second denoised image corresponding to the fused image; determining a second loss value between the second denoised image and the corresponding second image using a preset second loss function; if the second loss value is greater than a preset second threshold, using the next first image and the next second image to perform the next update on the first initial model.

[0025] The above technical solution allows any low-quality image A (such as the first image) to be processed by a denoising neural network (such as the first model) of an image enhancement model, resulting in a denoised image A and a difference map between the denoised image A and the low-quality image A (such as the first difference image, which is a pure noise map). By superimposing the pure noise map corresponding to the low-quality image A with any real high-quality image B (such as the second image), a low-quality image B can be synthesized (such as the fused image corresponding to the second image). The real high-quality image B can then be used as the enhancement target for synthesizing the low-quality image B to train the denoising neural network. That is, after denoising the synthesized low-quality image B using the denoising neural network, the resulting denoised image B should be as close as possible to the high-quality image B. In the training process of the first model, the denoising neural network performs denoising twice, with each denoising step corresponding to a loss function. Specifically, in the first denoising step, the low-quality image A is denoised to obtain a pure noise image. Therefore, the first loss function can be used to constrain the noise in this pure noise image, i.e., to measure the degree of noise presence in the pure noise image. In the second denoising step, the synthesized low-quality image B is denoised to obtain a denoised image B. The second loss function can be used to constrain the denoised image B, making it as close as possible to the high-quality image B. Through the above training method, the training difficulty of the first model can be reduced and the performance of the model can be improved based on the non-paired training strategy.

[0026] In one possible implementation, the initial model further includes a second initial model, and any one of the at least one iterative updates further includes: if the second loss value is less than or equal to the second threshold, inputting the second denoised image corresponding to any second image into the second initial model, using the second initial model to enhance the brightness of the second denoised image, obtaining a brightness-enhanced image corresponding to the second denoised image; using a preset third loss function to determine a third loss value between the brightness-enhanced image and the corresponding second image; if the third loss value is greater than a preset third threshold, using the second denoised image corresponding to the next second image to perform the next update on the second initial model; or, if the third loss value is less than or equal to the third threshold, determining that the initial model meets the preset requirement, and using the initial model as the image enhancement model.

[0027] The above technical solution allows the use of a third loss function to constrain the brightening neural network of the image enhancement model (such as the second model), so that the denoised image B before brightening (such as the second denoised image) is as close as possible to the original high-quality image B after passing through the brightening neural network, thereby improving the performance of the brightening neural network.

[0028] Furthermore, by iteratively updating the first and second models using the aforementioned technical solutions, the performance of the models can be continuously optimized, resulting in image enhancement models that meet the usage requirements.

[0029] In one possible implementation, the step of performing image degradation processing on the face template image to obtain a degraded image of the face template image includes: acquiring a preset third image; inputting the third image into the image enhancement model, using a first model of the image enhancement model to denoise the third image, obtaining a third denoised image corresponding to the third image, and a second difference image between the third image and the third denoised image; reducing the brightness of the face template image to obtain a brightness-reduced image; and fusing the second difference image with the brightness-reduced image to obtain the degraded image of the face template image.

[0030] The above technical solution eliminates the need for separately training a degradation neural network for image degradation. Instead, an image enhancement model can directly obtain a pure noise map corresponding to a low-quality image D (e.g., the third image). This pure noise map is then used as a noise image (e.g., the second difference image) for degrading the face template. With proper and sufficient training of the denoising model (e.g., the first model) in the aforementioned image enhancement model, the degraded image will possess very realistic noise interference, exhibiting a noise distribution similar to the real low-quality image D. At this point, facial feature extraction is performed on the degraded image, and a matching algorithm is used to compare the facial features in the face template image with those in the degraded image. This can mitigate the impact of noise to some extent, focus on differences in facial features, and thus improve the accuracy of face recognition.

[0031] Secondly, this application provides a terminal device, the terminal device including a memory and a processor: wherein the memory is used to store program instructions; the processor is used to read and execute the program instructions stored in the memory, and when the program instructions are executed by the processor, the terminal device performs the above-described face recognition method.

[0032] Thirdly, this application provides a chip coupled to a memory in a terminal device, the chip being used to control the processor of the terminal device to execute the above-described face recognition method.

[0033] Fourthly, this application provides a computer storage medium storing program instructions that, when executed on a terminal device, cause the processor of the terminal device to perform the aforementioned face recognition method.

[0034] Furthermore, the technical effects brought about by the second to fourth aspects can be found in the descriptions of the methods in the above-mentioned method section, and will not be repeated here. Attached Figure Description

[0035] Figure 1This is a schematic diagram of a low-light image and a related solution provided in an embodiment of this application.

[0036] Figure 2 This is a software architecture diagram of a terminal device provided in an embodiment of this application.

[0037] Figure 3 This is a flowchart of a face recognition method provided in an embodiment of this application.

[0038] Figure 4 This is a flowchart of a method for determining whether the current environment is a dark environment, provided in an embodiment of this application.

[0039] Figure 5 This is a flowchart of a training method for an image enhancement model provided in an embodiment of this application.

[0040] Figure 6 This is a schematic diagram of the application inference stage of an image enhancement model provided in an embodiment of this application.

[0041] Figure 7 This is a flowchart of a method for image degradation processing of a face template image provided in an embodiment of this application.

[0042] Figure 8 This is a schematic diagram of a template image degradation strategy provided in an embodiment of this application.

[0043] Figure 9 This is a schematic diagram of a non-paired training strategy provided in an embodiment of this application.

[0044] Figure 10 This is a flowchart of a method for updating a first initial model according to an embodiment of this application.

[0045] Figure 11 This is a flowchart of a method for updating a second initial model in any way, as provided in an embodiment of this application.

[0046] Figure 12 This is a flowchart of a method for face recognition of high-quality images provided in an embodiment of this application.

[0047] Figure 13 This is a schematic diagram of the overall architecture of a face recognition method provided in an embodiment of this application.

[0048] Figure 14 This is a flowchart of a face recognition method provided in an embodiment of this application.

[0049] Figure 15 This is a hardware architecture diagram of a terminal device provided in an embodiment of this application. Detailed Implementation

[0050] In one embodiment of this application, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of indicated technical features. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more features. In the description of embodiments of this application, words such as "exemplary" or "for example" are used to indicate examples, illustrations, or descriptions. Any embodiment or design described as "exemplary" or "for example" in one embodiment of this application should not be construed as being more preferred or advantageous than other embodiments or designs. Specifically, the use of words such as "exemplary" or "for example" is intended to present the relevant concepts in a concrete manner.

[0051] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application pertains. The terminology used in this application's specification is for the purpose of describing particular embodiments only and is not intended to limit the application. It should be understood that, unless otherwise stated, " / " in this application means "or". For example, A / B can mean A or B. "And / or" in this application is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A alone, A and B simultaneously, and B alone. "At least one" refers to one or more. "More than one" refers to two or more. For example, at least one of a, b, or c can represent: a, b, c, a and b, a and c, b and c, and a, b, and c. Where there is no conflict, the following embodiments and features described herein can be combined with each other.

[0052] In the field of computer vision, facial recognition technology is a technique for authenticating users in an image based on facial features. Facial recognition technology has been applied in various scenarios. For example, it involves using the camera of a terminal device (such as a mobile phone, tablet, or security access control system) to capture real-time images of users, and then performing facial recognition on the captured images to authenticate the user. This allows users to easily unlock terminal devices or access specific functional interfaces using their faces, improving the user experience.

[0053] However, in certain environments (e.g., low-light scenes), the image quality captured by conventional shooting devices is poor, leading to reduced accuracy and the inability to unlock the device due to incorrect recognition, resulting in a poor user experience when using related technologies for facial recognition. Although additional auxiliary devices can be added to terminal devices for facial recognition, this increases hardware costs.

[0054] See Figure 1 The image shown is a schematic diagram of a low-light image and a related solution provided in an embodiment of this application. For example, if the current environment has insufficient lighting, and the terminal device only has a conventional front-facing camera without other auxiliary equipment (such as a Time of Flight (TOF) sensor or other supplementary lighting), the low-light image captured by the camera is often dark and filled with significant noise. Figure 1 In the low-light image shown in Figure (a), only a very blurry outline of the human body is visible. Although the face has been pixelated, considering the blurred outline of the human body, the subtle details of the face are even more difficult to discern. In this situation, if only... Figure 1 As shown in Figure (b), simply increasing the brightness of an image results in an image with increased noise, which distorts facial features and makes accurate face recognition difficult. Related technologies, such as... Figure 1 As shown in Figure (c), using the maximum screen brightness for supplemental lighting not only results in limited and poor supplemental lighting, but also leads to extremely glaring brightness in low-light scenarios, resulting in a poor user experience; furthermore, as... Figure 1 As shown in Figure (d), adding additional auxiliary equipment to the terminal device, such as adding an infrared fill light, will increase the hardware cost significantly, leading to an increase in overall cost.

[0055] To address the aforementioned issues, this application provides a face recognition method. When performing face recognition on a terminal device without a supplementary lighting camera, the method can determine whether the current environment is dark based on the illuminance collected by the terminal device's ambient light sensor. The method then enhances the dark-light image captured in this environment, resulting in an enhanced image with significantly reduced noise, appropriate brightness restoration, and no significant damage to facial features. By performing feature matching between the facial features in the enhanced image and the facial features in a preset face template image, the accuracy of face recognition in dark environments can be improved. The specific process of the face recognition method will be discussed in conjunction with... Figure 3 The process shown is explained in detail below.

[0056] See Figure 2 The diagram shown is a software architecture diagram of a terminal device provided in an embodiment of this application. The layered architecture divides the software into several layers, each with a clear role and function. Layers communicate with each other through software interfaces. For example, the Android system, from top to bottom, consists of the application layer 101, framework layer 102, Android runtime and system libraries 103, hardware abstraction layer 104, kernel layer 105, and hardware layer 106.

[0057] Application layer 101 may include a series of application packages. For example, application packages may include applications such as camera, gallery, calendar, calling, map, navigation, WLAN, Bluetooth, music, video, SMS, device control services, etc.

[0058] The framework layer 102 provides an Application Programming Interface (API) and programming framework for applications in the application layer. The application framework layer includes predefined functions. For example, it may include a window manager, content provider, view system, phone manager, resource manager, notification manager, etc.

[0059] The window manager manages window programs. It can obtain screen size, determine the presence of a status bar, lock the screen, and capture screenshots. The content provider stores and retrieves data, making it accessible to applications. This data can include videos, images, audio, made and received calls, browsing history and bookmarks, phone books, etc. The view system includes visual controls, such as controls for displaying text and controls for displaying images. The view system can be used to build applications. The display interface can consist of one or more views. For example, a display interface including a text notification icon can include views for displaying text and views for displaying images. The phone manager provides communication functionality for terminal devices, such as managing call status (including connection, hang-up, etc.). The resource manager provides applications with various resources, such as localized strings, icons, images, layout files, and video files. The notification manager allows applications to display notification information in the status bar. It can be used to convey informational messages and can disappear automatically after a short pause without user interaction. For example, the notification manager is used to notify of download completion or message alerts. The notification manager can also display notifications as icons or scrolling text in the system's top status bar, such as notifications from background applications, or as dialog boxes on the screen. Examples include displaying text messages in the status bar, emitting alert sounds, causing the device to vibrate, and flashing indicator lights.

[0060] The Android Runtime consists of the core libraries and the virtual machine. The Android runtime is responsible for the scheduling and management of the Android system. The core libraries consist of two parts: one part contains the functionalities that the Java language needs to call, and the other part contains the core Android libraries.

[0061] Application layer 101 and framework layer 102 run in a virtual machine. The virtual machine executes the Java files of the application layer and framework layer as binary files. The virtual machine is used to perform functions such as object lifecycle management, stack management, thread management, security and exception management, and garbage collection.

[0062] System library 103 may include multiple functional modules. For example, a surface manager, media libraries, 3D graphics processing libraries (e.g., OpenGL ES), 2D graphics engines (e.g., SGL), etc.

[0063] The Surface Manager manages the display subsystem and provides fusion of 2D and 3D layers for multiple applications. The Media Library supports playback and recording of various common audio and video formats, as well as still image files. It supports multiple audio and video encoding formats, such as MPEG4, H.264, MP3, AAC, AMR, JPG, and PNG. The 3D Graphics Processing Library implements 3D graphics drawing, image rendering, compositing, and layer processing. The 2D Graphics Engine is the drawing engine for 2D graphics.

[0064] Hardware Abstraction Layer 104 runs in user space, encapsulates kernel-level drivers, and provides calling interfaces to the upper layers.

[0065] Kernel layer 105 is the layer between hardware and software. Kernel layer 105 contains at least the display driver, touch driver, audio driver, and sensor driver.

[0066] Kernel layer 105 is the core of the operating system for terminal devices. It is the first layer of software extension based on the hardware, providing the most basic functions of the operating system. It is the foundation for the operation of the operating system, responsible for managing system processes, memory, device drivers, files, and network systems, and determining the system's performance and stability. For example, the kernel layer can determine the timing of an application's operation on certain parts of the hardware.

[0067] Kernel layer 105 includes hardware-dependent programs such as interrupt handlers and device drivers, as well as basic, common, and frequently running modules such as clock management and process scheduling modules, and critical data structures. The kernel layer can be located within the processor or embedded in internal memory.

[0068] Hardware layer 106 includes the hardware of the terminal device, such as a display screen, buttons, camera, ambient light sensor, etc. Hardware layer 106 may not include auxiliary lighting devices, such as time-of-flight sensors or other supplementary lights.

[0069] See Figure 3The diagram shown is a flowchart of a face recognition method provided in an embodiment of this application. The face recognition method is applied in a terminal device and includes the following steps.

[0070] S101, in response to a user operation that triggers facial recognition, uses a camera to capture an image to be recognized and uses an ambient light sensor to obtain the current ambient light level.

[0071] In one embodiment of this application, the user operation used to trigger facial recognition may vary depending on the application scenario of the facial recognition method. For example, when the facial recognition method is applied to environments such as security access control, the user operation may include scanning their face in front of the access control system or security camera to verify their identity; when the facial recognition method is applied to scenarios such as unlocking terminal devices, the user operation may include using facial recognition to unlock devices such as mobile phones, tablets, or laptops, or using facial recognition to unlock applications (APPs) of the terminal device; when the facial recognition method is applied to environments such as payment and financial services, the user operation may include using facial recognition for payment verification or identity verification to access bank accounts. For example, when an application needs to verify the user's identity, it will prompt the user to perform facial recognition on an application interface. The user can trigger the operation of recognizing their face by clicking on the relevant icon or control on the application interface. The user operation can also be a preset gesture operation. For example, when the terminal device is a mobile phone, when the user raises the phone to face the screen of the terminal device, the camera device of the terminal device can start scanning and comparing the user's face. The above are only examples and are not limited to these in actual applications.

[0072] In one embodiment of this application, the image to be identified can be a multi-channel image. For example, a multi-channel image can be an RGB image where each pixel includes three color channels: red (R), green (G), and blue (B). The image to be identified can also be a grayscale image or an image in other formats, and this application does not impose specific limitations on this.

[0073] In one embodiment of this application, without the aid of auxiliary lighting equipment, the environment in which the imaging device of the terminal device captures the image to be recognized may be a dark environment. Dark images captured in a dark environment have low image quality and cannot be directly used for face recognition. Therefore, an ambient light sensor can be used to obtain the current ambient illuminance, and then the ambient illuminance can be used to determine whether the current environment is a dark environment. An ambient light sensor is a sensor used to sense the light intensity (hereinafter referred to as illuminance) of the surrounding environment. Terminal devices (such as mobile phones, laptops, tablets, etc.) are usually equipped with ambient light sensors. Ambient light sensors are generally used to inform the processing chip of the terminal device of the ambient illuminance so that the display brightness of the terminal device's screen can be automatically adjusted, thereby reducing the power consumption of the screen.

[0074] S102, if the current environment is determined to be a low-light environment based on the illumination, the image to be identified is input into a pre-trained image enhancement model to obtain the enhanced image of the image to be identified.

[0075] In one embodiment of this application, a dark environment refers to an environment with insufficient illumination, such as an environment where the illumination is below a preset illumination / illuminance threshold, for example, 0.005 lux. When determining whether the current environment is a dark environment based on illumination, since the illumination in the environment may change at any time, and due to fluctuations in ambient light sensors, the average of multiple consecutive illumination values ​​can be used as the illumination of the current environment. This avoids misjudging whether the current environment is a dark environment due to instantaneous increases or decreases in illumination, and improves the accuracy of dark environment determination. The method for determining whether the current environment is a dark environment can also be found in the following section... Figure 4 Description of the illustrated embodiment.

[0076] In one embodiment of this application, the image to be identified, captured in a low-light environment, can be referred to as a low-light image, low-illumination image, low-quality image, etc. Low-light images generally have low image quality, exhibiting issues such as missing image details, low contrast, and significant noise. This results in the inclusion of much useless and interfering information, severely impacting the accuracy of face detection and facial feature extraction algorithms during face recognition. By using a pre-trained image enhancement model and invoking the low-light image enhancement algorithm provided in this embodiment, the low-light image can be enhanced to eliminate noise, thereby improving the accuracy and reliability of face recognition and performing the first facial feature matching on the image to be identified.

[0077] In one embodiment of this application, the image enhancement model includes a first model and a second model. The first model may also be referred to as a denoising neural network for removing image noise in a low-light image; the second model may also be referred to as a brightening neural network for adaptively enhancing image brightness.

[0078] In related technologies, when training an image augmentation model, it is usually necessary to use real paired training samples to train the image augmentation model. Since each pair of training samples needs to include a low-quality image m and a high-quality image M corresponding to the low-quality image m, it is difficult to collect a large number of paired training samples to construct a sample set.

[0079] To address the aforementioned issues, this application provides a non-paired training strategy. This strategy eliminates the need for high-quality images corresponding to real low-light, low-quality images as ground truth labels. Instead, it allows the use of any combination of low-quality and high-quality images to form a sample set for training the image enhancement model. Specific training methods for the image enhancement model can be found below. Figure 5 Description of the illustrated embodiment.

[0080] In one embodiment of this application, the application inference stage of the image enhancement model does not require complex strategy design, referencing Figure 6 The diagram shown is a schematic representation of the image enhancement model provided in this application during the application inference stage. Figure 6 The final enhanced image C can be obtained by directly passing any real low-quality image C sequentially through a denoising neural network (such as the first model) and a brightening neural network (such as the second model). Specifically, the image to be recognized is input into a pre-trained image enhancement model to obtain an enhanced image of the image to be recognized, including: using the first model to denoise the image to be recognized and outputting the denoised image to be recognized; using the second model to enhance the brightness of the denoised image to be recognized, and using the output brightened image as the enhanced image.

[0081] In the model application process of the above embodiments, the first model is used first, and then the output of the first model is used as the input of the second model; in other embodiments, the second model can be used first, and then the output of the second model can be used as the input of the first model, and this application does not limit this.

[0082] S103, extract the first facial feature from the enhanced image using a preset facial feature extraction model.

[0083] In one embodiment of this application, the image to be identified captured by the imaging device may not include a face. Therefore, before extracting the first face feature, it can be determined whether the enhanced image contains a face to be identified. If the enhanced image contains a face to be identified, the first face feature of the face to be identified is extracted. If the enhanced image does not contain a face to be identified, there is no need to perform the subsequent process, and it can be directly determined that the user's identity verification has failed.

[0084] Specifically, the first facial feature is extracted from the enhanced image using a preset facial feature extraction model, including: determining whether there is a face to be identified in the enhanced image based on a preset facial detection model; if there is a face to be identified in the enhanced image, cropping the facial region image from the enhanced image; and extracting the first facial feature from the facial region image using the facial feature extraction model.

[0085] A face detection model is a model used to identify the presence and location of faces in an image or video frame. For example, a face detection model can be a Convolutional Neural Network (CNN)-based model that learns facial features by training on a large number of image samples containing both faces and non-faces. A CNN-based face detection model typically consists of multiple convolutional layers, pooling layers, and fully connected layers to extract facial features from the image and generate predicted bounding boxes for the face region. The image within the predicted bounding box is the face region image. The face detection model can employ existing detection models in related technologies. For example, face detection models can include, but are not limited to, the following models: Haar feature classifier, R-CNN (Region-based Convolutional Neural Networks) model, Fast R-CNN model, Faster R-CNN model, Single Shot MultiBox Detector (SSD) model, You Only Look Once (YOLO) model, etc. This application embodiment does not impose any limitations on these models.

[0086] A face feature extraction model is a model used to extract face feature vectors from an image. These vectors can indicate key information about a face, such as facial contours, eyes, nose, and mouth, as well as more abstract attributes like age, gender, and expression. The face feature extraction model can employ existing feature extraction models in related technologies. For example, it can include, but is not limited to, the following models: VGGFace, FaceNet, DeepFace, ArcFace, etc. This application does not impose any limitations on these models. The face feature vectors extracted by the face feature extraction model typically have a fixed dimension and can be used for subsequent tasks such as face verification. For example, when the face feature extraction model is VGGFace, the extracted first face feature can be a first face feature vector with a dimension of 4096.

[0087] Through the above embodiments, when performing facial feature recognition on the image to be enhanced, it is possible to first determine whether there is a face in the enhanced image and the region image where the face is present. This allows for targeted facial feature recognition in the facial region image, eliminating the need for the facial feature extraction model to consider features in regions that do not contain faces, reducing model computational power, and improving the efficiency and accuracy of facial feature extraction.

[0088] S104, determine whether the first similarity between the first facial feature and the second facial feature in the preset facial template image is greater than or equal to the preset similarity threshold. If the first similarity is greater than or equal to the similarity threshold, execute S105.

[0089] In one embodiment of this application, the face template image represents a pre-acquired face image of a legitimate user. The number of legitimate users may not be unique, and the face template image includes the face image of each legitimate user. If the face recognition method determines that the face to be recognized in the image to be recognized successfully matches a face in any face template image, then the face in the image to be recognized belongs to a legitimate user, and the user's identity verification is successful.

[0090] In one embodiment of this application, a second face feature can be extracted from a face template image using a face feature extraction model. For example, when the face feature extraction model is a VGGFace model, the extracted second face feature can be a second face feature vector with a dimension of 4096. The first face feature vector and the second face feature vector have the same dimension. A preset distance formula can be used to calculate the distance between the first face feature vector and the second face feature vector. Based on this distance, a first similarity between the first face feature and the second face feature is determined. The distance formula can include, but is not limited to, a combination of one or more of the following distance formulas: cosine distance formula, Euclidean distance formula, Chebyshev distance formula, etc. For example, the first similarity can be inversely proportional to the cosine distance between the first face feature vector and the second face feature vector. The formula used can include: First similarity = 0.5 × (1 - cosine distance between the first face feature vector and the second face feature vector).

[0091] In one embodiment of this application, the similarity threshold can be set according to actual needs. For example, the similarity threshold can be 0.9. This application does not impose any specific restrictions on this.

[0092] S105, user identity verification confirmed.

[0093] In one embodiment of this application, if the first similarity is greater than or equal to the similarity threshold, it indicates that the face in the enhanced image and the face in the face template image belong to the same user, and therefore it can be determined that the user's identity verification is successful.

[0094] Through the above embodiments, when performing face recognition on a terminal device without the aid of a supplementary light, the illumination collected by the ambient light sensor of the terminal device can be used to determine whether the current environment is dark, thereby determining whether the image to be recognized captured by the shooting device is a dark image; image enhancement is performed on the dark image captured in the dark environment to obtain an enhanced image with significantly reduced noise, appropriate brightness restoration, and no obvious damage to facial features; by performing feature matching between the facial features in the enhanced image and the facial features in the preset facial template image, the accuracy of face recognition in dark environments can be improved.

[0095] In one embodiment of this application, if the first similarity is less than a similarity threshold, it indicates that the first face feature matching of the image to be identified failed, and a second face feature matching can be performed. The second face feature matching can match the face features of the image to be identified before enhancement with the face features of the degraded face template image, and return the final result of the face feature matching.

[0096] Specifically, if the first similarity in S104 is less than the similarity threshold, the process proceeds to S106.

[0097] S106, Perform image degradation processing on the face template image to obtain a degraded image of the face template image.

[0098] In one embodiment of this application, the face template image can be degraded by adding noise or reducing its brightness. Specifically, the method for degrading the face template image can be found in the documentation for... Figure 7 Description of the illustrated embodiment. (As shown in the image) Figure 7 The diagram shown is a flowchart of a method for image degradation processing of a face template image according to an embodiment of this application. The method is applied in a terminal device, and the method for image degradation processing of a face template image includes:

[0099] S401, Obtain the preset third image.

[0100] In one embodiment of this application, the third image represents a low-quality image, such as a first image that was not used to train the image enhancement model, or the third image may be an image to be identified.

[0101] S402, input the third image into the image enhancement model, use the first model of the image enhancement model to denoise the third image, and obtain the third denoised image corresponding to the third image, and the second difference image between the third image and the third denoised image.

[0102] In one embodiment of this application, the following can be used: Figure 10 The method shown in S501 obtains the second difference image. For example... Figure 8 As shown, an image enhancement model can be used to denoise the low-quality image to be verified D (such as the third image) to obtain a denoised image D (such as the third denoised image), and a pure noise image D (such as the second difference image) can be obtained based on the low-quality image to be verified D and the denoised image D.

[0103] S403, reduce the brightness of the face template image to obtain the image with reduced brightness.

[0104] In one embodiment of this application, the method for reducing the brightness of a face template image includes, but is not limited to, a combination of one or more of the following methods: (1) Linear adjustment: reducing the brightness value of each pixel in the face template image by a fixed offset. For example, when the face template image is an RGB image, the RGB channel value of each pixel in the face template image can be subtracted by a preset constant (e.g., 5); (2) Gamma correction: obtaining an updated brightness value by exponentiation of the brightness value of each pixel in the face template image. For example, setting the updated brightness value = original brightness^gamma value; (3) Histogram equalization: increasing contrast and reducing brightness by redistributing the brightness values ​​of pixels in the face template image. For example, stretching the grayscale histogram of the face template image to achieve redistribution of the brightness values ​​of pixels in the face template image. Figure 8 As shown, after reducing the brightness of the template image E (such as a face template image), a darkened template image E (such as an image with reduced brightness) is obtained.

[0105] S404, perform image fusion between the second difference image and the image with reduced brightness to obtain a degraded image of the face template image.

[0106] In one embodiment of this application, the image with reduced brightness can be converted into a grayscale image, and then the second difference image and the grayscale image with reduced brightness can be fused at the pixel level to obtain a degraded image of the face template image. For example Figure 8 As shown, the degraded template image E (such as the degraded image) is obtained by fusing the pure noise image D (such as the second difference image) with the darkened template image E (such as the image after the brightness is reduced).

[0107] Through the above embodiments, it is possible to obtain a pure noise map corresponding to a low-quality image D (such as the third image) directly using an image enhancement model without separately training a degradation neural network for image degradation. This pure noise map is then used as a noise image (such as the second difference image) for degrading the face template. With proper and sufficient training of the denoising model (such as the first model) in the aforementioned image enhancement model, the degraded image will possess very realistic noise interference, with a noise distribution similar to the real low-quality image D. At this point, facial feature extraction is performed on the degraded image, and a matching algorithm is used to compare the facial features in the face template image with the facial features in the degraded image. This can reduce the impact of noise to a certain extent, focus on differences in facial features, and thus improve the accuracy of face recognition.

[0108] S107, using a face feature extraction model to extract a third face feature from the image to be identified, and using a face feature extraction model to extract a fourth face feature from the degraded image.

[0109] In one embodiment of this application, the third facial feature may be a facial feature vector of the image to be identified, and the fourth facial feature may be a facial feature vector of the degraded image.

[0110] S108, determine whether the second similarity between the third face feature and the fourth face feature is greater than or equal to the similarity threshold. If the second similarity is greater than or equal to the similarity threshold, proceed to S105; or, if the second similarity is less than the similarity threshold, proceed to S109.

[0111] In one embodiment of this application, the second similarity between the third face feature and the fourth face feature can be calculated using cosine similarity or Euclidean distance formula.

[0112] In one embodiment of this application, if the second similarity is greater than or equal to the similarity threshold, it indicates that the face to be identified in the image to be identified and the face in the degraded image belong to the same user, and therefore it can be determined that the user's identity verification is successful.

[0113] S109, It has been determined that the user's identity verification failed.

[0114] In one embodiment of this application, if both facial feature matching processes fail, it can be determined that the user's identity verification has failed.

[0115] In the above embodiments, if the face feature matching result using the enhanced image fails, the face template image can be further degraded by adding noise from the low-light image to it. The degraded image can then be used to perform a second matching attempt with the low-light image. For example, the face features of the unenhanced low-light image and the degraded template image can be compared. By introducing a face template image degradation strategy, the false rejection rate (FRR) of face recognition in low-light scenes can be further reduced, improving the accuracy of face recognition and user experience.

[0116] See Figure 4 The diagram shown is a flowchart of a method for determining whether the current environment is a low-light environment according to an embodiment of this application. The method is applied in a terminal device, and the method for determining whether the current environment is a low-light environment includes:

[0117] S201 uses an ambient light sensor to collect multiple continuous illuminance values ​​of the current environment.

[0118] In one embodiment of this application, multiple consecutive illuminances represent multiple illuminances having a continuous temporal sequence. The number of multiple consecutive illuminances can be set according to the sampling frequency of the ambient light sensor. For example, the number can be proportional to the sampling frequency of the ambient light sensor, and this application does not limit this. For instance, if the sampling frequency of the ambient light sensor is 10 times per second, then the number can be 5.

[0119] S202, based on the moving average algorithm, determines the average value of multiple consecutive illuminance values.

[0120] In one embodiment of this application, the specific process for determining the average value of multiple consecutive illuminances based on the sliding average algorithm includes: determining the size of the window, i.e., the number of illuminances the window can accommodate. The size of the window can be less than or equal to the total number of consecutive illuminances. For example, if the number of multiple consecutive illuminances is 5, the window size can be 3; initializing the window position, making the first data in the window the first illuminance among multiple consecutive illuminances, and calculating the average value p1 of the illuminance in the window; shifting the window position one data position to the right, making the first data in the window the second illuminance among multiple consecutive illuminances, and calculating the average value p2 of the illuminance in the window; and so on, continuously shifting the sliding window until the last data in the window is the last illuminance among multiple consecutive illuminances, and taking the current position of the window as the last placement position of the window; determining the number m of window placement positions (also called moving positions) and the average value pi of the illuminance in the window at each position, and determining the average value of multiple consecutive illuminances according to ∑pi / m, where i represents a natural number.

[0121] S203, determine whether the above average value is less than the preset illuminance threshold. If the average value is less than the illuminance threshold, execute S204, or if the average value is greater than or equal to the illuminance threshold, execute S205.

[0122] In one embodiment of this application, the illuminance threshold can be set according to the photosensitivity of the imaging device. For example, the illuminance threshold can be proportional to the photosensitivity of the imaging device, and this application does not limit this. For instance, the illuminance threshold can be set to 0.005 Lux.

[0123] S204, confirming that the current environment is a low-light environment.

[0124] In one embodiment of this application, when it is determined that the current environment is a dark environment based on the illuminance, the description of the embodiment shown in S102 can be referred to.

[0125] S205, the current environment is determined to be a non-dark environment.

[0126] Through the above embodiments, the average value of multiple consecutive illuminance values ​​can be determined as the illuminance of the current environment by using the moving average algorithm. This avoids misjudging whether the current environment is a dark environment due to instantaneous increases or decreases in illuminance or fluctuations in ambient light sensors, thereby improving the accuracy of dark environment judgment.

[0127] In one embodiment of this application, if the current environment is determined to be non-dark based on illumination, the image to be recognized can be considered a high-quality non-dark image that can be directly used for face recognition. In this case, the method for face recognition of the high-quality image is a traditional face recognition method, which can be referred to below for... Figure 12 Description of the illustrated embodiment.

[0128] The above embodiments have described the application process of the image enhancement model. Next, the process of training the image enhancement model based on the training strategy of non-paired training will be described.

[0129] First, the non-paired training strategy used in this application is introduced, referring to... Figure 9 The diagram illustrates a non-paired training strategy provided in this application embodiment. Specifically, the non-paired training strategy includes: passing any low-quality image A (hereinafter referred to as the first image) through a denoising neural network (such as the first model) of an image enhancement model to obtain a denoised image A, and a difference map between the denoised image A and the low-quality image A (hereinafter referred to as the first difference image, which is a pure noise map); superimposing the pure noise map corresponding to the low-quality image A with any real high-quality image B (hereinafter referred to as the second image) to synthesize the low-quality image B (such as the fused image corresponding to the second image); the real high-quality image B can thus be used as the enhancement target for synthesizing the low-quality image B to train the denoising neural network, meaning that after denoising the synthesized low-quality image B using the denoising neural network, the resulting denoised image B should be as close as possible to the high-quality image B. Furthermore, the real high-quality image B can also be used as the enhancement target for synthesizing the low-quality image B to train a brightening neural network (such as the second model), so that the denoised image B before brightening (such as the second denoised image), after passing through the brightening neural network, is as close as possible to the original high-quality image B.

[0130] In the training process of the first model, a denoising neural network is used for two denoising operations, each corresponding to a loss function. For example, in the first denoising operation, a low-quality image A is denoised to obtain a purely noisy image. Therefore, the first loss function can be used to constrain the noise in this purely noisy image, i.e., to measure the degree of noise presence. In the second denoising operation, a synthesized low-quality image B is denoised to obtain a denoised image B. The second loss function can be used to constrain this denoised image B, making it as close as possible to the high-quality image B. Furthermore, a third loss function can be used to constrain the brightening neural network of the image enhancement model (such as the second model), so that the denoised image B before brightening (such as the second denoised image), after passing through the brightening neural network, is as close as possible to the original high-quality image B. By adding loss functions to these three stages to create constraints, the model training can avoid getting trapped in local optima, thus improving the model's performance.

[0131] Then, the image enhancement model is trained according to the aforementioned non-paired training strategy. See also... Figure 5 The diagram shown is a flowchart of a training method for an image enhancement model provided in an embodiment of this application. The method is applied in a terminal device, and the training method for the image enhancement model includes:

[0132] S301, acquire multiple sample images.

[0133] In one embodiment of this application, the multiple sample images include multiple first images and multiple second images. Based on the non-paired training strategy provided in this application embodiment, multiple (e.g., 10,000) arbitrary low-quality images (e.g., images with a lot of noise and low brightness) and multiple (e.g., 10,000) arbitrary high-quality images (e.g., images with less noise and higher brightness) can be obtained from an open-source database, thereby obtaining multiple (e.g., 20,000) sample images. Here, the first image represents a low-quality image, the second image represents a high-quality image, and the noise level of any first image is greater than the noise level of any second image.

[0134] In one embodiment of this application, after acquiring multiple sample images, the sample images can be preprocessed to make them suitable for subsequent model training. The preprocessing may include, but is not limited to, a combination of one or more of the following processing methods: image cropping, image correction, contrast adjustment, and grayscale processing.

[0135] S302, construct a training set and a test set based on multiple sample images, and use the training set to iterate and update the preset initial model at least once until an image enhancement model that meets the preset requirements is obtained.

[0136] In one embodiment of this application, the multiple sample images can be divided into a training set and a test set according to a preset ratio (e.g., 8:2). For example, 80% of the images are selected from multiple first images to construct the training set, and 80% of the images are also selected from multiple second images to construct the training set. The remaining unselected images are used as images in the test set. In another embodiment, the multiple sample images can also be divided into a training set, a test set, and a validation set, for example, in a ratio of 7:2:1.

[0137] In one embodiment of this application, the initial model can be a model obtained by initializing a preset neural network (e.g., a graph-to-graph conversion neural network). The initial model may include a first initial model and a second initial model, wherein the first initial model may be a denoising neural network for removing image noise, and the second initial model may be a brightening neural network for enhancing image brightness. For details, please refer to the description in the following embodiments.

[0138] In one embodiment of this application, the process of initializing a preset neural network to obtain an initial model may include, but is not limited to: defining the network structure of the model, for example, determining the type, number, and connection method of the network layers; initializing the parameters of the model, for example, initializing the weights and biases of each network layer of the model to random values ​​or specific preset values, commonly used methods include zero initialization, random normal distribution initialization (such as Xavier initialization); defining the loss function of the model, which can be used to measure the difference between the model output and the target value. Based on the non-paired training strategy used in this application, the loss function of the model can include a first loss function, a second loss function, and a third loss function, as described in the following embodiments; determining the optimization algorithm of the model, for example, stochastic gradient descent, Adam algorithm, etc., which can be used to iteratively update the model parameters to minimize the loss function; and determining the sample batch used when training the model, which is used to determine the number of samples used in each iteration of training. Usually, training is performed in the form of mini-batches, which can improve training efficiency and stability, for example, using one first image each time.

[0139] In one embodiment of this application, when updating the initial model at least once, the input data (e.g., data from the training set) can be forward-propagated through the network to obtain the output result; the error between the output result and the ideal result is calculated according to the loss function, the gradient is calculated using the backpropagation algorithm, and the parameters in the network are updated; the above forward and backpropagation processes are repeated, and the model parameters are gradually optimized through multiple iterations, so that the model can better fit the training data until an image enhancement model that meets the preset requirements is obtained. Afterwards, the performance of the image enhancement model can be verified using data from the validation set. When the verification result does not meet expectations, the training set is updated and the model is tuned using the updated training set to avoid overfitting. Furthermore, the model can be further tested using a test set to further improve its generalization ability.

[0140] In one embodiment of this application, since the initial model includes a first initial model and a second initial model, the first initial model and the second initial model can be iteratively updated at least once each. The preset requirements include a combination of one or more of the following: the loss function corresponding to the initial model reaches a convergence condition; the test results obtained by testing the initial model using a test set indicate that the model performance (e.g., accuracy, recall, F1 score, etc.) of the initial model meets expectations, for example, the model's accuracy reaches a preset accuracy threshold (e.g., 0.9); the number of the at least one iterative update reaches a preset iteration number threshold (e.g., 10,000 times). The loss function may include a first loss function and a second loss function corresponding to the first initial model, and may also include a third loss function corresponding to the second initial model. The convergence condition of the loss function may include, but is not limited to: the sum of the first loss value corresponding to the first loss function and the second loss value corresponding to the second loss function reaches a preset threshold (hereinafter referred to as the fourth threshold); the third loss value corresponding to the third loss function is less than or equal to a preset threshold (hereinafter referred to as the third threshold).

[0141] The process for any update in at least one iteration can also be found in the following text. Figures 10 to 11 Description of the illustrated embodiment.

[0142] In one embodiment of this application, the first initial model may be updated at least once, see reference. Figure 10 The diagram shown is a flowchart of a method for updating a first initial model in any instance, according to an embodiment of this application. The method is applied in a terminal device, and the flowchart of the method for updating the first initial model in any instance includes:

[0143] S501, input any first image in the training set into the first initial model, use the first initial model to denoise any first image, and obtain the first denoised image corresponding to any first image, and the first difference image between any first image and the first denoised image.

[0144] In one embodiment of this application, the first initial model can be a general graph-to-graph conversion neural network, which can be selected according to actual needs, such as screening based on specific scenario performance, hardware limitations on parameter quantity, speed limitations, etc.

[0145] For example, the first initial model may include, but is not limited to, a combination of one or more of the following models: an autoencoder model: which includes an encoder and a decoder, the encoder compresses the input image (e.g., the first image) into a low-dimensional representation and the decoder decodes it back into an image representation (e.g., the first denoised image), which can effectively remove noise from the image; a U-Net model: a convolutional neural network with a symmetrical U-shaped structure, which can capture features of different scales of the input image and use a skip connection mechanism to connect different network layers in the decoding stage, thereby improving the performance of image denoising.

[0146] In one embodiment of this application, the method for obtaining the first difference image may include: obtaining a grayscale image (hereinafter referred to as the first grayscale image) corresponding to the first image and a grayscale image (hereinafter referred to as the second grayscale image) corresponding to the first denoised image; calculating the difference between the grayscale value of each pixel in the first grayscale image and the corresponding pixel in the second grayscale image, and using the obtained difference as the pixel value of the corresponding pixel in the first difference image to obtain the first difference image.

[0147] In one embodiment of this application, the first difference image is a pure noise image containing only noise. The preliminary denoising performance of the first initial model can be determined by analyzing the noise in the first difference image.

[0148] In one embodiment of this application, during the next update of each update, another first image is used to update the first initial model. This other first image refers to a first image that was not used in previous updates.

[0149] S502, determine the first loss value corresponding to the first difference image using a preset first loss function.

[0150] In one embodiment of this application, the first loss value calculated by the first loss function is used to indicate the degree of noise (or the degree of noise presence) in the first difference image. The larger the first loss value, the more noise there is in the first difference image.

[0151] Specifically, the first loss function can be a statistical analysis function. This can be achieved by statistically analyzing the mean, variance, and standard deviation of the grayscale values ​​of all pixels in the first difference image. The first loss function can be constructed based on these values; for example, it could be a weighted sum of the mean, variance, and standard deviation. Alternatively, the first loss function can be a spectral analysis function. This can be achieved by converting the first difference image to the frequency domain using methods such as Fast Fourier Transform, thereby performing spectral analysis to determine the distribution of the signal energy in the high-frequency range corresponding to the noise. Based on this distribution, a spectral analysis function can be constructed; for example, it could be the probability distribution of the signal energy in the high-frequency range corresponding to the noise.

[0152] S503, determine whether the first loss value is less than or equal to the preset first threshold. If the first loss value is less than or equal to the first threshold, execute S501; or, if the first loss value is greater than the first threshold, execute S504.

[0153] In one embodiment of this application, the first threshold can be set according to actual needs. For example, the first threshold can be proportional to the size of the first image. This application does not impose any specific restrictions on this.

[0154] In one embodiment of this application, a first loss value less than or equal to a first threshold indicates that the noise level in the first difference image is low. Therefore, the denoising performance of the first initial model is low in this iteration, and it needs to be updated using the next batch of training samples (e.g., another first image).

[0155] In another embodiment, it is not necessary to judge the first loss value. That is, after S502, the process can be transferred to S504, and the second loss value in the subsequent process S507 can be used to determine whether the first initial model should be updated again.

[0156] S504, perform image fusion between the first difference image and any second image in the training set to obtain the fused image corresponding to any second image.

[0157] In one embodiment of this application, a first loss value greater than a first threshold indicates that the first initial model can be used for preliminary denoising in this iteration. However, the performance of the first initial model in denoising a low-quality image and restoring it to the corresponding high-quality image still needs further verification.

[0158] Based on the non-paired training strategy of this application, the training samples used in this application do not contain corresponding pairs of low-quality images (e.g., images with more noise) and high-quality images (e.g., images with less noise). Therefore, the pure noise image (e.g., the first difference image) of the intermediate result generated during the training process of the first initial model can be used to reduce the quality of the high-quality image (e.g., the second image) to obtain the low-quality image corresponding to the second image (e.g., the fused image corresponding to the second image), thereby realizing the acquisition of pairs of low-quality images and high-quality images.

[0159] In one embodiment of this application, when fusing the first difference image with any second image in the training set, a grayscale image (hereinafter referred to as the third grayscale image) corresponding to the second image can be obtained. The grayscale value of each pixel in the first difference image is added to the grayscale value of the corresponding pixel in the third grayscale image using a pixel-by-pixel fusion method, thereby generating the fused image corresponding to the second image.

[0160] S505, the fused image is input into the first initial model, and the fused image is denoised using the first initial model to obtain the second denoised image corresponding to the fused image.

[0161] In one embodiment of this application, the first initial model is intended to denoise the input fused image so that the resulting second denoised image is infinitely close to the original second image.

[0162] S506, using a preset second loss function, determine the second loss value between the second denoised image and any corresponding second image.

[0163] In one embodiment of this application, the second loss value calculated by the second loss function is used to measure the difference between the second denoised image and the corresponding second image. The larger the second loss value, the greater the difference between the second denoised image and the corresponding second image. The second loss function may include, but is not limited to, a combination of one or more of the following functions: L1 norm, perceptual loss function.

[0164] S507, determine whether the second loss value is less than or equal to the preset second threshold. If the second loss value is greater than the second threshold, execute S501; or, if the second loss value is less than or equal to the second threshold, end the iterative update of the first initial model.

[0165] In one embodiment of this application, the second threshold can be set according to actual needs. For example, the second threshold can be 0.1. This application does not impose any specific restrictions on this.

[0166] In one embodiment of this application, a second loss value greater than a second threshold indicates a significant difference between the second denoised image and any corresponding second image. Therefore, the denoising performance of the first initial model is low in this iteration, requiring an update using the next batch of training samples. Specifically, the next first image and the next second image can be used to update the first initial model. In another embodiment, if the second loss value is greater than the second threshold, the next first difference image and the next second image can also be used to update the first initial model, i.e., the process proceeds to S504.

[0167] In one embodiment of this application, if the second loss value is less than or equal to the second threshold, it can be considered that the denoising performance of the first initial model obtained in this iteration has reached the expected level. Therefore, the iterative update of the first initial model can be stopped, and at least one iterative update of the second initial model can be performed.

[0168] In another embodiment, if S503 is not executed, that is, the process is directly transferred to S504 after S502, the sum of the first loss value and the second loss value can be used in S507 to determine whether to update the first initial model again. For example, if the sum of the first loss value and the second loss value is less than or equal to the fourth threshold, the first initial model is updated again using the next first image and the next second image; or, if the sum of the first loss value and the second loss value is greater than the fourth threshold, the iterative update of the first initial model is stopped, and at least one iterative update of the second initial model is performed.

[0169] See Figure 11 The diagram shown is a flowchart of a method for updating a second initial model in any instance, according to an embodiment of this application. The method is applied in a terminal device, and the steps of the method for updating the second initial model in any instance include:

[0170] S601, input the second denoised image corresponding to any second image into the second initial model, and use the second initial model to enhance the brightness of the second denoised image to obtain the brightness-enhanced image corresponding to the second denoised image.

[0171] In one embodiment of this application, based on the non-paired training strategy of this application, there are no corresponding pairs of low-quality images (e.g., images with low brightness) and high-quality images (e.g., images with high brightness) in the training samples used in this application. Therefore, the second denoised image corresponding to any second image generated as an intermediate result when training the first initial model can be used as a low-quality image, and the second image corresponding to the second denoised image can be used as a high-quality image, thereby obtaining pairs of low-quality images and high-quality images to train the second initial model.

[0172] In one embodiment of this application, the second initial model can be a general graph-to-graph conversion neural network, which can be selected according to actual needs, such as screening based on specific scenario performance, hardware limitations on parameter quantity, speed limitations, etc.

[0173] The second initial model may include, but is not limited to, a combination of one or more of the following models: a contrast enhancement network, which includes convolutional layers, batch normalization layers, and activation functions, and can perform contrast enhancement operations on the input image (e.g., the second denoised image) to make the image more vivid and clear, thereby improving the image brightness; and a gamma correction network, which can adjust the brightness and contrast of the image through nonlinear operations, thereby improving the display effect of the image.

[0174] In one embodiment of this application, during the next update, the second initial model is updated again using the second denoised image corresponding to the next second image. Here, the second denoised image corresponding to the next second image refers to the second denoised image corresponding to a second image that was not used in previous updates.

[0175] S602, using a preset third loss function, determine the third loss value between the brightness-enhanced image and any corresponding second image.

[0176] In one embodiment of this application, the third loss value calculated by the third loss function is used to indicate the difference between the brightness-enhanced image and the corresponding second image. The larger the third loss value, the greater the difference between the brightness-enhanced image and the corresponding second image. The third loss function may include, but is not limited to, a combination of one or more of the following functions: L1 norm, perceptual loss function.

[0177] S603, determine whether the third loss value is less than or equal to the preset third threshold. If the third loss value is greater than the third threshold, execute S601; or, if the third loss value is less than or equal to the third threshold, execute S604.

[0178] In one embodiment of this application, the third threshold can be set according to actual needs. For example, the third threshold can be 0.1. This application does not impose any specific restrictions on this.

[0179] In one embodiment of this application, a third loss value greater than a third threshold indicates that the difference between the brightness-enhanced image and the corresponding second image is large. Therefore, the brightness enhancement performance of the second initial model is low in this iteration, and it needs to be updated again using the next batch of training samples. Specifically, the second initial model can be updated again using the second denoised image corresponding to the next second image.

[0180] Execute S604 to determine if the initial model meets the preset requirements, and use the initial model as the image enhancement model.

[0181] In one embodiment of this application, if the third loss value is less than or equal to the third threshold, it can be considered that the brightness improvement of the second initial model obtained in this iteration has reached the expected level. Since the training of the first initial model has also been completed before, the iterative update of the initial model can be stopped, the initial model is determined to meet the preset requirements, and the initial model is used as the image enhancement model.

[0182] Through the above embodiments, compared with conventional schemes that include preprocessing operations such as noise removal or brightness enhancement, the denoising neural network provided in this application is trained with unpaired real data, which does not require assumptions about noise distribution, limitations on application scenarios, or manual adjustment of degradation parameters, and has better generalization performance. Furthermore, the above embodiments connect the denoising neural network and the brightness enhancement neural network in a serial manner, improving the usability and interpretability of the image enhancement model.

[0183] See Figure 12 The diagram shown is a flowchart of a method for face recognition of high-quality images according to an embodiment of this application. The method is applied in a terminal device, and the method for face recognition of high-quality images includes:

[0184] S701, based on the preset face detection model, determine whether there is a face to be identified in the image to be identified; if there is a face to be identified in the image to be identified, execute S702; if there is no face to be identified in the image to be identified, execute S706.

[0185] S702, Extract the face region image from the image to be recognized.

[0186] In one embodiment of this application, referring to the relevant description in S103, based on the result of face recognition, image processing techniques such as cropping and scaling are used to extract a corresponding face region image from the image to be recognized. The extracted face region image may include the entire face for subsequent processing and recognition.

[0187] S703 uses a face feature extraction model to extract the fifth face feature from a face region image.

[0188] In one embodiment of this application, the extraction of the fifth facial feature from the facial region image using a facial feature extraction model can be referred to the relevant description in S103.

[0189] S704, determine whether the third similarity between the fifth facial feature and the second facial feature is greater than or equal to the similarity threshold. If the third similarity is greater than or equal to the similarity threshold, proceed to S705; or, if the third similarity is less than the similarity threshold, proceed to S706.

[0190] In one embodiment of this application, the extraction and comparison of facial features can refer to the methods used in related technologies, and this embodiment of the application does not limit this to the specific methods used. For details, please refer to the relevant description in S103.

[0191] S705, confirming that the user's identity verification has passed.

[0192] S706, It has been determined that the user's identity verification failed.

[0193] The above technical solution allows us to determine that the captured image is not a dark image when the current environment is not dark, thus eliminating the need for image enhancement or other processing. We can directly perform face recognition on the image and crop the face region from it. Then, we extract face features from the cropped face region and match them with the face features in the face template image to determine the face recognition result. This enables efficient and accurate face recognition even in non-dark environments.

[0194] See Figure 13 The diagram shown is a schematic representation of the overall architecture of a face recognition method provided in an embodiment of this application. Figure 13 As can be seen, compared with the modular architecture of conventional face recognition systems, the face recognition method of this application adds a low-light image enhancement algorithm and a template image degradation strategy.

[0195] Specifically, a face recognition system typically consists of two parts: a face detection algorithm and a feature extraction and matching algorithm. The former determines whether a face can be found and detected in an image, while the latter is used to determine whether the detected face matches a preset template face. Low-brightness, high-noise images in low light conditions have a severe impact on both detection and feature extraction and matching.

[0196] The low-light image enhancement algorithm provided in this application is applied before the face detection algorithm. As a graph-to-graph transformation neural network model algorithm, the low-light image enhancement algorithm can take the original low-light image as input and output an enhanced image with significantly reduced noise, appropriate brightness restoration, and no obvious destruction of facial features. This enhanced image can be used in face detection, face feature extraction, and face feature matching modules. In another embodiment, the low-light image enhancement algorithm can also be applied after the face detection algorithm to enhance the image of the face region determined by the face detection algorithm.

[0197] The template image degradation strategy provided in this application does not require the introduction of a new neural network model. Instead, it introduces a secondary matching mechanism based on input image noise, a derivative of the aforementioned low-light image enhancement algorithm. Specifically, this application introduces a dual matching process for low-light images. The first matching attempt involves matching the enhanced image with the template image. If this fails, noise from the low-light image is added to the template image to degrade it, and a second matching attempt is made, comparing the features of the unenhanced low-light image and the degraded template image. By introducing this template image degradation strategy, the false rejection rate in low-light scenes can be further reduced, improving the user experience.

[0198] See Figure 14 The diagram shown is a flowchart of a face recognition method provided in another embodiment of this application. Figure 14 The flowchart shown can be summarized as follows: When the face recognition system is activated, the camera captures the original image (such as the image to be recognized), and basic preprocessing is performed on the original image (such as image correction, size adjustment, etc.), the ambient light sliding average value collected by the ambient light sensor is obtained. If the ambient light sliding average value is greater than the preset illuminance threshold p, the current environment is not a dark environment, and the normal face detection and recognition process is directly entered, and the recognition result is returned directly. If the ambient light sliding average value is less than the illuminance threshold p, the current environment is a dark environment, and the first matching process in the dark environment is entered.

[0199] In the first matching process in low-light environment, a low-light image enhancement algorithm is used to enhance the original image. The enhanced image is then used for face detection and recognition algorithms. The features of the enhanced face image are matched with the features of a template face image (or face template image). If the result is a match, it is returned directly; if the result is still a mismatch, the second matching process in low-light environment is executed. In the second matching process in low-light environment, based on the intermediate results from the first matching process, the low-light image noise extracted by the aforementioned image enhancement algorithm is used to actively degrade the template face image. Then, the features of the unenhanced low-light face image are matched with the features of the degraded template face image, and the final result of a match or a mismatch is returned.

[0200] Through the above embodiments, when performing face recognition using a terminal device without a supplementary lighting camera, the system can determine whether the current environment is dark based on the illuminance collected by the terminal device's ambient light sensor, thereby determining whether the image to be recognized captured by the imaging device is a dark-light image. Image enhancement is performed on the dark-light image captured in a dark-light environment to obtain an enhanced image with significantly reduced noise, appropriate brightness restoration, and no significant damage to facial features. By performing feature matching between the facial features in the enhanced image and the facial features in a preset facial template image, the accuracy of face recognition in dark-light environments can be improved. If the result of facial feature matching using the enhanced image is unsuccessful, the template image can be further degraded by adding noise from the dark-light image to it. A second matching attempt is then made using the degraded image and the dark-light image, i.e., comparing the facial features of the unenhanced dark-light image and the degraded template image. By introducing a facial template image degradation strategy, the false rejection rate (FRR) of face recognition in dark-light scenes can be further reduced, improving the accuracy of face recognition and user experience.

[0201] This application also provides a terminal device 100, see below. Figure 15 As shown, the terminal device 100 may be a mobile phone, tablet computer, desktop computer, laptop computer, handheld computer, notebook computer, ultra-mobile personal computer (UMPC), netbook, as well as cellular phone, personal digital assistant (PDA), augmented reality (AR) device, virtual reality (VR) device, artificial intelligence (AI) device, wearable device, in-vehicle device, smart home device and / or smart city device. The specific type of terminal device 100 is not specifically limited in the embodiments of this application.

[0202] Terminal device 100 may include a processor 110, an external memory interface 120, an internal memory 121, a Universal Serial Bus (USB) interface 130, a charging management module 140, a power management module 141, a battery 142, an antenna 1, an antenna 2, a mobile communication module 150, a wireless communication module 160, an audio module 170, a speaker 170A, a receiver 170B, a microphone 170C, a headphone jack 170D, a sensor module 180, buttons 190, a motor 191, an indicator 192, a camera 193, a display screen 194, and a Subscriber Identification Module (SIM) card interface 195, etc. The sensor module 180 may include a pressure sensor 180A, a gyroscope sensor 180B, a barometric pressure sensor 180C, a magnetic sensor 180D, an accelerometer sensor 180E, a distance sensor 180F, a proximity sensor 180G, a fingerprint sensor 180H, a temperature sensor 180J, a touch sensor 180K, an ambient light sensor 180L, a bone conduction sensor 180M, etc.

[0203] It is understood that the structures illustrated in the embodiments of the present invention do not constitute a specific limitation on the terminal device 100. In other embodiments of this application, the terminal device 100 may include more or fewer components than illustrated, or combine some components, or split some components, or have different component arrangements. The illustrated components may be implemented in hardware, software, or a combination of software and hardware.

[0204] Processor 110 may include one or more processing units, such as an application processor (AP), a modem processor, a graphics processing unit (GPU), an image signal processor (ISP), a controller, a video codec, a digital signal processor (DSP), a baseband processor, and / or a neural network processing unit (NPU). These different processing units may be independent devices or integrated into one or more processors.

[0205] The controller can generate operation control signals based on the instruction opcode and timing signals to complete the control of instruction fetching and execution.

[0206] The processor 110 may also include a memory for storing instructions and data. In one embodiment of this application, the memory in the processor 110 is a cache memory. The memory can store instructions or data that the processor 110 has just used or that are used repeatedly. If the processor 110 needs to use the instructions or data again, it can directly retrieve them from the memory. This avoids repeated accesses, reduces the waiting time of the processor 110, and thus improves the efficiency of the system.

[0207] In one embodiment of this application, the processor 110 may include one or more interfaces. These interfaces may include an Inter-integrated Circuit (I2C) interface, an Inter-integrated Circuit Sound (I2S) interface, a Pulse Code Modulation (PCM) interface, a Universal Asynchronous Receiver / Transmitter (UART) interface, a Mobile Industry Processor Interface (MIPI) interface, a General-Purpose Input / Output (GPIO) interface, a Subscriber Identity Module (SIM) interface, and / or a Universal Serial Bus (USB) interface, etc.

[0208] The I2C interface is a bidirectional synchronous serial bus, including a serial data line (SDA) and a serial clock line (SCL). In one embodiment of this application, the processor 110 may include multiple I2C buses. The processor 110 can couple to the touch sensor 180K, charger, flash, camera 193, etc., through different I2C bus interfaces. For example, the processor 110 can couple to the touch sensor 180K through the I2C interface, enabling the processor 110 and the touch sensor 180K to communicate through the I2C bus interface, thereby realizing the touch function of the terminal device 100.

[0209] The I2S interface can be used for audio communication. In one embodiment of this application, the processor 110 may include multiple I2S buses. The processor 110 can be coupled to the audio module 170 via the I2S bus to realize communication between the processor 110 and the audio module 170. In one embodiment of this application, the audio module 170 can transmit audio signals to the wireless communication module 160 via the I2S interface to realize the function of answering phone calls through a Bluetooth headset.

[0210] The PCM interface can also be used for audio communication, sampling, quantizing, and encoding analog signals. In one embodiment of this application, the audio module 170 and the wireless communication module 160 can be coupled via the PCM bus interface. In another embodiment of this application, the audio module 170 can also transmit audio signals to the wireless communication module 160 via the PCM interface, enabling the function of answering phone calls through a Bluetooth headset. Both the I2S interface and the PCM interface can be used for audio communication.

[0211] The UART interface is a universal serial data bus used for asynchronous communication. The bus can be a bidirectional communication bus. It converts the data to be transmitted between serial and parallel communication. In one embodiment of this application, the UART interface is typically used to connect the processor 110 and the wireless communication module 160. For example, the processor 110 communicates with the Bluetooth module in the wireless communication module 160 via the UART interface to implement Bluetooth functionality. In one embodiment of this application, the audio module 170 can transmit audio signals to the wireless communication module 160 via the UART interface to enable music playback via Bluetooth headphones.

[0212] The MIPI interface can be used to connect the processor 110 to peripheral devices such as the display screen 194 and the camera 193. The MIPI interface includes a Camera Serial Interface (CSI) and a Display Serial Interface (DSI). In one embodiment of this application, the processor 110 and the camera 193 communicate via the CSI interface to realize the shooting function of the terminal device 100. The processor 110 and the display screen 194 communicate via the DSI interface to realize the display function of the terminal device 100.

[0213] The GPIO interface can be configured via software. It can be configured as a control signal or a data signal. In one embodiment of this application, the GPIO interface can be used to connect the processor 110 to a camera 193, a display screen 194, a wireless communication module 160, an audio module 170, a sensor module 180, etc. The GPIO interface can also be configured as an I2C interface, an I2S interface, a UART interface, a MIPI interface, etc.

[0214] USB port 130 is a USB standard compliant interface, specifically a Mini USB port, Micro USB port, USB Type-C port, etc. USB port 130 can be used to connect a charger to charge terminal device 100, and can also be used for data transfer between terminal device 100 and peripheral devices. It can also be used to connect headphones for audio playback. Furthermore, the interface can be used to connect other terminal devices 100, such as AR devices.

[0215] It is understood that the interface connection relationships between the modules illustrated in the embodiments of the present invention are merely illustrative and do not constitute a structural limitation on the terminal device 100. In other embodiments of this application, the terminal device 100 may also employ different interface connection methods or combinations of multiple interface connection methods as described in the above embodiments.

[0216] The charging management module 140 receives charging input from a charger. The charger can be a wireless charger or a wired charger. In some wired charging embodiments, the charging management module 140 receives charging input from the wired charger via the USB interface 130. In some wireless charging embodiments, the charging management module 140 receives wireless charging input via the wireless charging coil of the terminal device 100. While charging the battery 142, the charging management module 140 can also supply power to the terminal device 100 via the power management module 141.

[0217] The power management module 141 connects the battery 142, the charging management module 140, and the processor 110. The power management module 141 receives input from the battery 142 and / or the charging management module 140, providing power to the processor 110, internal memory 121, display screen 194, camera 193, and wireless communication module 160, etc. The power management module 141 can also monitor parameters such as battery capacity, battery cycle count, and battery health status (leakage current, impedance). In some other embodiments, the power management module 141 may also be located within the processor 110. In other embodiments, the power management module 141 and the charging management module 140 may be located in the same device.

[0218] The wireless communication function of the terminal device 100 can be implemented through antenna 1, antenna 2, mobile communication module 150, wireless communication module 160, modem processor and baseband processor, etc.

[0219] Antennas 1 and 2 are used to transmit and receive electromagnetic wave signals. Each antenna in terminal device 100 can be used to cover one or more communication frequency bands. Different antennas can also be multiplexed to improve antenna utilization. For example, antenna 1 can be multiplexed as a diversity antenna for a wireless local area network. In some other embodiments, the antennas can be used in conjunction with a tuning switch.

[0220] The mobile communication module 150 can provide wireless communication solutions, including 2G / 3G / 4G / 5G, for use on the terminal device 100. The mobile communication module 150 may include at least one filter, switch, power amplifier, low-noise amplifier (LNA), etc. The mobile communication module 150 can receive electromagnetic waves via antenna 1, and perform filtering, amplification, and other processing on the received electromagnetic waves before transmitting them to a modem processor for demodulation. The mobile communication module 150 can also amplify the signal modulated by the modem processor and convert it into electromagnetic waves for radiation via antenna 1. In one embodiment of this application, at least some functional modules of the mobile communication module 150 may be housed in the processor 110. In another embodiment of this application, at least some functional modules of the mobile communication module 150 and at least some modules of the processor 110 may be housed in the same device.

[0221] The modem processor may include a modulator and a demodulator. The modulator modulates the low-frequency baseband signal to be transmitted into a mid-to-high frequency signal. The demodulator demodulates the received electromagnetic wave signal into a low-frequency baseband signal. The demodulator then transmits the demodulated low-frequency baseband signal to the baseband processor for processing. After processing by the baseband processor, the low-frequency baseband signal is transmitted to the application processor. The application processor outputs sound signals through audio devices (not limited to speaker 170A, receiver 170B, etc.) or displays images or videos through the display screen 194. In one embodiment of this application, the modem processor may be a separate device. In other embodiments, the modem processor may be independent of the processor 110 and housed within the same device as the mobile communication module 150 or other functional modules.

[0222] The wireless communication module 160 can provide solutions for wireless communication applications on the terminal device 100, including Wireless Local Area Networks (WLAN) (such as Wireless Fidelity (Wi-Fi) networks), Bluetooth (BT), Global Navigation Satellite System (GNSS), Frequency Modulation (FM), Near Field Communication (NFC), and Infrared (IR) technologies. The wireless communication module 160 can be one or more devices integrating at least one communication processing module. The wireless communication module 160 receives electromagnetic waves via antenna 2, modulates and filters the electromagnetic wave signals, and sends the processed signal to processor 110. The wireless communication module 160 can also receive signals to be transmitted from processor 110, modulate and amplify them, and then convert them into electromagnetic waves for radiation via antenna 2.

[0223] In one embodiment of this application, antenna 1 of terminal device 100 is coupled to mobile communication module 150, and antenna 2 is coupled to wireless communication module 160, enabling terminal device 100 to communicate with networks and other devices via wireless communication technology. The wireless communication technology may include Global System for Mobile Communications (GSM), General Packet Radio Service (GPRS), Code Division Multiple Access (CDMA), Wideband Code Division Multiple Access (WCDMA), Time Division Code Division Multiple Access (TD-SCDMA), Long Term Evolution (LTE), BT, GNSS, WLAN, NFC, FM, and / or IR technologies, etc. The GNSS may include the Global Positioning System (GPS), the Global Navigation Satellite System (GLONASS), the Beidou Navigation Satellite System (BDS), the Quasi-Zenith Satellite System (QZSS), and / or Satellite Based Augmentation Systems (SBAS).

[0224] Terminal device 100 implements display functions through a GPU, display screen 194, and application processor. The GPU is a microprocessor for image processing, connected to the display screen 194 and the application processor. The GPU is used to perform mathematical and geometric calculations and for graphics rendering. Processor 110 may include one or more GPUs, which execute program instructions to generate or modify display information.

[0225] The display screen 194 is used to display images, videos, etc. The display screen 194 includes a display panel. The display panel can be a Liquid Crystal Display (LCD), an Organic Light-Emitting Diode (OLED), an Active-Matrix Organic Light-Emitting Diode (AMOLED), a Flexible Light-Emitting Diode (FLED), a Minied, Microled, Micro-OLED, or a Quantum Dot Light-Emitting Diode (QLED), etc. In one embodiment of this application, the terminal device 100 may include one or N display screens 194, where N is a positive integer greater than 1.

[0226] Terminal device 100 can perform shooting functions through ISP, camera 193, video codec, GPU, display 194 and application processor.

[0227] The ISP is used to process data fed back from the camera 193. For example, when taking a picture, the shutter is opened, and light is transmitted through the lens to the camera's photosensitive element. The light signal is converted into an electrical signal, and the camera's photosensitive element transmits the electrical signal to the ISP for processing, converting it into an image visible to the naked eye. The ISP can also perform algorithmic optimization on image noise, brightness, and skin tone. The ISP can also optimize parameters such as exposure and color temperature of the shooting scene. In one embodiment of this application, the ISP can be set in the camera 193.

[0228] Camera 193 is used to capture still images or videos. An object is projected onto a photosensitive element by generating an optical image through the lens. The photosensitive element can be a charge-coupled device (CCD) or a complementary metal-oxide-semiconductor (CMOS) phototransistor. The photosensitive element converts the light signal into an electrical signal, which is then transmitted to an ISP for conversion into a digital image signal. The ISP outputs the digital image signal to a DSP for processing. The DSP converts the digital image signal into image signals in standard RGB, YUV, or other formats. In one embodiment of this application, the terminal device 100 may include one or N cameras 193, where N is a positive integer greater than 1.

[0229] A digital signal processor (DSP) is used to process digital signals. Besides digital image signals, it can also process other digital signals. For example, when terminal device 100 selects a frequency, the DSP can perform Fourier transforms on the frequency energy.

[0230] Video codecs are used to compress or decompress digital video. Terminal device 100 may support one or more video codecs. Thus, terminal device 100 can play or record videos in various encoding formats, such as Moving Picture Experts Group (MPEG) 1, MPEG 2, MPEG 3, MPEG 4, etc.

[0231] NPU stands for Neural Network (NN) computing processor. By borrowing the structure of biological neural networks, such as the transmission patterns between neurons in the human brain, it can rapidly process input information and continuously learn on its own. NPUs enable intelligent cognitive applications in terminal devices, such as image recognition, facial recognition, speech recognition, and text understanding.

[0232] Internal memory 121 may include one or more random access memory (RAM) and one or more non-volatile memory (NVM).

[0233] Random access memory can include static random-access memory (SRAM), dynamic random-access memory (DRAM), synchronous dynamic random-access memory (SDRAM), and double data rate synchronous dynamic random-access memory (DDR SDRAM, such as fifth-generation DDR SDRAM, which is generally called DDR5 SDRAM).

[0234] Non-volatile memory can include disk storage devices and flash memory.

[0235] Flash memory can be classified according to its operating principle, including NOR FLASH, NAND FLASH, 3D NAND FLASH, etc.; according to the level of the storage cell, including single-level cell (SLC), multi-level cell (MLC), triple-level cell (TLC), quad-level cell (QLC), etc.; and according to the storage specification, including universal flash storage (UFS) and embedded multi-media card (eMMC), etc.

[0236] The random access memory can be directly read and written by the processor 110. It can be used to store executable programs (such as machine instructions) of the operating system or other running programs, as well as user and application data.

[0237] Non-volatile memory can also store executable programs and user and application data, and can be pre-loaded into random access memory for direct reading and writing by the processor 110.

[0238] The external memory interface 120 can be used to connect to external non-volatile memory, thereby expanding the storage capacity of the terminal device 100. The external non-volatile memory communicates with the processor 110 through the external memory interface 120 to perform data storage functions. For example, music, video, and other files can be stored in the external non-volatile memory.

[0239] Internal memory 121 or external memory interface 120 is used to store one or more computer programs. The one or more computer programs are configured to be executed by processor 110. The one or more computer programs include multiple instructions, which, when executed by processor 110, can implement the screen display detection method executed on terminal device 100 in the above embodiments, so as to realize the screen display detection function of terminal device 100.

[0240] Terminal device 100 can implement audio functions, such as music playback and recording, through audio module 170, speaker 170A, receiver 170B, microphone 170C, headphone jack 170D, and application processor.

[0241] The audio module 170 is used to convert digital audio information into analog audio signals for output, and also to convert analog audio input into digital audio signals. The audio module 170 can also be used for encoding and decoding audio signals. In one embodiment of this application, the audio module 170 may be located in the processor 110, or some functional modules of the audio module 170 may be located in the processor 110.

[0242] The speaker 170A, also known as a "loudspeaker," is used to convert audio electrical signals into sound signals. The terminal device 100 can listen to music or make hands-free calls through the speaker 170A.

[0243] The receiver 170B, also known as the "earpiece," is used to convert audio electrical signals into sound signals. When the terminal device 100 answers a phone call or voice message, the receiver 170B can be brought close to the listener's ear to hear the voice.

[0244] Microphone 170C, also known as a "microphone" or "voice transducer," is used to convert sound signals into electrical signals. When making a phone call or sending a voice message, the user can speak by bringing their mouth close to microphone 170C, inputting the sound signal into microphone 170C. Terminal device 100 may be equipped with at least one microphone 170C. In some embodiments, terminal device 100 may be equipped with two microphones 170C, which, in addition to collecting sound signals, can also perform noise reduction. In other embodiments, terminal device 100 may be equipped with three, four, or more microphones 170C, which can collect sound signals, reduce noise, identify the sound source, and perform directional recording, etc.

[0245] The 170D headphone jack is used to connect wired headphones. The 170D headphone jack can be a USB 130 interface or a 3.5mm Open Mobile Terminal Platform (OMTP) standard interface, a CTIA (Cellular Telecommunications Industry Association of the USA) standard interface.

[0246] Buttons 190 include a power button, volume buttons, etc. Buttons 190 can be mechanical buttons or touch-sensitive buttons. Terminal device 100 can receive button input and generate key signal inputs related to user settings and function control of terminal device 100.

[0247] Motor 191 can generate vibration alerts. Motor 191 can be used for incoming call vibration alerts or for touch vibration feedback. For example, different vibration feedback effects can correspond to touch operations performed on different applications (such as taking photos, playing audio, etc.). Motor 191 can also correspond to different vibration feedback effects for touch operations performed on different areas of the display screen 194. Different application scenarios (such as time reminders, receiving messages, alarm clocks, games, etc.) can also correspond to different vibration feedback effects. The touch vibration feedback effect can also be customized.

[0248] Indicator 192 can be an indicator light, used to indicate charging status, power changes, or to indicate messages, missed calls, notifications, etc.

[0249] The SIM card interface 195 is used to connect a SIM card. The SIM card can be inserted into or removed from the SIM card interface 195 to make contact with and separate from the terminal device 100. The terminal device 100 can support one or N SIM card interfaces, where N is a positive integer greater than 1. The SIM card interface 195 can support Nano SIM cards, Micro SIM cards, SIM cards, etc. Multiple cards can be inserted into the same SIM card interface 195 simultaneously. The multiple cards can be of the same or different types. The SIM card interface 195 is also compatible with different types of SIM cards. The SIM card interface 195 is also compatible with external memory cards. The terminal device 100 interacts with the network through the SIM card to realize functions such as calls and data communication. In one embodiment of this application, the terminal device 100 uses an eSIM, i.e., an embedded SIM card. The eSIM card can be embedded in the terminal device 100 and cannot be separated from the terminal device 100. This application also provides a computer storage medium storing computer instructions. When the computer instructions are executed on the terminal device 100, the terminal device 100 performs the aforementioned related method steps to implement the face recognition method in the above embodiments.

[0250] This application also provides a computer program product that, when run on a computer, causes the computer to perform the aforementioned steps to implement the face recognition method described in the above embodiments.

[0251] In addition, embodiments of this application also provide an apparatus, which may specifically be a chip, component or module. The apparatus may include a connected processor and a memory; wherein, the memory is used to store computer execution instructions, and when the apparatus is running, the processor may execute the computer execution instructions stored in the memory to cause the chip to execute the face recognition methods in the above method embodiments.

[0252] In this embodiment, the terminal device, computer storage medium, computer program product or chip are all used to execute the corresponding methods provided above. Therefore, the beneficial effects they can achieve can be referred to the beneficial effects in the corresponding methods provided above, and will not be repeated here.

[0253] Through the above description of the embodiments, those skilled in the art can clearly understand that, for the sake of convenience and brevity, only the division of the above functional modules is used as an example. In actual applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above.

[0254] In the several embodiments provided in this application, it should be understood that the disclosed apparatus and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of modules or units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another device, or some features may be ignored or not executed. Furthermore, the mutual coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between devices or units may be electrical, mechanical, or other forms.

[0255] The units described as separate components may or may not be physically separate. A component shown as a unit can be one or more physical units; that is, it can be located in one place or distributed in multiple different locations. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0256] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0257] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a readable storage medium. Based on this understanding, the technical solutions of the embodiments of this application, essentially or in other words, the parts that contribute to the prior art, or all or part of the technical solutions, can be embodied in the form of a software product. The software product is stored in a storage medium and includes several instructions to cause a device (which may be a microcontroller, chip, etc.) or processor to execute all or part of the steps of the methods of the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0258] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of this application and are not intended to limit it. Although this application has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can be made to the technical solutions of this application without departing from the spirit and scope of the technical solutions of this application.

Claims

1. A face recognition method, applied to a terminal device, characterized in that, The terminal device includes a camera and an ambient light sensor, and the method includes: In response to a user's operation that triggers facial recognition, the camera captures an image to be recognized and the ambient light sensor obtains the current ambient light level. If the current environment is determined to be a low-light environment based on the illuminance, the image to be identified is input into a pre-trained image enhancement model to obtain an enhanced image of the image to be identified. The first facial feature is extracted from the enhanced image using a preset facial feature extraction model; If the first similarity between the first facial feature and the second facial feature in the preset facial template image is greater than or equal to a preset similarity threshold, the user's identity verification is deemed successful; or, If the first similarity is less than the similarity threshold, the face template image is subjected to image degradation processing to obtain the degraded image of the face template image; The face feature extraction model is used to extract a third face feature from the image to be identified, and the face feature extraction model is used to extract a fourth face feature from the degraded image; If the second similarity between the third facial feature and the four facial features is greater than or equal to the similarity threshold, the user's identity verification is deemed successful.

2. The face recognition method according to claim 1, characterized in that, The method further includes: If the second similarity is less than the similarity threshold, it is determined that the user's identity verification has failed.

3. The face recognition method according to claim 1, characterized in that, The method further includes: The ambient light sensor is used to collect multiple continuous illuminance values ​​of the current environment; The average value of the multiple consecutive illuminance values ​​is determined based on the moving average algorithm; If the average value is less than a preset illuminance threshold, the current environment is determined to be a dark environment.

4. The face recognition method according to claim 1, characterized in that, The step of extracting the first facial feature from the enhanced image using a preset facial feature extraction model includes: Based on a preset face detection model, determine whether there is a face to be identified in the enhanced image; If a face to be identified exists in the enhanced image, the face region image is extracted from the enhanced image; The first facial feature is extracted from the facial region image using the facial feature extraction model.

5. The face recognition method according to claim 4, characterized in that, The method further includes: If the enhanced image does not contain the face to be identified, the user's identity verification is deemed to have failed.

6. The face recognition method according to claim 1, characterized in that, The method further includes: If the current environment is determined to be a non-dark environment based on the illuminance, a preset face detection model is used to determine whether there is a face to be identified in the image to be identified. If a face to be identified exists in the image to be identified, the face region image is extracted from the image to be identified; The face feature extraction model is used to extract the fifth face feature from the face region image; If the third similarity between the fifth facial feature and the second facial feature is determined to be greater than or equal to the similarity threshold, the user's identity verification is deemed successful; or, If the third similarity is determined to be less than the similarity threshold, the user's identity verification is deemed to have failed.

7. The face recognition method according to claim 1, characterized in that, The image enhancement model includes a first model and a second model. The step of inputting the image to be recognized into the pre-trained image enhancement model to obtain the enhanced image of the image to be recognized includes: The first model is used to denoise the image to be identified, and the denoised image to be identified is output. The second model is used to enhance the brightness of the denoised image to be identified, and the output brightened image is used as the enhanced image.

8. The face recognition method according to claim 1, characterized in that, The training method for the image enhancement model includes: Multiple sample images are acquired, including multiple first images and multiple second images, wherein the image noise of any first image is greater than the image noise of any second image; A training set and a test set are constructed based on the multiple sample images. The training set is used to iterate and update the preset initial model at least once until the image enhancement model that meets the preset requirements is obtained.

9. The face recognition method according to claim 8, characterized in that, The preset requirements include a combination of one or more of the following requirements: the loss function corresponding to the initial model reaches the convergence condition; the test results obtained by testing the initial model using the test set indicate that the model performance of the initial model meets expectations; and the number of at least one iteration update reaches a preset iteration number threshold.

10. The face recognition method according to claim 8, characterized in that, The initial model includes a first initial model, and any one of the at least one iterative updates includes: Input any first image from the training set into the first initial model, and use the first initial model to denoise the first image to obtain a first denoised image corresponding to the first image, and a first difference image between the first image and the first denoised image. A first loss value corresponding to the first difference image is determined using a preset first loss function, and the first loss value is used to indicate the noise level in the first difference image; If the first loss value is less than or equal to a preset first threshold, the first initial model is updated again using another first image.

11. The face recognition method according to claim 10, characterized in that, Any of the updates in the at least one iterative update also includes: If the first loss value is greater than the first threshold, the first difference image is fused with any second image in the training set to obtain the fused image corresponding to the second image. The fused image is input into the first initial model, and the first initial model is used to denoise the fused image to obtain the second denoised image corresponding to the fused image. A second loss value between the second denoised image and any corresponding second image is determined using a preset second loss function; If the second loss value is greater than the preset second threshold, the first initial model is updated again using the next first image and the next second image.

12. The face recognition method according to claim 11, characterized in that, The initial model further includes a second initial model, and any one of the at least one iterative updates further includes: If the second loss value is less than or equal to the second threshold, the second denoised image corresponding to any second image is input into the second initial model, and the brightness of the second denoised image is enhanced by the second initial model to obtain the brightness-enhanced image corresponding to the second denoised image. A third loss value is determined between the brightness-enhanced image and any corresponding second image using a preset third loss function; If the third loss value is greater than a preset third threshold, the second initial model is updated again using the second denoised image corresponding to the next second image; or, If the third loss value is less than or equal to the third threshold, the initial model is determined to meet the preset requirements, and the initial model is used as the image enhancement model.

13. The face recognition method according to claim 2, characterized in that, The step of performing image degradation processing on the face template image to obtain the degraded image of the face template image includes: Obtain the preset third image; The third image is input into the image enhancement model, and the first model of the image enhancement model is used to denoise the third image to obtain the third denoised image corresponding to the third image, and the second difference image between the third image and the third denoised image. Reduce the brightness of the face template image to obtain an image with reduced brightness; The second difference image is fused with the image after the brightness is reduced to obtain the degraded image of the face template image.

14. A terminal device, characterized in that, The terminal device includes a memory and a processor: The memory is used to store program instructions; The processor is configured to read and execute the program instructions stored in the memory, and when the program instructions are executed by the processor, the terminal device performs the face recognition method as described in any one of claims 1 to 13.

15. A computer storage medium, characterized in that, The computer storage medium stores program instructions that, when executed on a terminal device, cause the processor of the terminal device to perform the face recognition method as described in any one of claims 1 to 13.

Images