Image processing method, electronic equipment and storage medium

Abstract

The invention discloses an image processing method, electronic equipment and a storage medium, and the method comprises the steps: determining the current real skin color information of an initial face image, and the current real skin color information comprises a current real skin color value; determining a current apparent skin color value of the initial face image; determining a current scene skin color value of the initial face image according to the current real skin color value and the current apparent skin color value; and according to the current scene skin color value, skin color correction is carried out on the initial face image to obtain a target face image, and the image quality of the target face image is higher than that of the initial face image. According to the method, skin color correction is carried out on the initial face image based on the current scene skin color value, the recognition accuracy of recognition of the apparent skin color of the face image is improved, the image quality of the face image is improved, and the shooting quality of the electronic equipment for shooting the face image is improved.

Description

Technical Field

[0001] This application relates to the field of image processing technology, and in particular to an image processing method, electronic device and storage medium. Background Technology

[0002] With the development of electronic devices, more and more users are taking portrait photos using electronic devices. In order to ensure that the captured facial images are clear, it is usually necessary to perform image correction on the facial preview images captured by the camera module of the electronic device.

[0003] Currently, the process of image correction for the preview face image captured by the camera module mainly involves recognizing the apparent skin color (skin color as observed by the human eye) of the preview face image and then correcting the image based on the recognized apparent skin color.

[0004] However, when the lighting in the shooting scene is too bright or too dark, the face image in the photo preview is dark, and it is easy to make mistakes in recognizing the apparent skin color of the face image in the photo preview. If the face image in the photo preview is corrected based on the incorrect apparent skin color, the resulting corrected face image will have low image quality, resulting in low shooting quality of face images captured by electronic devices. Summary of the Invention

[0005] In view of the above, embodiments of this application provide an image processing method, an electronic device, and a storage medium to overcome the problems of the prior art.

[0006] In a first aspect, embodiments of this application provide an image processing method, which includes: determining the current true skin color information of an initial face image, the current true skin color information including the current true skin color value; determining the current apparent skin color value of the initial face image; determining the current scene skin color value of the initial face image based on the current true skin color value and the current apparent skin color value; and performing skin color correction on the initial face image based on the current scene skin color value to obtain a target face image, wherein the image quality of the target face image is higher than the image quality of the initial face image.

[0007] The solution provided in this application verifies the current apparent skin color value based on the current real skin color value of the initial face image to obtain the current scene skin color value. This avoids errors in recognizing the apparent skin color of the face image when the shooting scene is too bright or too dark, thus improving the accuracy of recognizing the apparent skin color of the face image. Furthermore, it corrects the skin color of the initial face image based on the current scene skin color value, thereby improving the image quality of the face image and enhancing the shooting quality of face images captured by electronic devices.

[0008] In some optional embodiments, before determining the current true skin color information of the initial face image, the image processing method further includes: acquiring a preview image stream; and when it is determined that a face image frame exists in the preview image stream, generating an initial face image based on the face image frame.

[0009] The solution provided in this embodiment generates an initial face image based on the detected face image in the preview image stream, thereby improving the accuracy of the initial face image generation.

[0010] In some optional embodiments, when it is determined that a face image frame exists in the preview image stream, an initial face image is generated based on the face image frame, including: when it is determined that a face image frame exists in the preview image stream, the face image frame is cropped based on the face detection box of the face image frame to obtain a cropped face image; the cropped face image is scaled to obtain the initial face image.

[0011] The solution provided in this embodiment crops the face image frame based on the face detection bounding box of the face detection to obtain a cropped face image, and obtains an initial face image based on the cropped face image, which helps to improve the accuracy of the initial face image generation.

[0012] In some optional embodiments, when it is determined that a face image frame exists in the preview image stream, the face image frame is cropped according to the face detection box of the face image frame to obtain a cropped face image, including: when it is determined that a face image frame exists in the preview image stream, the face detection box is enlarged to obtain a cropping box; the face image frame is cropped according to the cropping box to obtain a cropped face image.

[0013] The solution provided in this embodiment cropps face image frames based on a cropping box obtained by magnifying the face detection box. This avoids incomplete face images in the cropped face image caused by setting the face detection box too small, thus improving the cropping accuracy of face image frames.

[0014] In some optional embodiments, determining the current true skin color information of the initial face image includes: inputting the initial face image into the true skin color classification model to obtain the current true skin color information. The true skin color classification model is trained based on the target historical skin color image. The target historical skin color image is labeled with historical true skin color information, which includes historical true skin color values.

[0015] The solution provided in this embodiment performs real skin color detection on the initial face image based on the real skin color classification model, thereby improving the detection efficiency and accuracy of real skin color detection on the initial face image.

[0016] In some optional embodiments, before inputting the initial face image into the real skin color classification model to obtain the current real skin color information, the image processing method further includes: acquiring an initial historical skin color image; filtering the initial historical skin color image to obtain a target historical skin color image; and inputting the target historical skin color image into the classification model for training to obtain the real skin color classification model.

[0017] The solution provided in this embodiment uses a target historical skin color image obtained by filtering and processing the initial historical skin color image to train a classification model to obtain a real skin color classification model. This reduces the number of training samples for the classification model and helps to improve the training efficiency of the classification model.

[0018] In some optional embodiments, the initial historical skin color images are filtered to obtain target historical skin color images, including: classifying the initial historical skin color images into categories to obtain preset types of skin color images, each type of skin color image corresponding to a real skin color; constructing a real skin color template color card based on each real skin color to obtain preset types of real skin color template color cards; and filtering the corresponding skin color image based on each real skin color template color card to obtain the target historical skin color image.

[0019] The solution provided in this embodiment filters images of a skin tone type based on a real skin tone template color card constructed for each real skin tone, thereby improving the sample data quality of the target historical skin tone images and improving the detection accuracy of the real skin tone classification model.

[0020] In some optional embodiments, the target historical skin color image is obtained by filtering images of a corresponding skin color type according to each real skin color template color card, including: removing skin color type images that do not match the corresponding real skin color template color card from each skin color type image.

[0021] The solution provided in this embodiment eliminates skin tone images based on the matching degree between each skin tone image and a corresponding real skin tone template color card, thereby improving the accuracy of screening skin tone images.

[0022] In some optional embodiments, determining the current apparent skin color value of the initial face image includes: determining the initial apparent skin color value of the initial face image; and determining the current apparent skin color value based on a pre-constructed apparent skin color template color card and the initial apparent skin color value.

[0023] The solution provided in this embodiment determines the current apparent skin tone value based on the apparent skin tone template color card and the initial apparent skin tone value, which helps to improve the accuracy of the current apparent skin tone value.

[0024] In some optional embodiments, determining the initial apparent skin color value of the initial face image includes: determining multiple skin color features of the initial face image; and calculating the initial apparent skin color value based on the multiple skin color features.

[0025] The solution provided in this embodiment calculates the initial apparent skin color value based on multiple skin color features of the initial face image, thereby improving the accuracy of the initial apparent skin color value calculation.

[0026] In some optional embodiments, determining multiple skin color features of the initial face image includes: performing face analysis on the initial face image to obtain an initial face skin region image; and determining a skin color feature for each pixel in the initial face skin region image to obtain multiple skin color features.

[0027] The solution provided in this embodiment obtains multiple skin color features through face analysis of the initial face image. Face analysis can exclude non-facial skin areas such as lips, eyebrows, and eyes, thereby improving the accuracy of multiple skin color features.

[0028] In some optional embodiments, face parsing is performed on the initial face image to obtain an initial face skin region image, including: inputting the initial face image to a face parsing model to obtain the initial face skin region image, wherein the face parsing model is obtained by training a deep learning neural network model based on historical face images, and the historical face images are labeled with historical face skin regions.

[0029] The solution provided in this embodiment performs face analysis on the initial face image based on a face analysis model, which improves the efficiency and accuracy of face analysis on the initial face image.

[0030] In some optional embodiments, determining a skin color feature for each pixel in the initial facial skin region image includes: converting the YUV value of each pixel into a Lab value; and determining each Lab value as a skin color feature for the corresponding pixel.

[0031] The solution provided in this embodiment is based on the fact that human skin color is mainly affected by the content of melanin, carotenoids and hemoglobin. The Lab value can represent colors that the YUV value cannot represent. Based on the Lab value to characterize skin color features, it is beneficial to improve the accuracy of apparent skin color values.

[0032] In some optional embodiments, calculating an initial apparent skin color value based on multiple skin color features includes: clustering multiple skin color features into a preset number of skin color clusters, each skin color cluster including at least one skin color feature; determining a skin color cluster percentage for each skin color cluster, each skin color cluster percentage being used to characterize the proportion of the number of skin color features contained in the corresponding skin color cluster to the total number of multiple skin color features; and calculating a weighted average of multiple skin color features based on each skin color cluster percentage to obtain the initial apparent skin color value.

[0033] The solution provided in this embodiment calculates an initial apparent skin color value by performing a weighted average of multiple skin color features based on the skin color clustering ratio of each skin color cluster, thereby improving the accuracy of the initial apparent skin color value calculation.

[0034] In some optional embodiments, the initial apparent skin color value is obtained by performing a weighted average calculation on multiple skin color features based on the proportion of each skin color cluster. This includes: filtering a preset number of skin color clusters according to the proportion of each skin color cluster to obtain a target number of skin color clusters, wherein the target number of skin color clusters includes a target number of skin color features; and performing a weighted average calculation on the target number of skin color features based on the target number of skin color clusters to obtain the initial apparent skin color value.

[0035] The solution provided in this embodiment obtains the target number of skin color clusters by filtering a preset number of skin color clusters, effectively eliminating abnormal skin color values, and further improves the accuracy of the initial apparent skin color value calculation by performing a weighted average calculation on the target number of skin color clusters based on the target skin color cluster ratio.

[0036] In some optional embodiments, filtering a preset number of skin color clusters based on the proportion of each skin color cluster to obtain a target number of skin color clusters includes: removing skin color clusters whose proportion of skin color clusters is less than a proportion threshold from the preset number of skin color clusters to obtain the target number of skin color clusters.

[0037] The solution provided in this embodiment filters a preset number of skin color clusters based on the skin color clustering ratio, which helps to improve the accuracy of the initial apparent skin color value calculation.

[0038] In some optional embodiments, the apparent skin tone template color chart includes multiple preset apparent skin tone values. Determining the current apparent skin tone value based on the pre-constructed apparent skin tone template color chart and the initial apparent skin tone value includes: calculating a skin tone similarity between the initial apparent skin tone value and each preset apparent skin tone value to obtain multiple skin tone similarities; determining the skin tone similarity among the multiple skin tone similarities that is greater than or equal to a similarity threshold as the target skin tone similarity; and determining the target preset apparent skin tone value corresponding to the target skin tone similarity as the current apparent skin tone value.

[0039] The solution provided in this embodiment determines the current apparent skin color value based on the skin color similarity between the initial apparent skin color value and each preset apparent skin color value, thereby improving the accuracy of the current apparent skin color value.

[0040] In some optional embodiments, the current true skin color information also includes the current confidence level. Based on the current true skin color value and the current apparent skin color value, the current scene skin color value of the initial face image is determined, including: when the current confidence level is greater than or equal to a confidence level threshold, calculating a first skin color difference between the current true skin color value and the current apparent skin color value; when the first skin color difference is less than a difference threshold, determining the current true skin color value as the current scene skin color value.

[0041] The solution provided in this embodiment verifies the current apparent skin color value based on the current confidence level and the current actual skin color value, thereby obtaining the current scene skin color value and improving the accuracy of the current scene skin color value.

[0042] In some optional embodiments, the image processing method further includes: when the current confidence level is less than a confidence level threshold, determining whether the face corresponding to the initial face image is in a normal lighting environment; when it is determined that the face is in a normal lighting environment and the first skin color difference is less than a difference threshold, determining the current real skin color value as the current scene skin color value; when it is determined that the face is in a normal lighting environment and the first skin color difference is greater than or equal to the difference threshold, determining the current apparent skin color value as the current scene skin color value.

[0043] The solution provided in this embodiment determines the current scene skin color value based on the skin color difference between the current true skin color value and the current apparent skin color value when the current confidence level of the current true skin color value is low and the face is in a normal lighting environment, thereby further improving the accuracy of the current scene skin color value.

[0044] In some optional embodiments, the initial face image is skin color corrected according to the skin color value of the current scene to obtain the target face image, including: determining the skin color compensation value of the initial face image according to the skin color value of the current scene; and correcting the skin color of the initial face image according to the skin color compensation value to obtain the target face image.

[0045] The solution provided in this embodiment corrects the skin color of the initial face image based on the skin color compensation value corresponding to the skin color value of the current scene, thereby improving the image quality of the face image and improving the shooting quality of the face image captured by the electronic device.

[0046] In some optional embodiments, determining the skin color compensation value of the initial face image based on the skin color value of the current scene includes: calculating a second skin color difference between a preset scene skin color value and the current scene skin color value, wherein the preset scene skin color value corresponds to the target clarity of the target face image; and determining the second skin color difference as the skin color compensation value.

[0047] The solution provided in this embodiment determines the skin color compensation value based on the skin color difference between the current scene skin color value and the preset scene skin color value, thereby improving the accuracy of the skin color compensation value.

[0048] Secondly, embodiments of this application provide an image processing apparatus, which includes a first determining module, a second determining module, a third determining module, and a correction module. The first determining module is used to determine the current true skin color information of an initial face image, the current true skin color information including a current true skin color value; the second determining module is used to determine the current apparent skin color value of the initial face image; the third determining module is used to determine the current scene skin color value of the initial face image based on the current true skin color value and the current apparent skin color value; the correction module is used to perform skin color correction on the initial face image based on the current scene skin color value to obtain a target face image, the image quality of the target face image being higher than the image quality of the initial face image.

[0049] Thirdly, embodiments of this application provide an electronic device, which includes: one or more processors and a memory; the memory is coupled to one or more processors, and the memory is used to store computer program code, the computer program code including computer instructions, and one or more processors call the computer instructions to cause the electronic device to perform the image processing method provided in the first aspect above.

[0050] Fourthly, embodiments of this application provide a chip system applied to an electronic device. The chip system includes one or more processors, which are used to invoke computer instructions to cause the electronic device to perform the image processing method provided in the first aspect above.

[0051] In some alternative embodiments, the chip system further includes a memory connected to one or more processors via circuits or wires.

[0052] In some alternative embodiments, the chip system also includes a communication interface.

[0053] Fifthly, embodiments of this application provide a computer-readable storage medium including instructions that, when executed on an electronic device, cause the electronic device to perform the image processing method provided in the first aspect above.

[0054] In a sixth aspect, embodiments of this application provide a computer program product that, when run on an electronic device, causes the electronic device to perform the image processing method provided in the first aspect above.

[0055] It is understood that the beneficial effects of the second to sixth aspects mentioned above can be found in the relevant descriptions in the first aspect mentioned above, and will not be repeated here. Attached Figure Description

[0056] To more clearly illustrate the technical solutions in the embodiments of this application, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0057] Figure 1 This diagram illustrates a scenario of existing face image capture.

[0058] Figure 2 A schematic diagram of the software system of an electronic device provided in an embodiment of this application is shown.

[0059] Figure 3 A schematic flowchart of an image processing method provided in an embodiment of this application is shown.

[0060] Figure 4 This illustration shows a scene diagram of an apparent skin tone template color card in the image processing method provided in an embodiment of this application.

[0061] Figure 5 This illustration shows another schematic flowchart of the image processing method provided in an embodiment of this application.

[0062] Figure 6 This illustration shows a scenario diagram of a cropping frame in the image processing method provided in an embodiment of this application.

[0063] Figure 7 This illustration shows another schematic flowchart of the image processing method provided in an embodiment of this application.

[0064] Figure 8 This illustration shows a scene diagram of a real skin tone template color card in the image processing method provided in the embodiments of this application.

[0065] Figure 9 A structural block diagram of an image processing apparatus provided in an embodiment of this application is shown.

[0066] Figure 10 A schematic diagram of a hardware structure of an electronic device provided in an embodiment of this application is shown.

[0067] Figure 11 A functional block diagram of an electronic device provided in an embodiment of this application is shown. Detailed Implementation

[0068] The embodiments of this application are described in detail below. Examples of the embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals denote the same or similar elements or elements having the same or similar functions throughout. The embodiments described below with reference to the accompanying drawings are exemplary and are only used to explain this application, and should not be construed as limiting this application.

[0069] The following disclosure provides many different implementations or examples for carrying out different structures of this application. To simplify the disclosure, specific examples of components and arrangements are described below. Of course, these are merely examples and are not intended to limit the scope of this application. Furthermore, reference numerals and / or reference letters may be repeated in different examples; such repetition is for simplification and clarity and does not in itself indicate a relationship between the various implementations and / or arrangements discussed.

[0070] With the development of electronic devices, more and more users are taking portrait photos using electronic devices. In order to ensure that the captured facial images are clear, it is usually necessary to perform image correction on the facial preview images captured by the camera module of the electronic device.

[0071] Currently, the process of image correction for the preview face image captured by the camera module mainly involves recognizing the apparent skin color (skin color as observed by the human eye) of the preview face image and then correcting the image based on the recognized apparent skin color.

[0072] However, when the lighting in the shooting scene is too bright or too dark, the preview image of the face is relatively dark, and errors can easily occur in recognizing the apparent skin tone of the preview image. If image correction is performed on the preview image based on the incorrectly identified skin tone, the resulting corrected image will have lower quality, leading to lower image quality when the electronic device captures the face. Figure 1 As shown.

[0073] To address the aforementioned issues, the image processing method, electronic device, and storage medium provided in this application determine the current true skin color information of an initial face image (including the current true skin color value), determine the current apparent skin color value of the initial face image, determine the current scene skin color value of the initial face image based on the current true skin color value and the current apparent skin color value, and perform skin color correction on the initial face image based on the current scene skin color value to obtain a target face image. The image quality of the target face image is higher than that of the initial face image. This achieves the verification of the current apparent skin color value based on the current true skin color value of the initial face image to obtain the current scene skin color value. This avoids errors in recognizing the apparent skin color of the face image when the shooting scene of the face image is too bright or too dark, thus improving the recognition accuracy of the apparent skin color of the face image. Furthermore, the skin color correction of the initial face image based on the current scene skin color value improves the image quality of the face image, which is beneficial to improving the shooting quality of the face image captured by the electronic device.

[0074] The technical solutions in the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings.

[0075] The image restoration method provided in this application can be applied to electronic devices. Electronic devices may include various terminal devices, which may also be referred to as terminals, user equipment (UE), mobile stations (MS), mobile terminals (MT), etc.

[0076] Terminal devices can include mobile phones, robot vacuum cleaners, drones, smart TVs, wearable devices, personal digital assistants (PDAs), computers with wireless transceiver capabilities, virtual reality (VR) terminal devices, augmented reality (AR) terminal devices, wireless terminals in industrial control, wireless terminals in self-driving, wireless terminals in remote medical surgery, wireless terminals in smart grids, wireless terminals in transportation safety, wireless terminals in smart cities, and wireless terminals in smart homes. The type of terminal device is not limited here; it can be configured according to actual needs.

[0077] The software system of an electronic device can adopt a layered architecture, event-driven architecture, microkernel architecture, microservice architecture, or cloud architecture. This application uses the layered architecture Android system as an example to illustrate the software structure of an electronic device.

[0078] Please see Figure 2 This illustration shows a schematic diagram of the software system of an electronic device according to an embodiment of this application. The software system includes several layers, each with a clear role and division of labor, and the layers communicate with each other through a software interface. In some embodiments, the Android system is divided into four layers, from top to bottom: the application layer, the application framework layer, the system library, and the kernel layer.

[0079] The application layer can include a range of applications, such as camera applications, gallery applications, calling applications, wireless local area network (WLAN) applications, video applications, media provider applications, filesystem in userspace (FUSE) applications, etc.

[0080] The camera application can be used to acquire a preview image stream and then to obtain a face image based on that preview image stream.

[0081] Media providers are used to create or access multimedia files within FUSE. Applications in the application layer can create or access multimedia files within FUSE through a MediaProvider.

[0082] FUSE is used to store multimedia files created by media providers. Of course, in other embodiments, FUSE can also be used to store other data.

[0083] The application framework layer provides an Application Programming Interface (API) and programming framework for applications in the application layer. The application framework layer includes predefined functions. For example, it may include a window manager, content provider, resource manager, and view system.

[0084] The window manager is used to manage windowed applications. It can retrieve screen size, determine the presence of a status bar, lock the screen, and capture screenshots, among other things.

[0085] Content providers store and retrieve data, making that data accessible to applications. This data can include videos, images, audio, phone calls made and received, browsing history and bookmarks, phone books, and more.

[0086] A view system includes visual controls, such as controls for displaying text and controls for displaying images. View systems can be used to build applications. A display interface can consist of one or more views. For example, a display interface including a text notification icon could include views for displaying text and views for displaying images.

[0087] The file explorer provides applications with various resources, such as localized strings, icons, images, layout files, video files, and more.

[0088] System libraries may include Surface Manager, Media Libraries, Android Rruntime, etc.

[0089] The Android runtime consists of the core libraries and the virtual machine. The Android runtime is responsible for scheduling and managing the Android system. The core libraries comprise two parts: one part contains the functionalities that Java needs to call, and the other part consists of the Android core libraries. The application layer and application framework layer run in the virtual machine. The virtual machine executes the Java files of the application layer and application framework layer as binary files. The virtual machine is used to perform functions such as object lifecycle management, stack management, thread management, security and exception management, and garbage collection.

[0090] The Surface Manager is used to manage the display subsystem and provides the blending of 2D and 3D layers for multiple applications.

[0091] The media library supports playback and recording of various common audio and video formats, as well as still image files. It supports multiple audio and video encoding formats, such as MPEG4, H.264, MP3, AAC, AMR, JPG, and PNG.

[0092] The kernel layer can include modules such as camera driver, display driver, Wi-Fi driver, Bluetooth driver, and audio driver.

[0093] Understandable Figure 2 The layers in the illustrated software structure and the components contained in each layer do not constitute a specific limitation on the electronic device. In other embodiments of this application, the electronic device may include more or fewer layers than illustrated, and each layer may include more or fewer components; this application does not impose any limitations.

[0094] Please see Figure 3This document illustrates a flowchart of an image processing method provided in one embodiment of this application. In a specific embodiment, the image processing method can be applied to an electronic device. The following example uses an electronic device to illustrate this method. Figure 3 The process shown is described in detail. The image processing method may include the following steps 110 to 140.

[0095] Step 110: Determine the current true skin color information of the initial face image.

[0096] In this embodiment of the application, when a user needs to capture a facial image, he / she can send a shooting command to an electronic device. The electronic device receives and responds to the shooting command, acquires an initial facial image, and determines the current real skin color information of the initial facial image.

[0097] The initial face image can be any of the face preview stream images during the photo preview process or face frame images during the video recording process, and there is no limitation here.

[0098] Current true skin color information can include the current true skin color value and the current confidence level. The current true skin color value represents the natural color of the skin and is determined by factors such as pigments in the skin (e.g., melanin, carotenoids, etc.) and hemoglobin in the blood. The current confidence level represents the degree of certainty regarding the current true skin color value.

[0099] In some implementations, when a user needs to capture a facial image, a capture command can be sent to an electronic device. The electronic device receives and responds to the capture command, acquires an initial facial image, and inputs the initial facial image into a true skin color classification model. The true skin color classification model receives and responds to the initial facial image, performs skin color detection on the initial facial image, obtains and outputs the current true skin color information. By performing true skin color detection on the initial facial image based on the true skin color classification model, the detection efficiency and accuracy of true skin color detection on the initial facial image are improved.

[0100] The true skin color classification model is trained on target historical skin color images. These images are labeled with historical true skin color information, including historical true skin color values ​​and historical confidence scores. Historical true skin color values ​​represent the natural color of historical skin, while historical confidence scores represent the degree of certainty regarding the historical true skin color values.

[0101] The classification model can be any of the following: Logistic Regression, Support Vector Machine (SVM), Decision Tree, Random Forest, K-Nearest Neighbors (KNN), Naive Bayes, Neural Networks, or Gradient Boosting Machine (GBM), etc., without limitation here.

[0102] In some implementations, the electronic device may be equipped with an input panel. When a user needs to capture a face image, they can input a shooting command on the input panel of the electronic device. For example, the user can input the shooting command by handwriting on the input panel of the electronic device or by pressing a button on the input panel of the electronic device. The electronic device receives the shooting command through the input panel.

[0103] In some implementations, the electronic device may be equipped with a voice recognition module. When a user needs to capture a facial image, they can send voice information within the voice acquisition range of the voice recognition module. The voice recognition module collects the voice information sent by the user and performs voice recognition on the collected voice information to obtain a voice recognition result. When it is determined that the voice recognition result contains keywords used to instruct the electronic device to capture a facial image, such as the keyword "capture a facial image", or the keywords "capture" and "facial image", it is determined that a capture instruction has been received.

[0104] As an example, if the user sends a voice message: "Take a picture of the user's face," then the voice recognition result contains the keywords "take a picture" and "face image," confirming that the shooting instruction has been received.

[0105] In some implementations, when a user needs to capture a facial image, a shooting command can be sent to the client. The client receives and responds to the shooting command, and forwards the shooting command to the electronic device via the network. The electronic device receives the shooting command forwarded by the client.

[0106] The client connects to the electronic device via a network and interacts with the electronic device through the network. The client can be any of the following: a mobile client (e.g., a mobile phone client, a PDA client, a Tablet PC client, a laptop client, a smartwatch client, a smart bracelet client, or a wearable client) or a fixed client (e.g., a desktop computer client, a smart panel client). The type of client is not limited here and can be configured according to actual needs.

[0107] The network can be any of the following: ZigBee network, Bluetooth (BT) network, Wireless Fidelity (Wi-Fi) network, Thread network, Long Range Radio (LoRa) network, Low-Power Wide-Area Network (LPWAN), infrared network, Narrow Band Internet of Things (NB-IoT), Controller Area Network (CAN), Digital Living Network Alliance (DLNA) network, Wide Area Network (WAN), Local Area Network (LAN), Metropolitan Area Network (MAN), or Wireless Personal Area Network (WPAN). The type of network is not limited here; it can be configured according to actual needs.

[0108] Step 120: Determine the current apparent skin color value of the initial face image.

[0109] In this embodiment of the application, the electronic device can determine the initial apparent skin color value of the initial face image, and determine the current apparent skin color value based on the pre-constructed apparent skin color template color card and the initial apparent skin color value. Determining the current apparent skin color value based on the apparent skin color template color card and the initial apparent skin color value is beneficial to improving the accuracy of the current apparent skin color value.

[0110] The initial apparent skin color value is used to characterize the skin color initially observed by the human eye, and the current apparent skin color value is used to characterize the skin color currently observed by the human eye. The pre-constructed apparent skin color template color card can include multiple preset apparent skin color values, and each preset apparent skin color value can be used to characterize a preset skin color observed by the human eye.

[0111] As an example, a pre-constructed apparent skin tone template color chart can include 10 preset apparent skin tone values, each of which can be used to characterize a preset skin tone as observed by the human eye, such as... Figure 4 As shown.

[0112] Regarding the process by which the electronic device determines the initial apparent skin color value of the initial face image, in some embodiments, the electronic device can determine multiple skin color features of the initial face image and calculate the initial apparent skin color value based on the multiple skin color features. Calculating the initial apparent skin color value based on the multiple skin color features of the initial face image improves the accuracy of the initial apparent skin color value calculation.

[0113] Each skin color feature can correspond to a pixel in the facial skin image of the initial face image.

[0114] Regarding the process by which the aforementioned electronic device determines multiple skin color features of an initial face image, in some embodiments, the electronic device can perform face analysis on the initial face image to obtain an initial face skin region image, and determine a skin color feature for each pixel in the initial face skin region image to obtain multiple skin color features. By obtaining multiple skin color features through face analysis of the initial face image, face analysis can exclude non-face skin regions such as lips, eyebrows, and eyes, thereby improving the accuracy of multiple skin color features.

[0115] The initial facial skin region image is the image corresponding to the facial skin region in the facial image. The facial image includes facial skin regions and non-facial skin regions. The non-facial skin regions are the facial regions corresponding to lips, eyebrows, eyes and their appendages.

[0116] Regarding the process by which the aforementioned electronic device performs face analysis on an initial face image to obtain an initial face skin region image, in some embodiments, the electronic device can input the initial face image to a face analysis model. The face analysis model receives and responds to the initial face image, performs skin analysis on the initial face image, obtains and outputs the initial face skin region image. By performing face analysis on the initial face image based on the face analysis model, the analysis efficiency and accuracy of face analysis on the initial face image are improved.

[0117] Among them, the face analysis model can be obtained by training a deep learning neural network model based on historical face images, which are labeled with the skin regions of the historical faces.

[0118] Deep learning neural network models can be Convolutional Neural Networks (CNN), Deep Belief Networks (DBN), Stacked Auto Encoder Networks (SAE), Recurrent Neural Networks (RNN), Deep Neural Networks (DNN), Long Short-Term Memory (LSTM), or Gated Recurring Units (GRU), etc. The type of deep learning neural network model is not limited here; it can be set according to actual needs.

[0119] Regarding the process by which the aforementioned electronic device determines a skin color feature for each pixel in the initial facial skin region image, in some embodiments, the electronic device can convert the YUV value of each pixel into a Lab value and determine each Lab value as a skin color feature for the corresponding pixel. Since facial skin color is mainly affected by the content of melanin, carotenoids, and hemoglobin, Lab values ​​can represent colors that YUV values ​​cannot represent. Characterizing skin color features based on Lab values ​​is beneficial to improving the accuracy of apparent skin color values.

[0120] Where Lab values ​​are two-dimensional vectors, representing the colors within the corresponding Lab color gamut. L represents luminance, indicating the amount of melanin. a represents the red component, indicating the amount of hemoglobin. b represents the yellow component, indicating the amount of carotenoids.

[0121] Regarding the process by which the aforementioned electronic device calculates the initial apparent skin color value based on multiple skin color features, in some embodiments, the electronic device can cluster multiple skin color features into a preset number of skin color clusters, determine a skin color clustering percentage for each skin color cluster, and perform a weighted average calculation on multiple skin color features based on each skin color clustering percentage to obtain the initial apparent skin color value. This improves the accuracy of the initial apparent skin color value calculation.

[0122] Each skin color cluster can include at least one skin color feature, and each skin color cluster percentage can be used to characterize the proportion of the number of skin color features contained in a corresponding skin color cluster to the total number of skin color features.

[0123] Electronic devices can cluster multiple skin color features based on clustering algorithms with controllable cluster centers. The clustering algorithm with controllable cluster centers can be any one of the k-means clustering algorithm or the k-means++ clustering algorithm, etc., without limitation here.

[0124] As an example, the preset number of skin color clusters can be 30. Electronic devices can cluster multiple skin color features into 30 skin color clusters based on the k-means clustering algorithm, without any limitation here.

[0125] Regarding the process by which the aforementioned electronic device calculates a weighted average of multiple skin color features based on the proportion of each skin color cluster to obtain an initial apparent skin color value, in some embodiments, the electronic device can filter a preset number of skin color clusters according to the proportion of each skin color cluster to obtain a target number of skin color clusters, and calculate a weighted average of the target number of skin color features based on the target number of skin color clusters to obtain an initial apparent skin color value. By filtering the preset number of skin color clusters to obtain the target number of skin color clusters, abnormal skin color values ​​are effectively eliminated, and by calculating a weighted average of the target number of skin color clusters based on the target number of skin color clusters, the accuracy of the initial apparent skin color value calculation is further improved.

[0126] The target number of skin color clusters can include a target number of skin color features. Electronic devices can remove skin color clusters with a cluster ratio less than a threshold from the preset number of skin color clusters to obtain the target number of skin color clusters. Filtering the preset number of skin color clusters based on the skin color cluster ratio helps improve the accuracy of the initial apparent skin color value calculation.

[0127] The percentage threshold can be 5%, 10%, 15%, etc., and is not limited here. As an example, with a percentage threshold of 5%, the electronic device can remove skin color clusters with a percentage of less than 5% from a preset number of skin color clusters to obtain the target number of skin color clusters.

[0128] Regarding the process by which the aforementioned electronic device determines the current apparent skin tone value based on a pre-constructed apparent skin tone template card and an initial apparent skin tone value, in some embodiments, the electronic device can calculate a skin tone similarity between the initial apparent skin tone value and each preset apparent skin tone value to obtain multiple skin tone similarities. The skin tone similarity among these multiple similarities that is greater than or equal to a similarity threshold is determined as the target skin tone similarity, and the target preset apparent skin tone value corresponding to the target skin tone similarity is determined as the current apparent skin tone value. Determining the current apparent skin tone value based on a skin tone similarity between the initial apparent skin tone value and each preset skin tone value improves the accuracy of the current apparent skin tone value.

[0129] Among them, a skin color similarity between the initial apparent skin color value and each preset apparent skin color value can be calculated based on a preset algorithm. The preset algorithm can be any one of Euclidean distance similarity algorithm, cosine similarity algorithm, Hamming distance similarity algorithm, Mahalanobis distance similarity algorithm or Minkowski distance algorithm, etc., without limitation here.

[0130] It should be noted that in the embodiments of this application, there is no specific order between steps 110 and 120. The electronic device may determine the current apparent skin color value of the initial face image after determining the current real skin color information of the initial face image, or the electronic device may determine the current real skin color information of the initial face image after determining the current apparent skin color value of the initial face image. No limitation is made here.

[0131] Step 130: Determine the current scene skin color value of the initial face image based on the current true skin color value and the current apparent skin color value.

[0132] In this embodiment, the electronic device can determine the current scene skin color value of the initial face image based on the current real skin color value and the current apparent skin color value, so as to perform skin color correction on the initial face image based on the current scene skin color value.

[0133] Among them, the current scene skin color value can be used to characterize the skin color in the shooting scene recognized by the electronic device.

[0134] Regarding the process by which the aforementioned electronic device determines the current scene skin color value of the initial face image based on the current true skin color value and the current apparent skin color value, in some embodiments, when the current confidence level is greater than or equal to the confidence level threshold, the electronic device can calculate a first skin color difference between the current true skin color value and the current apparent skin color value, and determine the current scene skin color value based on the first skin color difference. The current apparent skin color value is then verified based on the current confidence level and the current true skin color value to obtain the current scene skin color value, thereby improving the accuracy of the current scene skin color value.

[0135] Regarding the process by which the electronic device determines the skin color value of the current scene based on the first skin color difference, in some embodiments, when the first skin color difference is less than the difference threshold, the electronic device can determine the current real skin color value as the skin color value of the current scene; when the first skin color difference is greater than or equal to the difference threshold, the electronic device can determine the preset scene skin color value as the skin color value of the current scene. By determining the skin color value of the current scene based on the skin color difference between the current real skin color value and the current apparent skin color value, the accuracy of the skin color value of the current scene is improved.

[0136] Among them, the preset scene skin color value corresponds to the target clarity of the target face image.

[0137] Regarding the process by which the aforementioned electronic device determines the current scene skin color value of the initial face image based on the current true skin color value and the current apparent skin color value, in some embodiments, when the current confidence level is less than a confidence threshold, the electronic device can determine whether the face corresponding to the initial face image is in a normal lighting environment. When it is determined that the face is in a normal lighting environment and the first skin color difference is less than the difference threshold, the current true skin color value is determined as the current scene skin color value. When the current confidence level of the current true skin color value is low, the face is in a normal lighting environment, and the skin color difference between the current true skin color value and the current apparent skin color value is small, the current true skin color value is determined as the current scene skin color value, further improving the accuracy of the current scene skin color value.

[0138] Among them, the electronic device can obtain the light intensity of the environment in which the face is located corresponding to the initial face image, and determine whether the face is in a normal lighting environment based on the light intensity.

[0139] When the light intensity is greater than or equal to the light intensity threshold, the face is determined to be in a normal lighting environment; when the light intensity is less than the light intensity threshold, the face is determined to be in an abnormal lighting environment.

[0140] Regarding the process by which the aforementioned electronic device determines the current scene skin color value of the initial face image based on the current true skin color value and the current apparent skin color value, in some embodiments, when the electronic device determines that the face is in a normal lighting environment and the first skin color difference is greater than or equal to the difference threshold, the current apparent skin color value can be determined as the current scene skin color value. When the current confidence level of the current true skin color value is low, the face is in a normal lighting environment, and the skin color difference between the current true skin color value and the current apparent skin color value is large, the current apparent skin color value is determined as the current scene skin color value, further improving the accuracy of the current scene skin color value.

[0141] Regarding the process by which the aforementioned electronic device determines the current scene skin color value of the initial face image based on the current true skin color value and the current apparent skin color value, in some embodiments, when the electronic device determines that the face is in an abnormal lighting environment, it determines the preset scene skin color value as the current scene skin color value. When the current confidence level of the current true skin color value is low and the face is in an abnormal lighting environment, it determines the preset scene skin color value corresponding to the target clarity as the current scene skin color value, thereby further improving the accuracy of the current scene skin color value.

[0142] Step 140: Correct the skin tone of the initial face image based on the skin tone value of the current scene to obtain the target face image.

[0143] In this embodiment, the electronic device can correct the skin tone of the initial face image based on the current scene skin tone value to obtain a target face image. The image quality of the target face image is higher than that of the initial face image. This achieves the verification of the current apparent skin tone value based on the current real skin tone value of the initial face image to obtain the current scene skin tone value. This avoids errors in the recognition of the apparent skin tone of the face image caused by excessively bright or dark lighting conditions in the face image shooting scene, thus improving the recognition accuracy of the apparent skin tone of the face image. Furthermore, by correcting the skin tone of the initial face image based on the current scene skin tone value, the image quality of the face image is improved, which is beneficial to improving the shooting quality of the face image captured by the electronic device.

[0144] Image quality may include at least one of sharpness, contrast, or brightness, etc., without limitation here.

[0145] Specifically, the electronic device can determine the skin color compensation value of the initial face image based on the skin color value of the current scene, and perform skin color correction on the initial face image based on the skin color compensation value to obtain the target face image. The skin color correction of the initial face image based on the skin color compensation value corresponding to the skin color value of the current scene improves the image quality of the face image, which is beneficial to improving the shooting quality of the face image captured by the electronic device.

[0146] Among them, the electronic device can calculate the second skin color difference between the preset scene skin color value and the current scene skin color value, and determine the second skin color difference as the skin color compensation value. The skin color compensation value is determined based on the skin color difference between the current scene skin color value and the preset scene skin color value, which improves the accuracy of the skin color compensation value.

[0147] The solution provided in this application determines the current true skin color information of the initial face image, including the current true skin color value, and determines the current apparent skin color value of the initial face image. Based on the current true skin color value and the current apparent skin color value, the current scene skin color value of the initial face image is determined. Skin color correction is then applied to the initial face image based on the current scene skin color value to obtain a target face image. The image quality of the target face image is higher than that of the initial face image. This achieves the verification of the current apparent skin color value based on the current true skin color value of the initial face image to obtain the current scene skin color value. This avoids errors in the recognition of the apparent skin color of the face image caused by excessively bright or dark lighting conditions during the capture of the face image, improving the accuracy of the apparent skin color recognition. Furthermore, the skin color correction of the initial face image based on the current scene skin color value improves the image quality of the face image, which is beneficial for improving the capture quality of face images by electronic devices.

[0148] Please see Figure 5This document illustrates a flowchart of an image processing method provided in another embodiment of this application. In a specific embodiment, the image processing method can be applied to an electronic device. The following example uses an electronic device to illustrate this method. Figure 5 The process shown is described in detail. The image processing method may include the following steps 210 to 260.

[0149] Step 210: Obtain the preview image stream.

[0150] In this embodiment, when a user needs to capture a facial image, they can send a shooting command to the electronic device. The electronic device receives and responds to the shooting command, captures the image through the camera, and obtains a preview image stream.

[0151] In some implementations, after the electronic device acquires the preview image stream, it can perform face detection on the preview image stream to obtain the face detection result, and determine whether there are face image frames in the preview image stream based on the face detection result.

[0152] The face detection results may include a first detection result for characterizing face images detected in the preview face image stream, and a second detection result for characterizing face images not detected in the preview face image stream.

[0153] When the face detection result is the first detection result, it is determined that a face image frame exists in the preview image stream; when the face detection result is the second detection result, it is determined that no face image frame exists in the preview image stream.

[0154] Step 220: When it is determined that a face image frame exists in the preview image stream, an initial face image is generated based on the face image frame.

[0155] In this embodiment, when the electronic device determines that a face image frame exists in the preview image stream, it can crop the face image frame according to the face detection box of the face image frame to obtain a cropped face image, and then scale the cropped face image to obtain an initial face image. Cropping the face image frame according to the face detection box of the face detection to obtain a cropped face image, and obtaining the initial face image based on the cropped face image, helps to improve the accuracy of the initial face image generation.

[0156] The scaling process for cropped face images can be achieved by scaling the current pixels of the cropped face image to a preset pixel size. The preset pixel size can be a user-defined pixel value or a pixel value automatically generated by the electronic device based on multiple face image captures; there is no limitation here. For example, the preset pixel size could be 112x112 or 220x220, etc.

[0157] Regarding the process described above, when the electronic device determines that a face image frame exists in the preview image stream, it can crop the face image frame based on the face detection box of the face image frame to obtain a cropped face image. In some embodiments, when the electronic device determines that a face image frame exists in the preview image stream, it can enlarge the face detection box to obtain a cropping box, and then crop the face image frame based on the cropping box to obtain a cropped face image. Cropping the face image frame based on the cropping box obtained by enlarging the face detection box can avoid the face image in the cropped face image being incomplete due to the face detection box being set too small, thus improving the cropping accuracy of the face image frame.

[0158] The electronic device can increase the width of the face detection frame by a first preset multiple and increase the height of the face detection frame by a second preset multiple. The first preset multiple and the second preset multiple can be the same or different. For example, the first preset multiple is 0.3 times and the second preset multiple is 0.3 times; the first preset multiple is 0.5 times and the second preset multiple is 0.4 times, etc., and there is no limitation here.

[0159] As an example, such as Figure 6 As shown in the image, the white solid line box represents the face detection bounding box. The cropping box obtained by enlarging both the width and height of the face detection bounding box by a factor of 0.3 is shown below. Figure 6 As shown in the white dashed box.

[0160] Regarding the process described above where an electronic device determines that a face image frame exists in the preview image stream, and then crops the face image frame based on the face detection box of the face image frame to obtain a cropped face image, in some embodiments, when the electronic device determines that a face image frame exists in the preview image stream, it can determine the first area of ​​the face detection box of the face image frame, determine the second area of ​​the face image frame, calculate the area ratio of the first area to the second area, and when the area ratio is greater than or equal to the area ratio threshold, crop the face image frame based on the face detection box to obtain a cropped face image.

[0161] Step 230: Determine the current true skin color information of the initial face image.

[0162] Step 240: Determine the current apparent skin color value of the initial face image.

[0163] Step 250: Determine the current scene skin color value of the initial face image based on the current true skin color value and the current apparent skin color value.

[0164] Step 260: Correct the skin tone of the initial face image based on the skin tone value of the current scene to obtain the target face image.

[0165] In this embodiment, steps 230, 240, 250 and 260 can be referred to the corresponding steps in the previous embodiments, and will not be repeated here.

[0166] The solution provided in this embodiment acquires a preview image stream, and when it is determined that a face image frame exists in the preview image stream, generates an initial face image based on the face image frame, determines the current real skin tone information of the initial face image, determines the current apparent skin tone value of the initial face image, and determines the current scene skin tone value of the initial face image based on the current real skin tone value and the current apparent skin tone value, and performs skin tone correction on the initial face image based on the current scene skin tone value to obtain the target face image. This achieves the verification of the current apparent skin tone value based on the current real skin tone value of the initial face image to obtain the current scene skin tone value, avoiding errors in the recognition of the apparent skin tone of the face image when the shooting scene of the face image is too bright or too dark, thus improving the recognition accuracy of the apparent skin tone of the face image. Furthermore, the skin tone correction of the initial face image based on the current scene skin tone value improves the image quality of the face image, which is beneficial to improving the shooting quality of face images captured by electronic devices.

[0167] Furthermore, the accuracy of generating the initial face image is improved by generating the initial face image based on the detected face image in the preview image stream.

[0168] Please see Figure 7 This document illustrates a flowchart of an image processing method provided in another embodiment of this application. In a specific embodiment, the image processing method can be applied to an electronic device. The following example uses an electronic device to illustrate this method. Figure 7 The process shown is described in detail. The image processing method may include the following steps 310 to 370.

[0169] Step 310: Obtain the initial historical skin tone image.

[0170] In this embodiment, the electronic device can acquire an initial historical skin color image to facilitate training of the classification model based on the initial historical skin color image.

[0171] Regarding the process of the electronic device acquiring the initial historical skin color image, in some embodiments, the electronic device pre-stores the initial historical skin color image, and the electronic device can read the pre-stored initial historical skin color image.

[0172] Regarding the process of the aforementioned electronic device acquiring the initial historical skin tone image, in some embodiments, the electronic device can generate upload prompt information and receive the initial historical skin tone image uploaded by the user according to the upload prompt information.

[0173] The upload prompt message is used to prompt the user to upload the initial historical skin tone image to the electronic device. The upload prompt message can be at least one of the following: sound prompt message, text prompt message, or light prompt message, etc., without limitation.

[0174] Regarding the process by which the aforementioned electronic device acquires initial historical skin tone images, in some implementations, the server pre-stores initial historical skin tone images, the server is connected to the electronic device via a network, and communicates with the electronic device via the network.

[0175] Electronic devices can send a request to a server via a network. The server receives and responds to the request, sending a pre-stored initial historical skin tone image to the electronic device via the network. The electronic device then receives the initial historical skin tone image returned by the server.

[0176] The server can be a standalone physical server, a server cluster or distributed system consisting of multiple physical servers, or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, content delivery networks (CDN), big data and artificial intelligence platforms, etc. There are no restrictions here.

[0177] Step 320: Filter the initial historical skin color image to obtain the target historical skin color image.

[0178] In this embodiment, the electronic device can filter the initial historical skin color image to obtain the target historical skin color image.

[0179] Regarding the process of the aforementioned electronic device filtering initial historical skin color images to obtain target historical skin color images, in some embodiments, the electronic device can classify the initial historical skin color images into categories to obtain preset types of skin color images, and construct a real skin color template color card based on each type of real skin color to obtain preset types of real skin color template color cards. Then, it can filter the corresponding skin color image based on each real skin color template color card to obtain the target historical skin color image. Filtering a skin color image based on a real skin color template color card constructed for each type of real skin color improves the sample data quality of the target historical skin color image and helps improve the detection accuracy of the real skin color classification model.

[0180] Among them, preset skin color images correspond to multiple real skin colors, with each skin color image corresponding to a real skin color.

[0181] As an example, real skin tones can include black skin tones, white skin tones, and yellow skin tones, and real skin tone template color charts can include black skin tone template color charts, black skin tone template color charts, and yellow skin tone template color charts, etc.

[0182] Electronic devices can create a black skin tone template color chart based on black skin tones, such as... Figure 8 As shown in (a), a white skin tone template color chart is constructed based on white skin tone. The white skin tone template color chart is as follows: Figure 8 As shown in (b) above, a yellow skin tone template color chart is constructed based on yellow skin tone. The yellow skin tone template color chart is as follows: Figure 8 As shown in (c) in the figure.

[0183] Regarding the process by which the aforementioned electronic device filters images of a corresponding skin tone type based on each real skin tone template color chart to obtain a target historical skin tone image, in some embodiments, the electronic device can remove skin tone type images that do not match the corresponding real skin tone template color chart from each skin tone type image to obtain the target historical skin tone image. By removing skin tone type images based on the matching degree between each skin tone type image and the corresponding real skin tone template color chart, the accuracy of filtering skin tone type images is improved.

[0184] Step 330: Input the target historical skin color image into the classification model for training to obtain the true skin color classification model.

[0185] In this embodiment, the electronic device can input a target historical skin color image into the classification model. The classification model receives and responds to the target historical skin color image, and trains based on the target historical skin color image to obtain a true skin color classification model. Based on the target historical skin color image obtained by filtering the initial historical skin color image, the classification model is trained to obtain a true skin color classification model, which reduces the number of training samples for the classification model and helps to improve the training efficiency of the classification model.

[0186] Step 340: Input the initial face image into the real skin color classification model to obtain the current real skin color information.

[0187] Step 350: Determine the current apparent skin color value of the initial face image.

[0188] Step 360: Determine the current scene skin color value of the initial face image based on the current real skin color value and the current apparent skin color value.

[0189] Step 370: Correct the skin tone of the initial face image based on the skin tone value of the current scene to obtain the target face image.

[0190] In this embodiment, steps 340, 350, 360 and 370 can be referred to the corresponding steps in the foregoing embodiments, and will not be repeated here.

[0191] This embodiment provides a solution that obtains an initial historical skin tone image, filters and processes it to obtain a target historical skin tone image, inputs the target historical skin tone image into a classification model for training to obtain a true skin tone classification model, inputs the initial face image into the true skin tone classification model to obtain the current true skin tone information, determines the current apparent skin tone value of the initial face image, and determines the current scene skin tone value of the initial face image based on the current true skin tone value and the current apparent skin tone value, and performs skin tone correction on the initial face image based on the current scene skin tone value to obtain the target face image. This achieves the verification of the current apparent skin tone value based on the current true skin tone value of the initial face image to obtain the current scene skin tone value, avoiding errors in the recognition of the apparent skin tone of the face image when the shooting scene of the face image is too bright or too dark, thus improving the recognition accuracy of the apparent skin tone of the face image. Furthermore, the skin tone correction of the initial face image based on the current scene skin tone value improves the image quality of the face image, which is beneficial to improving the shooting quality of face images captured by electronic devices.

[0192] Furthermore, based on the target historical skin color image obtained by filtering the initial historical skin color image, the classification model is trained to obtain the real skin color classification model, which reduces the number of training samples for the classification model and helps to improve the training efficiency of the classification model.

[0193] Please see Figure 9 This illustration shows an image processing apparatus 500 provided in one embodiment of this application. The image processing apparatus 500 can be applied to electronic devices. The following uses an electronic device as an example to illustrate... Figure 9 The image processing apparatus 500 shown will be described in detail. The image processing apparatus 500 may include a first determining module 510, a second determining module 520, a third determining module 530, and a correction module 540.

[0194] The first determining module 510 can be used to determine the current true skin color information of the initial face image, which may include the current true skin color value; the second determining module 520 can be used to determine the current apparent skin color value of the initial face image; the third determining module 530 can be used to determine the current scene skin color value of the initial face image based on the current true skin color value and the current apparent skin color value; the correction module 540 can be used to perform skin color correction on the initial face image based on the current scene skin color value to obtain a target face image, the image quality of the target face image being higher than the image quality of the initial face image.

[0195] In some embodiments, the image processing apparatus 500 may further include a first acquisition module and a generation module.

[0196] The first acquisition module can be used to acquire a preview image stream before the first determination module 510 determines the current real skin color information of the initial face image; the generation module can be used to generate an initial face image based on the face image frame when it is determined that a face image frame exists in the preview image stream.

[0197] In some implementations, the generation module may include a cropping submodule and a scaling submodule.

[0198] The cropping submodule can be used to crop the face image frame based on the face detection box of the face image frame when it is determined that there is a face image frame in the preview image stream, so as to obtain a cropped face image; the scaling submodule can be used to scale the cropped face image to obtain the initial face image.

[0199] In some implementations, the trimming submodule may include an enlargement submodule and a trimming submodule.

[0200] The magnification submodule can be used to magnify the face detection box when a face image frame is determined to exist in the preview image stream, thus obtaining a cropping box; the cropping submodule can be used to crop the face image frame according to the cropping box, thus obtaining a cropped face image.

[0201] In some implementations, the first determining module 510 may include an input submodule.

[0202] The input submodule can be used to input an initial face image into the real skin color classification model to obtain the current real skin color information. The real skin color classification model can be trained based on the target historical skin color image. The target historical skin color image can be labeled with historical real skin color information, which can include historical real skin color values.

[0203] In some embodiments, the image processing apparatus 500 may further include a second acquisition module, a filtering module, and an input module.

[0204] The second acquisition module can be used to input the initial face image into the real skin color classification model before obtaining the current real skin color information, and to acquire the initial historical skin color image; the filtering module can be used to filter the initial historical skin color image to obtain the target historical skin color image; the input module can be used to input the target historical skin color image into the classification model for training to obtain the real skin color classification model.

[0205] In some implementations, the filtering module may include a partitioning submodule, a construction submodule, and a filtering submodule.

[0206] The segmentation submodule can be used to classify the initial historical skin color images into preset skin color image types, and each skin color image type can correspond to a real skin color; the construction submodule can be used to construct a real skin color template color card based on each real skin color, and obtain preset real skin color template color cards; the filtering submodule can be used to filter the corresponding skin color image type based on each real skin color template color card, and obtain the target historical skin color image.

[0207] In some implementations, the filtering submodule may include a rejection submodule.

[0208] The removal submodule can be used to remove skin tone images that do not match the corresponding real skin tone template color card from each skin tone image to obtain the target historical skin tone image.

[0209] In some implementations, the second determining module 520 may include a first determining submodule and a second determining submodule.

[0210] The first determining submodule can be used to determine the initial apparent skin color value of the initial face image; the second determining submodule can be used to determine the current apparent skin color value based on the pre-constructed apparent skin color template color card and the initial apparent skin color value.

[0211] In some implementations, the first determining submodule may include a first determining secondary submodule and a first calculating secondary submodule.

[0212] The first determining submodule can be used to determine multiple skin color features of the initial face image; the first calculating submodule can be used to calculate the initial apparent skin color value based on the multiple skin color features.

[0213] In some implementations, the first determining submodule may include a parsing unit and a determining unit.

[0214] The parsing unit can be used to perform face parsing on the initial face image to obtain the initial face skin region image; the determination unit can be used to determine a skin color feature for each pixel in the initial face skin region image to obtain multiple skin color features.

[0215] In some implementations, the parsing unit may include an input subunit.

[0216] The input sub-unit can be used to input the initial face image into the face parsing model to obtain the initial face skin region image. The face parsing model can be obtained by training a deep learning neural network model based on historical face images, which can be labeled with historical face skin regions.

[0217] In some implementations, the determining unit may include a conversion subunit and a first determining subunit.

[0218] The transformation subunit can be used to convert the YUV value of each pixel into a Lab value; the first determination subunit can be used to determine each Lab value as a skin color feature of the corresponding pixel.

[0219] In some implementations, the first computational submodule may include a clustering unit, a second determination unit, and a computation unit.

[0220] The clustering unit can be used to cluster multiple skin color features into a preset number of skin color clusters, and each skin color cluster can include at least one skin color feature; the second determining unit can be used to determine a skin color cluster ratio for each skin color cluster, and each skin color cluster ratio can be used to characterize the proportion of the number of skin color features contained in the corresponding skin color cluster to the total number of multiple skin color features; the calculation unit can be used to perform a weighted average calculation on multiple skin color features based on each skin color cluster ratio to obtain an initial apparent skin color value.

[0221] In some implementations, the computing unit may include a filtering subunit and a computing subunit.

[0222] The filtering subunit can be used to filter a preset number of skin color clusters according to the proportion of each skin color cluster to obtain a target number of skin color clusters, which may include a target number of skin color features; the calculation subunit can be used to perform a weighted average calculation of the target number of skin color features based on the target number of skin color clusters and the proportion of the target skin color clusters to obtain the initial apparent skin color value.

[0223] In some implementations, the filtering subunit may include a rejection subunit.

[0224] The removal of sub-units can be used to remove skin color clusters whose skin color cluster ratio is less than the ratio threshold from a preset number of skin color clusters, thus obtaining the target number of skin color clusters.

[0225] In some implementations, the apparent skin tone template color chart may include multiple preset apparent skin tone values, and the second determination submodule may include a second calculation submodule, a second determination submodule, and a third determination submodule.

[0226] The second calculation submodule can be used to calculate a skin color similarity between the initial apparent skin color value and each preset apparent skin color value to obtain multiple skin color similarities; the second determination submodule can be used to determine the skin color similarity among the multiple skin color similarities that is greater than or equal to the similarity threshold as the target skin color similarity; the third determination submodule can be used to determine the target preset apparent skin color value corresponding to the target skin color similarity as the current apparent skin color value.

[0227] In some implementations, the current true skin color information may also include the current confidence level, and the third determination module 530 may include the first calculation submodule and the third determination submodule.

[0228] The first calculation submodule can be used to calculate the first skin color difference between the current true skin color value and the current apparent skin color value when the current confidence level is greater than or equal to the confidence level threshold; the third determination submodule can be used to determine the current true skin color value as the current scene skin color value when the first skin color difference is less than the difference threshold.

[0229] In some embodiments, the image processing apparatus 500 may further include a fourth determining module, a fifth determining module, and a sixth determining module.

[0230] The fourth determination module can be used to determine whether the face corresponding to the initial face image is in a normal lighting environment when the current confidence level is less than the confidence level threshold; the fifth determination module can be used to determine the current real skin color value as the current scene skin color value when it is determined that the face is in a normal lighting environment and the first skin color difference is less than the difference threshold; the sixth determination module can be used to determine the current apparent skin color value as the current scene skin color value when it is determined that the face is in a normal lighting environment and the first skin color difference is greater than or equal to the difference threshold.

[0231] In some implementations, the correction module 540 may include a fourth determining submodule and a correction submodule.

[0232] The fourth determination submodule can be used to determine the skin color compensation value of the initial face image based on the skin color value of the current scene; the correction submodule can be used to correct the skin color of the initial face image based on the skin color compensation value to obtain the target face image.

[0233] In some implementations, the fourth determination submodule may include the third calculation submodule and the fourth determination submodule.

[0234] The third calculation submodule can be used to calculate the second skin color difference between the preset scene skin color value and the current scene skin color value. The preset scene skin color value can correspond to the target clarity of the target face image. The fourth determination submodule can be used to determine the second skin color difference as the skin color compensation value.

[0235] The solution provided in this embodiment determines the current true skin color information of the initial face image, including the current true skin color value, and determines the current apparent skin color value of the initial face image. Based on the current true skin color value and the current apparent skin color value, the current scene skin color value of the initial face image is determined. Skin color correction is then applied to the initial face image based on the current scene skin color value to obtain a target face image. The image quality of the target face image is higher than that of the initial face image. This achieves the verification of the current apparent skin color value based on the current true skin color value of the initial face image to obtain the current scene skin color value. This avoids errors in the recognition of the apparent skin color of the face image caused by excessively bright or dark lighting conditions during the capture of the face image, improving the accuracy of the apparent skin color recognition. Furthermore, the skin color correction of the initial face image based on the current scene skin color value improves the image quality of the face image, which is beneficial for improving the capture quality of face images by electronic devices.

[0236] It should be noted that the various embodiments in this specification are described in a progressive manner, with each embodiment focusing on its differences from other embodiments. Similar or identical parts between embodiments can be referred to interchangeably. For device embodiments, since they are basically similar to method embodiments, the description is relatively simple; relevant parts can be referred to in the description of the method embodiments. Any processing method described in the method embodiments can be implemented in the device embodiments through corresponding processing modules, and will not be elaborated upon further in the device embodiments.

[0237] Furthermore, the functional modules in the various embodiments of this application can be integrated into one processing module, or each module can exist physically separately, or two or more modules can be integrated into one module. The integrated modules described above can be implemented in hardware or as software functional modules.

[0238] Please see Figure 10 This illustrates a schematic diagram of the hardware structure of an electronic device 600 provided in one embodiment of this application. Figure 10As shown, the electronic device 600 may include a processor 610, an external memory interface 620, an internal memory 621, a Universal Serial Bus (USB) interface 630, a charging management module 640, a power management module 641, a battery 642, an antenna 1, an antenna 2, a mobile communication module 650, a wireless communication module 660, an audio module 670, a speaker 670A, a receiver 670B, a microphone 670C, a headphone jack 670D, a sensor module 680, buttons 690, a motor 691, an indicator 692, a camera 693, a display screen 694, and a Subscriber Identification Module (SIM) card interface 695, etc. The sensor module 680 may include a pressure sensor 680A, a gyroscope sensor 680B, a barometric pressure sensor 680C, a magnetic sensor 680D, an accelerometer sensor 680E, a distance sensor 680F, a proximity light sensor 680G, a fingerprint sensor 680H, a temperature sensor 680J, a touch sensor 680K, an ambient light sensor 680L, a bone conduction sensor 680M, etc.

[0239] It is understood that the structures illustrated in the embodiments of this application do not constitute a specific limitation on the electronic device 600. In other embodiments of this application, the electronic device 600 may include more or fewer components than illustrated, or combine some components, or split some components, or have different component arrangements. The illustrated components may be implemented in hardware, software, or a combination of software and hardware.

[0240] For example, Figure 10 The processor 610 shown may include one or more processing units, such as an application processor (AP), a modem processor, a graphics processing unit (GPU), an image signal processor (ISP), a controller, memory, a video codec, a digital signal processor (DSP), a baseband processor, and / or a neural network processing unit (NPU). These different processing units may be independent devices or integrated into one or more processors.

[0241] The AP can be used to control and manage camera applications; for example, the AP can control the camera application to capture facial images.

[0242] The controller can be the nerve center and command center of the electronic device 600. The controller can generate operation control signals according to the instruction opcode and timing signals to complete the control of instruction fetching and execution.

[0243] The processor 610 may also include a memory for storing instructions and data. In some embodiments, the memory in the processor 610 is a cache memory. This memory can store instructions or data that the processor 610 has just used or that are used repeatedly. If the processor 610 needs to use the instruction or data again, it can retrieve it directly from the memory. This avoids repeated accesses, reduces the waiting time of the processor 610, and thus improves the efficiency of the system.

[0244] In some embodiments, the processor 610 may include one or more interfaces. Interfaces may include an Inter-Integrated Circuit (I2C) interface, an Inter-Integrated Circuit Sound (I2S) interface, a Pulse Code Modulation (PCM) interface, a Universal Asynchronous Receiver / Transmitter (UART) interface, a Mobile Industry Processor Interface (MIPI) interface, a General Purpose Input / Output (GPIO) interface, a Subscriber Identity Module (SIM) interface, and / or a Universal Serial Bus (USB) interface, etc.

[0245] In some embodiments, the I2C interface is a bidirectional synchronous serial bus, including a serial data line (SDA) and a serial clock line (SCL). The processor 610 may include multiple I2C buses. The processor 610 can couple to the touch sensor 680K, charger, flash, camera 693, etc., through different I2C bus interfaces. For example, the processor 610 can couple to the touch sensor 680K through the I2C interface, enabling communication between the processor 610 and the touch sensor 680K via the I2C bus interface, thus realizing the touch function of the electronic device 600.

[0246] In some embodiments, the MIPI interface can be used to connect the processor 610 to peripheral devices such as the display screen 694 and the camera 693. The MIPI interface includes a Camera Serial Interface (CSI) and a Display Serial Interface (DSI). The processor 610 and the camera 693 communicate via the CSI interface to enable the electronic device 600 to capture images. The processor 610 and the display screen 694 communicate via the DSI interface to enable the electronic device 600 to display images.

[0247] In some embodiments, the GPIO interface can be configured via software. The GPIO interface can be configured as a control signal or a data signal. The GPIO interface can be used to connect the processor 610 to a camera 693, a display screen 694, a wireless communication module 660, an audio module 670, a sensor module 680, etc. The GPIO interface can also be configured as an I2C interface, an I2S interface, a UART interface, a MIPI interface, etc.

[0248] Electronic device 600 implements display functions through a GPU, a display screen 694, and an application processor. The GPU is a microprocessor for image processing, connected to the display screen 694 and the application processor. The GPU performs mathematical and geometric calculations and is used for graphics rendering. Processor 610 may include one or more GPUs, which execute program instructions to generate or modify display information.

[0249] Display screen 694 is used to display images, videos, etc. Display screen 694 includes a display panel. The display panel can be a liquid crystal display (LCD), an organic light-emitting diode (OLED), an active-matrix organic light-emitting diode (AMOLED), a flexible light-emitting diode (FLED), a Mini-LED, a Micro-LED, a Micro-OLED, a quantum dot light-emitting diode (QLED), etc. In some embodiments, electronic device 600 may include one or N displays 694, where N is a positive integer greater than 1.

[0250] Electronic device 600 can achieve shooting function through ISP, camera 693, video codec, GPU, display 694 and application processor.

[0251] The ISP (Image Signal Processor) is used to process data fed back from the camera 693. For example, when taking a picture, the shutter is opened, and light is transmitted through the lens to the camera's photosensitive element. The light signal is converted into an electrical signal, and the camera's photosensitive element transmits the electrical signal to the ISP for processing, transforming it into an image visible to the naked eye. The ISP can also perform algorithmic optimization of image noise, brightness, and skin tone. The ISP can also optimize parameters such as exposure and color temperature of the shooting scene. In some embodiments, the ISP can be set in the camera 693.

[0252] Camera 693 is used to capture still images or videos. An object is projected onto a photosensitive element by generating an optical image through the lens. The photosensitive element can be a charge-coupled device (CCD) or a complementary metal-oxide-semiconductor (CMOS) phototransistor. The photosensitive element converts the light signal into an electrical signal, which is then passed to an ISP for conversion into a digital image signal. The ISP outputs the digital image signal to a DSP for processing. The DSP converts the digital image signal into image signals in standard RGB, YUV, or other formats. In some embodiments, electronic device 600 may include one or N cameras 693, where N is a positive integer greater than 1.

[0253] A digital signal processor (DSP) is used to process digital signals. Besides digital image signals, it can also process other digital signals. For example, when electronic device 600 is selecting a frequency, the DSP is used to perform Fourier transforms on the frequency energy.

[0254] Video codecs are used to compress or decompress digital video. Electronic device 600 may support one or more video codecs. Thus, electronic device 600 can play or record video in various encoding formats, such as Moving Picture Experts Group (MPEG) 1, MPEG2, MPEG3, MPEG4, etc.

[0255] NPU stands for Neural Network (NN) computing processor. By borrowing the structure of biological neural networks, such as the transmission patterns between neurons in the human brain, it can rapidly process input information and continuously learn on its own. NPUs can enable intelligent cognitive applications in electronic devices, such as image recognition, facial recognition, speech recognition, and text understanding.

[0256] The external memory interface 620 can be used to connect an external memory card, such as a Secure Digital (SD) card, to expand the storage capacity of the electronic device 600. The external memory card communicates with the processor 610 through the external memory interface 620 to perform data storage functions. For example, it can save captured images, videos, and other files to the external memory card.

[0257] Internal memory 621 can be used to store executable program code, including instructions. Processor 610 executes various functional applications and data processing of electronic device 600 by running the instructions stored in internal memory 621. Internal memory 621 may include a program storage area and a data storage area. The program storage area may store the operating system, at least one application program required for a function (such as image capture, video recording, etc.), etc. The data storage area may store data created during the use of electronic device 600 (such as image data, video data, etc.). Furthermore, internal memory 621 may include high-speed random access memory and may also include non-volatile memory, such as at least one disk storage device, flash memory device, Universal Flash Storage (UFS), etc.

[0258] It is understood that the structures illustrated in the embodiments of this application do not constitute a specific limitation on the electronic device 600. In other embodiments of this application, the electronic device 600 may include more or fewer components than illustrated, or combine some components, or split some components, or have different component arrangements. The illustrated components may be implemented in hardware, software, or a combination of software and hardware.

[0259] Please see Figure 11 This illustrates a functional block diagram of an electronic device 700 according to an embodiment of this application. Figure 11 As shown, the electronic device 700 includes: one or more processors 710 ( Figure 11 Only one processor is shown in the diagram) and a memory 720, which is coupled to one or more processors 710. The memory 720 is used to store computer program code 730, which includes computer instructions. One or more processors 710 call the computer instructions to cause the electronic device 700 to perform the steps in any of the above methods.

[0260] Those skilled in the art will understand that Figure 11This is merely an example of electronic device 700 and does not constitute a limitation on electronic device 700. In practice, electronic device 700 may include more or fewer components than shown, or combine certain components, or different components. For example, it may also include input / output devices, network access devices, etc. Electronic device 700 may also be the same device as electronic device 600 described in the above embodiments.

[0261] The processor 710 can be a Central Processing Unit (CPU), other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor can be a microprocessor or any conventional processor.

[0262] In some embodiments, memory 720 may be an internal storage unit of electronic device 700, such as a hard disk or memory of electronic device 700. In other embodiments, memory 720 may be an external storage device of electronic device 700, such as a plug-in hard disk, smart media card (SMC), secure digital (SD) card, flash card, etc., provided on electronic device 700. Optionally, memory 720 may include both internal and external storage units of electronic device 700. Memory 720 is used to store operating system, application programs, bootloaders, data, and other programs, such as program code of computer programs. Memory 720 may also be used to temporarily store data that has been output or will be output.

[0263] It should be noted that the information interaction and execution process between the above-mentioned devices / units are based on the same concept as the method embodiments of this application. For details on their specific functions and technical effects, please refer to the method embodiments section, and they will not be repeated here.

[0264] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the above-described division of functional units and modules is merely an example. In practical applications, the above functions can be assigned to different functional units and modules as needed, that is, the internal structure of the above device can be divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the embodiments can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit. Furthermore, the specific names of the functional units and modules are only for easy differentiation and are not intended to limit the scope of protection of this application. The specific working process of the units and modules in the above system can be referred to the corresponding process in the foregoing method embodiments, and will not be repeated here.

[0265] This application also provides a chip system applied to an electronic device. The chip system includes one or more processors, which are used to invoke computer instructions to cause the electronic device to implement the steps in any of the above methods.

[0266] In some implementations, the chip system also includes a memory connected to one or more processors via circuitry or wiring.

[0267] In some implementations, the chip system also includes a communication interface.

[0268] This application also provides a computer-readable medium including instructions that, when executed on an electronic device, cause the electronic device to perform the methods described in the above-described method embodiments.

[0269] This application also provides a computer program product that, when run on an electronic device, causes the electronic device to perform the aforementioned related steps to implement the methods described in the various method embodiments above.

[0270] If the integrated units described above are implemented as software functional units and sold or used as independent products, they can be stored in a computer-readable medium. Based on this understanding, all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a computer-readable medium, and when executed by a processor, it can implement the steps of the various method embodiments described above. The computer program includes computer program code, which can be in the form of source code, object code, executable files, or certain intermediate forms. A computer-readable medium can include at least: any entity or device capable of carrying computer program code to a photographic device / electronic device, a recording medium, computer memory, read-only memory (ROM), random access memory (RAM), electrical carrier signals, telecommunication signals, and software distribution media. Examples include USB flash drives, portable hard drives, magnetic disks, or optical disks. In some jurisdictions, according to legislation and patent practice, computer-readable media cannot be electrical carrier signals or telecommunication signals.

[0271] In the above embodiments, the descriptions of each embodiment have different focuses. For parts that are not described in detail or recorded in a certain embodiment, please refer to the relevant descriptions of other embodiments.

[0272] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0273] In the embodiments provided in this application, it should be understood that the disclosed apparatus / device and method can be implemented in other ways. For example, the apparatus / device embodiments described above are merely illustrative. For instance, the division of modules or units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between apparatuses or units may be electrical, mechanical, or other forms.

[0274] The units described above as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0275] It should be understood that, when used in this application specification and the appended claims, the term "comprising" indicates the presence of the described features, integrals, steps, operations, elements and / or components, but does not exclude the presence or addition of one or more other features, integrals, steps, operations, elements, components and / or a collection thereof.

[0276] It should also be understood that the term “and / or” as used in this application specification and the appended claims means any combination of one or more of the associated listed items and all possible combinations, and includes such combinations.

[0277] Furthermore, in the description of this application and the appended claims, the terms "first," "second," "third," etc., are used only to distinguish descriptions and should not be construed as indicating or implying relative importance.

[0278] References to "one embodiment" or "some embodiments" in this specification mean that one or more embodiments of this application include a specific feature, structure, or characteristic described in connection with that embodiment. Therefore, the phrases "in one embodiment," "in some embodiments," "in other embodiments," "in still other embodiments," etc., appearing in different parts of this specification do not necessarily refer to the same embodiment, but rather mean "one or more, but not all, embodiments," unless otherwise specifically emphasized. The terms "comprising," "including," "having," and variations thereof mean "including but not limited to," unless otherwise specifically emphasized.

[0279] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application.

Claims

1. An image processing method, characterized in that, The image processing method includes: Determine the current true skin color information of the initial face image, wherein the current true skin color information includes the current true skin color value; Determine the current apparent skin color value of the initial face image; Based on the current true skin color value and the current apparent skin color value, determine the current scene skin color value of the initial face image; The initial face image is corrected based on the skin color value of the current scene to obtain a target face image, the image quality of which is higher than that of the initial face image.

2. The image processing method according to claim 1, characterized in that, Before determining the current true skin color information of the initial face image, the image processing method further includes: Get the preview image stream; When it is determined that a face image frame exists in the preview image stream, the initial face image is generated based on the face image frame.

3. The image processing method according to claim 2, characterized in that, When it is determined that a face image frame exists in the preview image stream, generating the initial face image based on the face image frame includes: When it is determined that the face image frame exists in the preview image stream, the face image frame is cropped according to the face detection box of the face image frame to obtain a cropped face image; The cropped face image is scaled to obtain the initial face image.

4. The image processing method according to claim 3, characterized in that, When it is determined that the face image frame exists in the preview image stream, the face image frame is cropped according to the face detection box of the face image frame to obtain a cropped face image, including: When it is determined that the face image frame exists in the preview image stream, the face detection box is enlarged to obtain a cropping box; The face image frame is cropped according to the cropping box to obtain the cropped face image.

5. The image processing method according to any one of claims 1 to 4, characterized in that, Determining the current true skin color information of the initial face image includes: The initial face image is input into the real skin color classification model to obtain the current real skin color information. The real skin color classification model is trained based on the target historical skin color image. The target historical skin color image is labeled with historical real skin color information, which includes historical real skin color values.

6. The image processing method according to claim 5, characterized in that, Before inputting the initial face image into the real skin color classification model to obtain the current real skin color information, the image processing method further includes: Obtain the initial historical skin tone image; The initial historical skin color image is filtered to obtain the target historical skin color image; The target historical skin color image is input into the classification model for training to obtain the real skin color classification model.

7. The image processing method according to claim 6, characterized in that, The step of filtering the initial historical skin color image to obtain the target historical skin color image includes: The initial historical skin color images are classified into categories to obtain preset skin color category images, and each skin color category image corresponds to a real skin color; A real skin tone template color card is constructed based on each real skin tone, resulting in a preset real skin tone template color card; The target historical skin color image is obtained by filtering the corresponding skin color type image based on each real skin color template color card.

8. The image processing method according to claim 7, characterized in that, The step of filtering images of a corresponding skin tone type based on each real skin tone template color chart to obtain the target historical skin tone image includes: Images of skin tones that do not match the corresponding real skin tone template color chart are removed from each skin tone image to obtain the target historical skin tone image.

9. The image processing method according to any one of claims 1 to 8, characterized in that, Determining the current apparent skin color value of the initial face image includes: Determine the initial apparent skin color value of the initial face image; The current apparent skin tone value is determined based on the pre-constructed apparent skin tone template color chart and the initial apparent skin tone value.

10. The image processing method according to claim 9, characterized in that, Determining the initial apparent skin color value of the initial face image includes: Determine multiple skin color features of the initial face image; The initial apparent skin color value is calculated based on the multiple skin color features.

11. The image processing method according to claim 10, characterized in that, Determining multiple skin tone features of the initial face image includes: The initial face image is analyzed to obtain an initial face skin region image; Determine a skin color feature for each pixel in the initial facial skin region image to obtain the plurality of skin color features.

12. The image processing method according to claim 11, characterized in that, The step of performing face analysis on the initial face image to obtain an initial face skin region image includes: The initial face image is input into the face analysis model to obtain the initial face skin region image. The face analysis model is obtained by training a deep learning neural network model based on historical face images, and the historical face images are labeled with historical face skin regions.

13. The image processing method according to claim 11 or 12, characterized in that, Determining a skin color feature for each pixel in the initial facial skin region image includes: Convert the YUV value of each pixel into a Lab value; Each Lab value is assigned as a skin color feature for a corresponding pixel.

14. The image processing method according to any one of claims 10 to 13, characterized in that, The step of calculating the initial apparent skin color value based on the multiple skin color features includes: The multiple skin color features are clustered into a preset number of skin color clusters, and each skin color cluster includes at least one skin color feature; Determine a skin color cluster percentage for each skin color cluster. Each skin color cluster percentage is used to characterize the proportion of skin color features contained in a corresponding skin color cluster relative to the total number of skin color features. The initial apparent skin color value is obtained by calculating a weighted average of the multiple skin color features based on the proportion of each skin color cluster.

15. The image processing method according to claim 14, characterized in that, The step of calculating the initial apparent skin color value by weighting the multiple skin color features based on the proportion of each skin color cluster includes: The preset number of skin color clusters are filtered according to the proportion of each skin color cluster to obtain a target number of skin color clusters, wherein the target number of skin color clusters includes a target number of skin color features; Based on the target number of skin color clusters and their target skin color cluster proportions, a weighted average is calculated on the target number of skin color features to obtain the initial apparent skin color value.

16. The image processing method according to claim 15, characterized in that, The step of filtering the preset number of skin color clusters according to the proportion of each skin color cluster to obtain the target number of skin color clusters includes: Skin color clusters with a proportion less than a threshold are removed from the preset number of skin color clusters to obtain the target number of skin color clusters.

17. The image processing method according to any one of claims 9 to 16, characterized in that, The apparent skin tone template color chart includes multiple preset apparent skin tone values. Determining the current apparent skin tone value based on the pre-constructed apparent skin tone template color chart and the initial apparent skin tone values ​​includes: Calculate a skin color similarity between the initial apparent skin color value and each preset apparent skin color value to obtain multiple skin color similarities; The skin color similarity values ​​that are greater than or equal to the similarity threshold among the plurality of skin color similarities are determined as the target skin color similarity; The target preset apparent skin color value corresponding to the target skin color similarity is determined as the current apparent skin color value.

18. The image processing method according to any one of claims 1 to 17, characterized in that, The current true skin color information also includes the current confidence level. Determining the current scene skin color value of the initial face image based on the current true skin color value and the current apparent skin color value includes: When the current confidence level is greater than or equal to the confidence threshold, calculate the first skin color difference between the current true skin color value and the current apparent skin color value; When the first skin color difference is less than the difference threshold, the current real skin color value is determined as the current scene skin color value.

19. The image processing method according to claim 18, characterized in that, The image processing method further includes: When the current confidence level is less than the confidence threshold, it is determined whether the face corresponding to the initial face image is in a normal lighting environment; When it is determined that the face is in the normal lighting environment and the first skin color difference is less than the difference threshold, the current real skin color value is determined as the current scene skin color value; When it is determined that the face is in the normal lighting environment and the first skin color difference is greater than or equal to the difference threshold, the current apparent skin color value is determined as the current scene skin color value.

20. The image processing method according to any one of claims 1 to 19, characterized in that, The step of correcting the skin tone of the initial face image based on the current scene skin tone value to obtain the target face image includes: The skin color compensation value of the initial face image is determined based on the skin color value of the current scene; The initial face image is corrected based on the skin color compensation value to obtain the target face image.

21. The image processing method according to claim 20, characterized in that, Determining the skin tone compensation value of the initial face image based on the current scene skin tone value includes: Calculate a second skin color difference between a preset scene skin color value and the current scene skin color value, wherein the preset scene skin color value corresponds to the target clarity of the target face image; The second skin color difference is determined as the skin color compensation value.

22. An electronic device, characterized in that, The electronic device includes: one or more processors, and a memory; The memory is coupled to one or more processors, and the memory is used to store computer program code, the computer program code including computer instructions, wherein the one or more processors invoke the computer instructions to cause the electronic device to perform the image processing method as described in any one of claims 1 to 21.

23. A computer-readable storage medium, characterized in that, The computer-readable storage medium includes instructions that, when executed on an electronic device, cause the electronic device to perform the image processing method as described in any one of claims 1 to 21.

Images