Method and device for determining skin type, electronic equipment and storage medium

Abstract

The application provides a personalized skin type determination method and device, electronic equipment and storage medium. The personalized skin type determination method comprises the following steps: acquiring a face global image corresponding to different shooting light source types; extracting a face local image from the face global image, wherein the face local image corresponds to face position area information and light source type information; inputting the face local image into a corresponding skin detection model according to the face position area information and the light source type information to obtain a skin type detection result; wherein the skin detection model is trained according to a face local image sample and a corresponding skin type label. Through the method, the skin type of the face skin can be determined, the time consumption for determining the skin type is shortened, and the work efficiency for determining the skin type is improved.

Description

Technical Field

[0001] This invention relates to the field of skin detection technology, and more specifically, to a method, apparatus, electronic device, and storage medium for determining personalized skin types. Background Technology

[0002] The skin is the outermost barrier of the human body, responsible for protecting the body and defending against external aggressors; its condition is also a direct reflection of beauty. Cosmetics act directly on the skin's surface, and different skin types require different cosmetics. Improper use can lead to counterproductive results. Therefore, it is essential for consumers to have a scientific understanding of their own skin type.

[0003] The Bergman classification of sixteen skin types was proposed by dermatologist Dr. Robert Bergman. This theory uses a questionnaire survey to analyze the skin type of respondents. However, the questionnaire includes a large number of subjective and complex questions, resulting in a time-consuming and inefficient survey method. Summary of the Invention

[0004] In view of this, the purpose of this application is to provide a method, apparatus, electronic device and storage medium for determining personalized skin type, which can determine the skin type of human face skin, shorten the time required to determine skin type and improve the efficiency of skin type determination.

[0005] In a first aspect, embodiments of this application provide a method for determining a personalized skin type, the method comprising:

[0006] Acquire global facial images under different shooting light source types;

[0007] Extracting local facial images from a global facial image, each local facial image contains information about the facial location region and the type of light source.

[0008] Based on the facial location region information and light source type information, the local facial image is input into the corresponding skin detection model to obtain the skin type detection result;

[0009] The skin detection model is trained based on local facial image samples and corresponding skin type labels.

[0010] In one possible implementation, acquiring global facial images corresponding to different shooting light source types includes:

[0011] Acquire global images of a human face under shooting conditions of standard light, cross-polarized light, and parallel-polarized light;

[0012] Extracting local facial images from a global facial image includes:

[0013] If the shooting conditions are standard lighting, extract the cheek area image or the full profile image from the global face image;

[0014] If the shooting conditions are cross-polarized light, extract the cheek area image, cheekbone area image and / or full profile image from the global face image;

[0015] If the shooting conditions are parallel polarized light, extract the full profile image from the global face image.

[0016] In one possible implementation, extracting a cheek area image or a full profile image from a global face image includes:

[0017] If the global face image is a standard frontal face image, then extract the cheek area image from the global face image;

[0018] If the global face image is a side profile image, then extract the full side profile image from the global face image.

[0019] In one possible implementation, extracting cheek region images, cheekbone region images, and / or full profile images from a global face image includes:

[0020] If the global face image is a frontal red area image, then extract the cheek area image from the global face image;

[0021] If the global face image is a frontal brown spot image, then extract the cheekbone area image from the global face image;

[0022] If the global face image is a side profile image, then extract the cheek area image, cheekbone area image, and full side profile image from the global face image.

[0023] In one possible implementation, extracting a full profile face image from a global face image includes:

[0024] If the global face image is a side profile image, then extract the full side profile image from the global face image.

[0025] In one possible implementation, based on facial location region information and light source type information, a local facial image is input into a corresponding skin detection model to obtain skin type detection results, including:

[0026] The image of the cheek area, with the facial location information as the cheek region and the light source type information as standard light, is input into the first skin detection model to determine whether the skin type detection result is dry skin type or oily skin type.

[0027] The image of the cheek area with the facial location region information as cheek region and the light source type information as cross-polarized light is input into the second skin detection model to determine whether the skin type detection result is sensitive skin type or tolerant skin type;

[0028] The image of the cheekbone region with facial location information as cheekbone area and light source type information as cross-polarized light is input into the third skin detection model to determine whether the skin type detection result is pigmented skin type or non-pigmented skin type.

[0029] The full-side face image, whose face location information is the entire side face region, is input into the fourth skin detection model to determine whether the skin type detection result is wrinkled skin type or tight skin type.

[0030] In one possible implementation, the method for determining the personalized skin type further includes:

[0031] Obtain partial facial image samples and their corresponding skin types;

[0032] Using local facial image samples as sample data and the corresponding skin type as a label, the skin detection model is trained.

[0033] Secondly, embodiments of this application also provide a personalized skin type determination device, which includes:

[0034] The acquisition module is used to acquire global facial images corresponding to different shooting light source types;

[0035] The extraction module is used to extract local facial images from the global facial image. The local facial images contain facial location information and light source type information.

[0036] The input module is used to input a local image of a face into the corresponding skin detection model to obtain the skin type detection result based on the face location region information and the light source type information;

[0037] The skin detection model is trained based on local facial image samples and corresponding skin type labels.

[0038] In one possible implementation, the acquisition module is specifically used to acquire global images of a face under shooting conditions of standard light, cross-polarized light, and parallel-polarized light, respectively.

[0039] The extraction module is specifically used to extract the cheek area image or the full profile image from the global face image if the shooting conditions are standard light; to extract the cheek area image, the cheekbone area image and / or the full profile image from the global face image if the shooting conditions are cross-polarized light; and to extract the full profile image from the global face image if the shooting conditions are parallel polarized light.

[0040] In one possible implementation, the extraction module is specifically used to extract a cheek area image from the global face image if the global face image is a standard frontal face image; and to extract a full side profile image from the global face image if the global face image is a side profile image.

[0041] In one possible implementation, the extraction module is specifically used to extract the cheek area image from the global face image if the global face image is a frontal red area image; to extract the cheekbone area image from the global face image if the global face image is a frontal brown area image; and to extract the cheek area image, cheekbone area image, and full side profile image from the global face image if the global face image is a side profile image.

[0042] In one possible implementation, the extraction module is specifically used to extract a full profile image from the global face image if the global face image is a profile image.

[0043] In one possible implementation, the input module is specifically used to input an image of a cheek region with face location information as cheek area and light source type information as standard light into a first skin detection model, and determine the skin type detection result as dry skin type or oily skin type; input an image of a cheek region with face location information as cheek area and light source type information as cross-polarized light into a second skin detection model, and determine the skin type detection result as sensitive skin type or tolerant skin type; input an image of a cheekbone region with face location information as cheekbone area and light source type information as cross-polarized light into a third skin detection model, and determine the skin type detection result as pigmented skin type or non-pigmented skin type; input a full-side face image with face location information as full-side face area into a fourth skin detection model, and determine the skin type detection result as wrinkled skin type or firm skin type.

[0044] In one possible implementation, the personalized skin type determination device further includes: a training module;

[0045] The acquisition module is also used to acquire partial facial image samples and their corresponding skin types;

[0046] The training module is used to train the skin detection model by using local facial image samples as sample data and the skin type corresponding to the local facial image samples as labels.

[0047] Thirdly, embodiments of this application also provide an electronic device, including: a processor, a storage medium, and a bus. The storage medium stores machine-readable instructions executable by the processor. When the electronic device is running, the processor communicates with the storage medium via the bus, and the processor executes the machine-readable instructions to perform the steps of the personalized skin type determination method as described in any of the first aspects.

[0048] Fourthly, embodiments of this application also provide a computer-readable storage medium storing a computer program, which, when executed by a processor, performs the steps of the personalized skin type determination method according to any of the first aspects.

[0049] This application provides a method, apparatus, electronic device, and storage medium for determining personalized skin types. The method includes: acquiring global facial images corresponding to different shooting light source types; extracting local facial images from the global facial images, where each local facial image contains facial location region information and light source type information; and inputting the local facial images into a corresponding skin detection model based on the facial location region information and light source type information to obtain a skin type detection result. The skin detection model is trained based on local facial image samples and corresponding skin type labels. This application, by extracting local facial images from global facial images and then inputting them into a corresponding skin detection model based on the facial location region information and light source type information to obtain a skin type detection result, can determine the skin type of a face, shortening the time required for skin type determination and improving the efficiency of skin type determination. Attached Figure Description

[0050] To more clearly illustrate the technical solutions of the embodiments of this application, the accompanying drawings used in the embodiments will be briefly introduced below. It should be understood that the following drawings only show some embodiments of this application and should not be regarded as a limitation of the scope. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort.

[0051] Figure 1 A flowchart illustrating a method for determining a personalized skin type according to an embodiment of this application is shown;

[0052] Figure 2 A flowchart illustrating another method for determining a personalized skin type provided in an embodiment of this application is shown;

[0053] Figure 3 A schematic diagram of a global facial feature image provided for an embodiment of this application is shown;

[0054] Figure 4 A schematic diagram of the structure of the first skin detection model provided in an embodiment of this application is shown;

[0055] Figure 5 A schematic diagram of the structure of the second skin detection model provided in an embodiment of this application is shown;

[0056] Figure 6 A schematic diagram of the structure of the third skin detection model provided in an embodiment of this application is shown;

[0057] Figure 7 A schematic diagram of the structure of the fourth skin detection model provided in an embodiment of this application is shown;

[0058] Figure 8 This illustration shows a schematic diagram of a personalized skin type determination device provided in an embodiment of this application;

[0059] Figure 9 A schematic diagram of the structure of an electronic device provided in an embodiment of this application is shown. Detailed Implementation

[0060] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. It should be understood that the accompanying drawings in this application are for illustrative and descriptive purposes only and are not intended to limit the scope of protection of this application. Furthermore, it should be understood that the schematic drawings are not drawn to scale. The flowcharts used in this application illustrate operations implemented according to some embodiments of this application. It should be understood that the operations in the flowcharts may not be implemented in sequence, and steps without logical contextual relationships may be reversed or implemented simultaneously. In addition, those skilled in the art, guided by the content of this application, may add one or more other operations to the flowcharts, or remove one or more operations from the flowcharts.

[0061] Furthermore, the described embodiments are merely some, not all, of the embodiments of this application. The components of the embodiments of this application described and illustrated herein can typically be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of this application provided in the accompanying drawings is not intended to limit the scope of the claimed application, but merely to illustrate selected embodiments of the application. All other embodiments obtained by those skilled in the art based on the embodiments of this application without inventive effort are within the scope of protection of this application.

[0062] To enable those skilled in the art to utilize the content of this application, and in conjunction with the specific application scenario of "skin detection technology," the following implementation methods are provided. For those skilled in the art, the general principles defined herein can be applied to other embodiments and application scenarios without departing from the spirit and scope of this application. Although this application is primarily described in the field of "skin detection technology," it should be understood that this is merely an exemplary embodiment.

[0063] It should be noted that the term "comprising" will be used in the embodiments of this application to indicate the presence of the features declared thereafter, but does not exclude the addition of other features.

[0064] The following is a detailed description of a method for determining a personalized skin type provided in an embodiment of this application.

[0065] Reference Figure 1 The diagram shown is a flowchart illustrating a method for determining a personalized skin type according to an embodiment of this application. The exemplary steps of this embodiment are described below:

[0066] S101. Obtain global facial images corresponding to different shooting light source types.

[0067] Specifically, global images of the face are acquired under shooting conditions of standard light, cross-polarized light, and parallel-polarized light.

[0068] In this application embodiment, the light source types include standard light, cross-polarized light, and parallel-polarized light. Standard light includes standard light 1 and standard light 2. A global face image refers to a complete face image captured at angles representing the front, left, and right sides of the face. A global face image includes standard frontal face images captured under standard light 1 and standard light 2 respectively, left and right face images captured under cross-polarized light respectively, a frontal face red area image and a frontal face brown spot image captured under cross-polarized light, and a left and right face image captured under parallel-polarized light respectively.

[0069] Standard light 1 uses universal dim light, providing balanced cross-beams to evaluate most skin characteristics. Standard light 2 is flat light, balancing the strongest light points and shadow areas in the image, making it easier to observe intuitive skin characteristics. Standard light 1 and 2 have no significant difference in their use for skin type detection and can be collectively referred to as standard light. The left and right face images are collectively referred to as profile images.

[0070] S102. Extract local face images from the global face image.

[0071] The partial face image includes information on the face location region and the light source type. A partial face image refers to a portion of the face image within the global face image. The face location region information includes the cheek region, cheekbone region, and full profile region. The light source type information includes standard light, cross-polarized light, and parallel-polarized light. The partial face image includes images where the light source type is standard light and the face location region is either the cheek region or the full profile region; images where the light source type is cross-polarized light and the face location region is either the cheek region or the cheekbone region; images where the light source type is cross-polarized light and the face location region is the full profile region; and images where the light source type is parallel-polarized light and the face location region is the full profile region.

[0072] Optionally, if the shooting conditions are standard lighting, the cheek area image or the full profile image can be extracted from the global face image.

[0073] Specifically, if the global face image is a standard frontal face image, then the cheek area image is extracted from the global face image; if the global face image is a side face image, then the full side face image is extracted from the global face image.

[0074] In this embodiment, the facial location region information corresponding to the cheek image is the cheek region, and the light source type information is standard light. The facial location region information corresponding to the profile image is the entire profile region, and the light source type information is standard light.

[0075] Optionally, if the shooting conditions are cross-polarized light, the cheek area image, the cheekbone area image, and / or the full profile image can be extracted from the global face image.

[0076] Specifically, if the global face image is a frontal red area image, then the cheek area image is extracted from the global face image; if the global face image is a frontal brown spot image, then the cheekbone area image is extracted from the global face image; if the global face image is a profile image, then the cheek area image, cheekbone area image, and full profile image are extracted from the global face image.

[0077] In this embodiment, the facial location region information corresponding to the cheek image captured under cross-polarized light conditions is the cheek region, and the light source type information is cross-polarized light. The facial location region information corresponding to the cheekbone image captured under cross-polarized light conditions is the cheekbone region, and the light source type information is cross-polarized light. The facial location region information corresponding to the full profile image captured under cross-polarized light conditions is the entire profile, and the light source type information is cross-polarized light.

[0078] Optionally, if the shooting conditions are parallel polarized light, a full profile image can be extracted from the global face image.

[0079] Specifically, if the global face image is a side profile image, then the full side profile image is extracted from the global face image.

[0080] In this embodiment of the application, the facial position area information corresponding to the full profile image captured under the condition of parallel polarized light is the entire profile, and the light source type information is parallel polarized light.

[0081] S103. Based on the face location information and light source type information, input the local face image into the corresponding skin detection model to obtain the skin type detection result.

[0082] The skin detection model is trained based on local face image samples and corresponding skin type labels. The skin type detection result is the skin type detection result of the entire face in the global face image corresponding to the local face image.

[0083] Optionally, the cheek image, with the face location information being the cheek region and the light source type information being standard light, is input into the first skin detection model to determine whether the skin type detection result is dry skin type or oily skin type.

[0084] In this embodiment of the application, the first skin detection model is used to detect whether the skin type is dry or oily. Specifically, it determines whether the skin type detection result is dry or oily by using the face location area information as the cheek area and the light source type information as a standard light image of the cheek area.

[0085] In existing technologies, skin type (dry or oily) is typically determined by examining the central area of ​​the forehead. Generally, the T-zone and U-zone of the face are used to determine skin type, but the forehead only belongs to the T-zone. Therefore, determining skin type solely based on the forehead is inaccurate. The cheeks, located at the intersection of the T-zone and U-zone, are a more accurate indicator of skin type compared to the forehead.

[0086] Optionally, the image of the cheek area with facial location information as cheek region and light source type information as cross-polarized light is input into the second skin detection model to determine whether the skin type detection result is sensitive skin type or tolerant skin type.

[0087] In this embodiment, the second skin detection model is used to detect whether the skin type is sensitive or tolerant. Specifically, it determines the skin type as sensitive or tolerant by using a cheek image with face location information as the cheek region and light source information of cross-polarized light. The cheek image with face location information as the cheek region and light source information of cross-polarized light includes: a cheek image extracted from a frontal face red zone image and a cheek image extracted from a side face image; wherein, the cheek image extracted from the side face image includes cheek images extracted from the left side face image and cheek images extracted from the right side face image.

[0088] In existing technologies, skin type (sensitive or tolerant) is typically determined by examining both the frontal red area image and the full profile image. However, this application determines skin type solely through an image of the cheek region, which is the facial area. Compared to existing technologies, this method uses a smaller facial area.

[0089] Optionally, the image of the cheekbone region, with the facial location information being the cheekbone area and the light source type information being cross-polarized light, is input into the third skin detection model to determine whether the skin type detection result is pigmented skin type or non-pigmented skin type.

[0090] In this embodiment, the third skin detection model is used to detect whether the skin type is pigmented or non-pigmented. Specifically, it determines whether the skin type detection result is pigmented or non-pigmented by using an image of the cheekbone region with facial location information as the cheekbone region and light source type information as cross-polarized light.

[0091] Existing technologies typically determine skin type (pigmented or non-pigmented) by capturing images of the entire profile and frontal face under standard, cross-polarized, and parallel polarized light conditions, respectively. However, this application's embodiment determines skin type (pigmented or non-pigmented) solely through an image of the cheekbone region; therefore, the method used in this application covers a smaller facial area.

[0092] Optionally, the full-side face image with the face location region information as the full-side face region is input into the fourth skin detection model to determine whether the skin type detection result is wrinkled skin type or tight skin type.

[0093] In this embodiment, the fourth skin detection model is used to detect whether the skin type is wrinkled or firm. Specifically, it determines whether the skin type detection result is wrinkled or firm by using full-face images corresponding to standard light, cross-polarized light, and parallel polarized light, respectively, based on the light source type information.

[0094] Here, existing technologies generally determine skin type as wrinkled or firm by using images of the corner of the eye or nasolabial fold area corresponding to standard light, cross-polarized light, or parallel polarized light, respectively. However, wrinkles can also exist on the forehead and under the eyes; therefore, this application uses a full profile image for greater accuracy.

[0095] In the embodiments of this application, the use of facial local images with different facial location regions and different light source types will affect the skin type determination results. As shown in Table 1, this is a table of the accuracy of skin type detection results determined by facial local images with different facial location regions and light source types provided in the embodiments of this application.

[0096] Table 1

[0097]

[0098] This application provides a method for determining personalized skin type. The method includes: acquiring global facial images corresponding to different shooting light source types; extracting local facial images from the global facial images, where each local facial image contains facial location region information and light source type information; and inputting the local facial images into a corresponding skin detection model based on the facial location region information and light source type information to obtain a skin type detection result. The skin detection model is trained based on local facial image samples and corresponding skin type labels. This application, by extracting local facial images from global facial images and then inputting them into a corresponding skin detection model based on the facial location region information and light source type information, can determine the skin type of the face. Determining the skin type detection result using local facial images shortens the time required for skin type determination and improves the efficiency of skin type determination.

[0099] Reference Figure 2 The diagram shown is a flowchart illustrating a method for determining a personalized skin type according to an embodiment of this application. The exemplary steps of this embodiment are described below:

[0100] S201. Obtain a partial image sample of the face and the corresponding skin type.

[0101] In this embodiment, a global facial image sample of the user is acquired, and then the Dlib-68 facial feature point detection model is used to detect the global facial image sample to obtain a global facial feature image. This global facial feature image includes the contour points of the face and the coordinate information of the position points of the eyes, nose, and mouth. The image of the facial position region is extracted using the coordinate information of these position points to obtain a local facial image sample. Figure 3 The image shown is a schematic diagram of a global facial feature image provided in an embodiment of this application. The image includes the facial location region DO of the first skin detection model; the facial location region SR of the second skin detection model; and the facial region PN of the third skin detection model.

[0102] In this embodiment, after obtaining local facial image samples through a facial feature point detection model, image preprocessing and Gaussian filtering are performed to reduce image noise. Then, a binarization algorithm is used to highlight image features. The binarized local facial image samples are then divided into a training set and a validation set in a 7:3 ratio, and 20% is extracted from the validation set as the test set. Offline data augmentation and normalization are performed on the local facial image samples in the training set to obtain the final local facial image samples. This offline data augmentation uses two data augmentation methods: flipping and rotation. The number of local facial image samples is transformed into an augmentation factor multiplied by the original number of local facial image samples, where the augmentation factor refers to the multiplier by which the data increases after offline augmentation.

[0103] In this embodiment, horizontal and vertical flipping data augmentation methods and random 90-degree rotation data augmentation methods are used. If the enhancement factor is 2, the data growth factor after offline enhancement is 2, and the face local image sample is horizontally or vertically flipped. If the enhancement factor is defined as 4, the data growth factor after offline enhancement is 4, and the face local image sample is randomly rotated 90 degrees clockwise or counterclockwise.

[0104] Among them, the facial partial image samples include: standard light source type information, cheek area or full profile image sample with face location region information, cross-polarized light source type, cheek area or cheekbone area image sample with face location region information, cross-polarized light source type, full profile image sample with face location region information, and parallel polarized light source type, full profile image sample with face location region information.

[0105] Furthermore, the questionnaire results of users' BST (Baumann Skin Type) were collected, and then the questionnaire results were reviewed and corrected based on the dermatologist's judgment of the facial images to obtain the skin type corresponding to the user's facial partial image sample.

[0106] S202. Use local facial image samples as sample data and the skin type corresponding to the local facial image samples as labels to train the skin detection model.

[0107] In this embodiment, the skin detection model includes a first skin detection model, a second skin detection model, a third skin detection model, and a fourth skin detection model. The skin detection model in this embodiment is built upon the Inception-v3 convolutional neural network model, retaining only the top-level global average pooling layer of the Inception-v3 network.

[0108] The Inception-v3 convolutional neural network model, built using the TensorFlow deep learning framework, is trained on a pre-trained ImageNet dataset to obtain optimized initial parameters, thereby accelerating model training speed, recognition rate, and generalization ability. The convolutional neural network model consists of a 42-layer structure: eight 3x3 convolutional layers, two 3x3 pooling layers, eleven Inception modules, one 8x8 pooling layer, one linear output layer, and one "Softmax" classifier.

[0109] In convolutional neural networks, all pooling layers use max pooling. The first group of Inception modules contains three modules, each containing three 3x3 convolutional layers, four 1x1 convolutional layers, and one average pooling layer. The second group of Inception modules also contains three modules. The first module contains three 3x3 convolutional layers, one 1x1 convolutional layer, and one 3x3 max pooling layer; the second module contains four 1x1 convolutional layers, three 1x7 convolutional layers, three 7x1 convolutional layers, and one 3x3 average pooling layer; the third module has the same composition as the second module. The third group of Inception modules contains three modules. The first module includes two 1x1 convolutional layers, two 3x3 convolutional layers, one 1x7 convolutional layer, one 7x1 convolutional layer, and one 3x3 max pooling layer. The second module includes four 1x1 convolutional layers, two 1x3 convolutional layers, two 3x1 convolutional layers, one 3x3 convolutional layer, and one 3x3 average pooling layer. The third module has the same composition as the second module.

[0110] Here, the Inception module greatly reduces the amount of computation and the computation time by dividing the ordinary 3*3 convolutional layer into 1*3 convolutional layers and 3*1 convolutional layers.

[0111] In existing technologies, since there are obvious differences between images of skin with skin diseases and normal skin, the diagnosis results of using artificial intelligence to classify and diagnose skin images for skin diseases are very accurate. However, BST focuses on the type identification of healthy skin. Images of healthy skin are very similar and have little difference, which leads to certain obstacles in the type identification of healthy skin by artificial intelligence.

[0112] Based on the understanding of skin physiology and dermometry, this application addresses the issue from two aspects: local facial image samples and skin detection models. Firstly, it reduces noise in local facial image samples and amplifies their image features. Secondly, depending on the light source type and facial location of the local facial image samples, different skin detection models are input into them to determine different skin type detection results. Table 2 shows the accuracy of skin type detection results obtained by training the skin detection model with local facial image samples taken under different light source types, as provided in this application embodiment. By comparing different light source types and their corresponding accuracy results, the light source type used in this application was finally determined, achieving significant progress. Table 2 reveals that the light source types used in existing technologies suffer from severe underfitting, while the light source type used in this application yields a high-precision model with good fitting.

[0113] Table 2

[0114]

[0115] On the other hand, transfer learning was applied to optimize the skin detection model and parameters. A convolutional neural network model validated by ImageNet was used to optimize the structure and parameters of the convolutional neural network model, highlighting its advantages in classifying images on small sample sets, especially different types of healthy skin. It perfectly integrated subjective questionnaires with objective evaluations, improving the efficiency of BST classification.

[0116] Optionally, such as Figure 4The diagram shown is a structural schematic of the first skin detection model provided in this application embodiment. It extracts a 299*299*3 cheek area image sample from a standard frontal face image taken under standard light as the input to the InputLayer in the first skin detection model. The model retains Inception-v3. After the top-level global average pooling layer in the f convolutional neural network model, the model is fine-tuned by adding a 3*3 convolutional layer Conv2D. Conv2D includes a 3*3*2048*32 kernel and a 32-dimensional bias, using the ReLU activation function. L2 weight regularization is used to reduce overfitting. An average pooling layer GlobalAveragePooling2D, a fully connected layer Flatten, and three dense layers are added. The first dense layer includes a 32*1024 kernel and a 1024-dimensional bias; the second dense layer includes a 1024*10 kernel and a 10-dimensional bias; and the third dense layer includes a 10*1 kernel and a 1-dimensional bias. The first and second dense layers use the ReLU activation function, and the third dense layer uses a sigmoid classifier, passing through a dense-6 array to output the skin type detection result of the first skin detection model. During model training, all Inception-v3 network layers were frozen, with only the last 20 layers open. The model was trained and its parameters were tuned using the training set. Once the model converged, training was paused and the training data was saved. The changes in the model's loss function value over the training cycle were observed. Finally, the optimal model was obtained when the batch size was 16, the number of epochs was 30, and the dropout was 0.3. The training set accuracy was 100%, the validation set accuracy was 91.11%, and the test set accuracy was 91.7%.

[0117] Optionally, such as Figure 5The diagram shows the structure of the second skin detection model provided in this embodiment. It extracts cheek image samples from the red zone image of a frontal face captured under cross-polarized light, specifically from the cheek area on both sides of the nose. It also extracts cheek image samples from one side of the cheek in the left face image and cheek image samples with dimensions of 299*299*3 from the right face image. These samples are used as input to the InputLayer of the second skin detection model. The model retains the top-level global average pooling layer from the Inception-v3f convolutional neural network model and then fine-tunes it by adding one average pooling layer (GlobalAveragePooling2D) and one fully connected layer (Flatten). Dropout is used to discard 30% of the features to reduce overfitting. Two dense layers are added. The first dense layer includes a 2048*1024 kernel and an output bias of 1024; the second dense layer includes a 1024*2 kernel and an output bias of 2. The first dense layer uses the ReLU activation function. The fully connected layer and the second dense layer use a Sigmoid classifier, which is then passed through a dense-1 layer to output the skin type detection results of the second skin detection model. During model training, all Inception-v3 network layers are frozen, with only the last 20 layers open. The model is trained and its parameters are tuned using the training set. Once the model converges, training is paused and the training data is saved. The changes in the model's loss function value over the training cycle are observed. Finally, the optimal model is obtained when the batch size is 32, the number of epochs is 30, and the dropout is 0.3, with a training set accuracy of 85.57%, a validation set accuracy of 81.13%, and a test set accuracy of 76.2%.

[0118] Optionally, such as Figure 6The diagram shows the structure of the third skin detection model provided in this application embodiment. It extracts cheekbone image samples from a frontal face image with brown spots captured under cross-polarized light, extracts cheekbone image samples from one side of the cheekbone region in a left-side face image, and extracts cheekbone image samples with dimensions of 299*299*3 from one side of the cheekbone region in a right-side face image. These samples are used as input to the InputLayer in the third skin detection model. The model retains the top-level global average pooling layer from the Inception-v3f convolutional neural network model and then fine-tunes it by adding one average pooling layer (GlobalAveragePooling2D) and one fully connected layer (Flatten). Dropout is used to discard 30% of the features to reduce overfitting. Two dense layers are added: the first dense layer includes a 2048*1024 kernel and an output bias of 1024; the second dense layer includes a 1024*2 kernel and an output bias of 2. The first Dense layer uses the ReLU activation function. A softmax classifier is used through a fully connected layer and the second Dense layer, followed by a dense-1 layer to output the skin type detection results of the third skin detection model. During model training, all Inception-v3 network layers are frozen, with only the last 20 layers open. The model is trained and its parameters are tuned using the training set. Once the model converges, training is paused and the training data is saved. The changes in the model's loss function value over the training cycle are observed. The optimal model is obtained when the batch size is 32, the number of epochs is 40, and the dropout is 0.3, achieving a training set accuracy of 99.50% and a validation set accuracy of 91.72%.

[0119] Optionally, such as Figure 7The diagram shows the structure of the fourth skin detection model provided in this application embodiment. A full-profile image of 299*299*3 pixels extracted from left and right face images captured under standard light, cross-polarized light, and parallel polarized light is used as the input to the InputLayer in the fourth skin detection model. The model retains the top-level global average pooling layer from the Inception-v3f convolutional neural network model and then fine-tunes it by adding one average pooling layer (GlobalAveragePooling2D) and one fully connected layer (Flatten). Dropout is used to discard 20% of features to reduce overfitting, and two dense layers are added. The first dense layer includes a 2048*1024 kernel and an output bias of 1024; the second dense layer includes a 1024*2 kernel and an output bias of 2. The first dense layer uses the ReLU activation function, and the skin type detection result of the fourth skin detection model is output through a dense-5 layer via the fully connected layer and the second dense layer using a Sigmoid classifier. During model training, all Inception-v3 network layers were frozen, with only the last 20 layers enabled. The model was trained and its parameters were tuned using the training set. Once the model converged, training was paused and the training data was saved. The changes in the model's loss function value over the training cycle were observed. Finally, the optimal model was obtained when the batch size was 16 and the number of epochs was 30, with a training set accuracy of 72.5%, a validation set accuracy of 74.9%, and a test set accuracy of 75.9%.

[0120] Here, this embodiment does not directly use the Inception-v3 convolutional neural network model to determine skin type. Instead, it establishes a first skin detection model, a second skin detection model, a third skin detection model, and a fourth skin detection model based on the Inception-v3 model, and then trains the optimal model to determine the skin type. Table 3 shows a comparison of the accuracy results of using the convolutional neural network model as the skin detection model and using the optimal model as the skin detection model in this embodiment.

[0121] Table 3

[0122]

[0123] Furthermore, existing technologies typically use MobileNet, ResNet50, and Vgg16 models as the base models for skin detection. Table 4 shows a comparison of the accuracy of existing technologies using MobileNet, ResNet, and Vgg16 models as base models with the Inception-v3 model used in this application, as provided in the embodiments of this application.

[0124] Table 4

[0125]

[0126] Here, "fine-tuning" refers to a transfer learning method. Specifically, it involves using the parameters of a model trained for other tasks as the initial parameters for the current task model, thereby reducing training time and improving robustness. "Revised Linear Unit Layer 'ReLU'" is a mapping function, and its formula is as follows:

[0127] The "Sigmoid classifier" refers to a commonly used supervised binary classification function, whose function formula is f(x) = 1 / (1+e^x). -x The Sigmoid function's range is limited to (0,1), the same as the probability distribution. Furthermore, since its derivation properties are identical to the Bernoulli distribution function followed by binary classification problems, this model calculates the probabilities of each of the two classifications and selects the result with the higher probability as the output.

[0128] The "Softmax classifier" refers to a commonly used supervised multi-class classification function, and its function calculation formula is as follows: Where z i Let C be the output value of the i-th node, and C be the number of output nodes, i.e. the number of categories. The Softmax function can be used to convert the output values ​​of multi-class classification into a probability distribution with a range of [0,1] and a value of 1.

[0129] This application provides another method for determining personalized skin types. The method includes: acquiring partial facial image samples and their corresponding skin types; using the partial facial image samples as sample data and the corresponding skin types as labels to train a skin detection model. This method can train a skin detection model to obtain a skin detection model with high accuracy.

[0130] Reference Figure 8 The diagram shown is a schematic of a personalized skin type determination device provided in an embodiment of this application. The personalized skin type determination device includes:

[0131] The acquisition module 801 is used to acquire global face images corresponding to different shooting light source types;

[0132] The extraction module 802 is used to extract a partial face image from the global face image. The partial face image contains information about the face location region and the type of light source.

[0133] The input module 803 is used to input a local image of a face into the corresponding skin detection model to obtain the skin type detection result based on the face location region information and the light source type information;

[0134] The skin detection model is trained based on local facial image samples and corresponding skin type labels.

[0135] In one possible implementation, the acquisition module 801 is specifically used to acquire global images of a face under shooting conditions of standard light, cross-polarized light, and parallel-polarized light, respectively.

[0136] The extraction module 802 is specifically used to extract the cheek area image or the full profile image from the global face image if the shooting conditions are standard light; to extract the cheek area image, the cheekbone area image and / or the full profile image from the global face image if the shooting conditions are cross-polarized light; and to extract the full profile image from the global face image if the shooting conditions are parallel polarized light.

[0137] In one possible implementation, the extraction module 802 is specifically used to extract the cheek area image from the global face image if the global face image is a standard frontal face image; and to extract the full side profile image from the global face image if the global face image is a side profile image.

[0138] In one possible implementation, the extraction module 802 is specifically used to extract the cheek area image from the global face image if the global face image is a frontal red area image; extract the cheekbone area image from the global face image if the global face image is a frontal brown spot image; and extract the cheek area image, cheekbone area image, and full side profile image from the global face image if the global face image is a side profile image.

[0139] In one possible implementation, the extraction module 802 is specifically used to extract a full side profile image from the global face image if the global face image is a side profile image.

[0140] In one possible implementation, the input module 803 is specifically used to input an image of a cheek region with face location information as cheek area and light source type information as standard light into a first skin detection model to determine the skin type detection result as dry skin type or oily skin type; input an image of a cheek region with face location information as cheek area and light source type information as cross-polarized light into a second skin detection model to determine the skin type detection result as sensitive skin type or tolerant skin type; input an image of a cheekbone region with face location information as cheekbone area and light source type information as cross-polarized light into a third skin detection model to determine the skin type detection result as pigmented skin type or non-pigmented skin type; and input a full-side face image with face location information as full-side face area into a fourth skin detection model to determine the skin type detection result as wrinkled skin type or firm skin type.

[0141] In one possible implementation, the personalized skin type determination device further includes: a training module 804;

[0142] The acquisition module 801 is also used to acquire partial facial image samples and corresponding skin types;

[0143] Training module 804 is used to train the skin detection model by using local face image samples as sample data and the skin type corresponding to the local face image samples as labels.

[0144] This application provides a personalized skin type determination device, comprising: an acquisition module 801 for acquiring global facial images corresponding to different shooting light source types; an extraction module 802 for extracting local facial images from the global facial images, wherein the local facial images correspond to facial location region information and light source type information; and an input module 803 for inputting the local facial images into a corresponding skin detection model based on the facial location region information and light source type information to obtain a skin type detection result; wherein the skin detection model is trained based on local facial image samples and corresponding skin type labels. This application, by extracting local facial images from the global facial images and then inputting the local facial images into a corresponding skin detection model based on the facial location region information and light source type information to obtain a skin type detection result, can determine the skin type of the face, shortening the time required for skin type determination and improving the efficiency of skin type determination.

[0145] like Figure 9As shown in the embodiment of this application, an electronic device 900 includes a processor 901, a memory 902, and a bus. The memory 902 stores machine-readable instructions that can be executed by the processor 901. When the electronic device is running, the processor 901 communicates with the memory 902 through the bus. The processor 901 executes the machine-readable instructions to perform the steps of the personalized skin type determination method described above.

[0146] Specifically, the memory 902 and processor 901 mentioned above can be general-purpose memory and processor, without any specific limitations. When the processor 901 runs the computer program stored in the memory 902, it can execute the above-mentioned method for determining the personalized skin type.

[0147] Corresponding to the above-described method for determining personalized skin types, this application also provides a computer-readable storage medium storing a computer program, which, when executed by a processor, performs the steps of the above-described method for determining personalized skin types.

[0148] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working processes of the systems and devices described above can be referred to the corresponding processes in the method embodiments, and will not be repeated here. In the several embodiments provided in this application, it should be understood that the disclosed systems, devices, and methods can be implemented in other ways. The device embodiments described above are merely illustrative. For example, the division of modules is only a logical functional division, and in actual implementation, there may be other division methods. Furthermore, multiple modules or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the displayed or discussed mutual coupling or direct coupling or communication connection can be through some communication interfaces; the indirect coupling or communication connection of devices or modules can be electrical, mechanical, or other forms.

[0149] The modules described as separate components may or may not be physically separate. The components shown as modules may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0150] In addition, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit.

[0151] If the aforementioned functions are implemented as software functional units and sold or used as independent products, they can be stored in a processor-executable, non-volatile, computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or a portion of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the information processing methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, ROM, RAM, magnetic disks, or optical disks.

[0152] The above are merely specific embodiments of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the technical scope disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A method for determining a personalized skin type, characterized in that, The method for determining the personalized skin type includes: Acquire global facial images corresponding to different shooting light source types, wherein the shooting light source types include standard light, cross-polarized light, and parallel-polarized light; A partial face image is extracted from the global face image, and the partial face image corresponds to face location region information and light source type information; Based on the facial location region information and light source type information, the local facial image is input into the corresponding skin detection model to obtain the skin type detection result; The skin detection model is trained based on local facial image samples and corresponding skin type labels. The step of extracting a partial face image from the global face image includes: If the shooting conditions are standard light, extract the cheek area image or the full profile image from the global face image; If the shooting conditions are cross-polarized light, extract the cheek area image, cheekbone area image and / or full profile image from the global face image; If the shooting conditions are parallel polarized light, extract the full profile image from the global face image; The step of inputting the local facial image into the corresponding skin detection model to obtain skin type detection results based on the facial location region information and light source type information includes: The image of the cheek area, with the facial location information as the cheek region and the light source type information as standard light, is input into the first skin detection model to determine whether the skin type detection result is dry skin type or oily skin type. The image of the cheek area with the facial location region information as cheek region and the light source type information as cross-polarized light is input into the second skin detection model to determine whether the skin type detection result is sensitive skin type or tolerant skin type; The image of the cheekbone region with facial location information as cheekbone area and light source type information as cross-polarized light is input into the third skin detection model to determine whether the skin type detection result is pigmented skin type or non-pigmented skin type. The full-side face image with the face location information as the full-side face region is input into the fourth skin detection model to determine whether the skin type detection result is wrinkled skin type or tight skin type.

2. The method for determining personalized skin type according to claim 1, characterized in that, Extracting cheek area images or full profile images from the global face image includes: If the global face image is a standard frontal face image, then the cheek area image is extracted from the global face image; If the global face image is a side profile image, then the full side profile image is extracted from the global face image.

3. The method for determining personalized skin type according to claim 1, characterized in that, Extracting cheek area images, cheekbone area images, and / or full profile images from the global facial image includes: If the global face image is a frontal red area image, then the cheek area image is extracted from the global face image; If the global face image is a frontal brown spot image, then the cheekbone area image is extracted from the global face image; If the global face image is a side profile image, then the cheek area image, cheekbone area image, and full side profile image are extracted from the global face image.

4. The method for determining personalized skin type according to claim 1, characterized in that, Extracting the full profile image from the global face image includes: If the global face image is a side profile image, then the full side profile image is extracted from the global face image.

5. The method for determining personalized skin type according to claim 1, characterized in that, The method for determining personalized skin types also includes: Obtain partial facial image samples and their corresponding skin types; The facial partial image samples are used as sample data, and the skin type corresponding to the facial partial image samples is used as a label to train the skin detection model.

6. A device for determining a personalized skin type, characterized in that, The device for determining the personalized skin type includes: The acquisition module is used to acquire global facial images corresponding to different shooting light source types, wherein the shooting light source types include standard light, cross-polarized light, and parallel-polarized light; The extraction module is used to extract a partial face image from the global face image, wherein the partial face image corresponds to face location region information and light source type information; The input module is used to input the local face image into the corresponding skin detection model to obtain the skin type detection result based on the face location region information and light source type information; The skin detection model is trained based on local facial image samples and corresponding skin type labels. The extraction module is further configured to extract a cheek image or a full profile image from the global face image if the shooting conditions are standard light; extract a cheek image, a cheekbone image, and / or a full profile image from the global face image if the shooting conditions are cross-polarized light; and extract a full profile image from the global face image if the shooting conditions are parallel-polarized light. The input module is further configured to input an image of a cheek with face location information as cheek region and light source type information as standard light into a first skin detection model to determine whether the skin type detection result is dry skin or oily skin; input an image of a cheek with face location information as cheek region and light source type information as cross-polarized light into a second skin detection model to determine whether the skin type detection result is sensitive skin or tolerant skin; input an image of a cheekbone with face location information as cheekbone region and light source type information as cross-polarized light into a third skin detection model to determine whether the skin type detection result is pigmented skin or non-pigmented skin; and input a full-side face image with face location information as the entire side face region into a fourth skin detection model to determine whether the skin type detection result is wrinkled skin or firm skin.

7. An electronic device, characterized in that, include: The device includes a processor, a storage medium, and a bus, wherein the storage medium stores machine-readable instructions executable by the processor, and when the electronic device is in operation, the processor communicates with the storage medium via the bus, and the processor executes the machine-readable instructions to perform the steps of the method for determining a personalized skin type as described in any one of claims 1 to 5.

8. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a processor, performs the steps of the method for determining a personalized skin type as described in any one of claims 1 to 5.

Images