Skin classification method fusing smoothness and spot recognition

Abstract

The invention belongs to the technical field of skin analysis, and particularly relates to a skin classification method fusing smoothness and spot recognition, which comprises the following steps: acquiring a user skin image through a mobile terminal, and outputting a standardized image after self-adaptive histogram equalization and non-local mean filtering preprocessing; a smoothness feature vector and a spot feature vector are extracted in parallel based on a standardized image, a fusion feature vector is obtained through calculation based on the smoothness feature vector and the spot density feature vector, and after the fusion feature vector is input into a skin classification model, key parameters can be extracted and decoded based on the obtained fusion adjustment vector. Depending on a spot density enhancement value, smoothness principal component strength and comparison between a covariant weight parameter and a threshold value, the skin type of the user is determined hierarchically, and decoding based on the smoothness principal component strength can filter noise and is more sensitive than a smoothness mean value.

Description

Technical Field

[0001] The present invention belongs to the technical field of skin quality analysis, and specifically is a skin quality classification method integrating smoothness and spot recognition. Background Art

[0002] With the development of society and the improvement of living standards, people pay more and more attention to their personal image, especially the skin quality. Generally speaking, some common problematic skin can be identified through ordinary human observation, but if you want to detect skin quality more accurately, you need to use scientific means.

[0003] In the current existing technology, traditional solutions directly use original features, such as spot density , smoothness mean However, there is a physiological coupling between the two. For example, oily skin usually has fewer spots, which leads to the decision boundary being limited by the statistical deviation of the original data. In addition, the smooth eigenvector Generally, they contain 64-dimensional grid features, but through mean calculation, the smoothness mean can only reflect local key information.

[0004] To this end, the present invention provides a skin quality classification method that integrates smoothness and spot recognition. Summary of the Invention

[0005] In order to make up for the deficiencies of the prior art, at least one technical problem raised in the background technology is solved.

[0006] The technical solution adopted by the present invention to solve the technical problem is: a skin quality classification method integrating smoothness and spot recognition according to the present invention comprises the following steps: S1: Obtain the user's skin image through the mobile terminal, and output a standardized image after preprocessing with adaptive histogram equalization and non-local mean filtering. ; S2: Based on standardized image Parallel extraction: Smoothness eigenvector ,in, is the entropy value of the gray-level co-occurrence matrix; Blob feature vector , blob density calculated based on the U-Net segmentation network ; S3: Constructing the covariance matrix , and generate the fusion feature vector ,in: ; S4: Fusion feature vector Input the skin type classification model and output the skin type classification results.

[0007] Preferably, in S4, the method for outputting the skin type classification result is: from Decode key parameters, including speckle density enhancement value , smoothness principal component intensity and covariation weights ; Among them, if , it is marked as pathological skin; like ,and , then the output is oily skin; like ,and ,and , then the output is photoaged skin; Otherwise, output normal / combination skin.

[0008] Preferably, in S2, the smoothness feature extraction includes: Will Divide into 8×8 grid; Calculate the entropy of the gray-level co-occurrence matrix at 0°, 45°, 90°, and 135° for each grid: ; in, Grayscale value The probability of occurrence; when When the skin is thicker than normal, it is judged as a high sebum secretion area.

[0009] Preferably, in S2, the spot density By improving the U-Net network calculation: The network adds a SE attention module to the encoder; The loss function is calculated according to the formula: ; Output: .

[0010] Preferably, in S3, the covariance matrix The calculation basis includes: ; in, is the number of samples; For the The smoothness feature vector of samples, For the The spot density of each sample, is the global mean of the smoothness feature, Global mean of spot density; represents the smoothness deviation vector, Scalar representing the blob density deviation.

[0011] Preferably, the S1 further includes: S11: Standardized image Perform multi-level image quality diagnosis, including posture angle detection, dynamic blur assessment, and illumination uniformity analysis. If the image quality meets the standard, the smoothness feature vector and the speckle feature vector are extracted; Otherwise, start real-time guided correction.

[0012] Preferably, in said S11, the method for posture angle detection is: Use facial keypoint detection model to locate the standardized image The coordinates of the user's nose tip and pupils; Calculating Euler angle deviation : ; in, is the current facial normal vector, is the normal reference vector; when , the posture is judged to be unqualified.

[0013] Preferably, in S11, the method of dynamic fuzzy evaluation is: Calculate Laplace variance : ; in, is the image Laplacian response, is the pixel coordinate; is the Laplace response mean; Indicates local deviation; when , it is determined that the dynamic blur exceeds the standard.

[0014] Preferably, in S11, the method for analyzing illumination uniformity is: Segment the face into four regions: forehead, left cheek, right cheek, and chin; Calculate the average brightness of each area in HSV space ; like or The lighting is judged to be unqualified.

[0015] Preferably, in S11, the method for real-time guidance correction is: At the same time, when When the direction of the arrow is displayed, a directional arrow is displayed; When the light is not uniform, a yellow flashing warning is displayed; When the light is not uniform, a yellow flashing warning is displayed; When the light is not uniform, a yellow flashing warning is displayed; Capture 5 consecutive images, and automatically select the frame image with the largest Euler angle deviation , and .

[0016] The beneficial effects of the present application are as follows: 1. The skin classification method fusing smoothness and spot recognition according to the present application, by calculating the fusion feature vector based on the smoothness feature vector and the spot density feature vector, then inputting the fusion feature vector into the skin classification model, can extract and decode the key parameters based on the obtained fusion adjustment vector, including the spot density enhancement value , the smoothness principal component intensity , and the covariance weight , and finally rely on the size comparison of the spot density enhancement value , the smoothness principal component intensity , and the covariance weight parameters and thresholds to determine the user's skin type in layers, wherein the decoding based on the smoothness principal component intensity can filter noise, and since the smoothness principal component intensity fuses the multi-grid entropy value distribution feature, it is more sensitive than the smoothness mean value. BRIEF DESCRIPTION OF DRAWINGS

[0017] The present application will be further described below in conjunction with the drawings.

[0018] Figure 1 is a flowchart of the present application. DETAILED DESCRIPTION

[0019] In order to make the technical means, creative features, purposes and effects realized by the present application easy to understand, the present application will be further described below in conjunction with specific embodiments.

[0020] Example 1: As shown in Figure 1 , the skin classification method fusing smoothness and spot recognition according to the present application includes the following steps: S1: Obtain the user's skin image through the mobile terminal, and output the standardized image after adaptive histogram equalization and non-local mean filtering preprocessing; S2: Based on the standardized image , extract in parallel: the smoothness feature vector , wherein, is a gray level co-occurrence matrix entropy value; spot feature vector , a spot density calculated based on a U-Net segmentation network ; S3: constructing a covariance matrix , and generating a fusion feature vector , wherein: ; S4: inputting the fusion feature vector into a skin quality classification model to output a skin quality classification result.

[0021] In the S4, the method for outputting the skin quality classification result is: decoding key parameters from the , including a spot density enhancement value , a smoothness principal component intensity , and a covariance weight ; If , it is marked as pathological skin quality; If , and , it is output as oily skin; If , and , and , it is output as photoaging skin; Otherwise, it is output as neutral / mixed skin.

[0022] The traditional scheme directly uses original features such as spot density , smoothness mean value to make decisions, but both of them have physiological coupling, such as oily skin usually has few spots, which limits the decision boundary to the statistical bias of the original data, and in addition, the smooth feature vector generally contains 64-dimensional grid features, but through mean value calculation, the smoothness mean value can only reflect local key information.

[0023] The following uses a use case constructed with specific data to explain and illustrate: Suppose there is a female user, aged 35, and an outdoor worker, with facial pigmentation and dryness problems. When the user uses a mobile terminal to take an image, an original image is obtained, and the original image does not have any problems. After preprocessing, the original image is defined as a standardized image corresponding to the user After obtaining the standardized image corresponding to the user , based on the standardized image smoothness feature extraction, the specific method comprising dividing the facial image into an 8x8 grid, corresponding to containing 64 sub-regions, and then calculating the GLCM entropy value of the right cheek region, assuming that all the calculated entropy values are as follows: ; the smoothness mean value is obtained ; According to the above, all the obtained entropy values are collected as a smoothness feature vector ; The user's spot feature vector is obtained again ; Based on this, the obtained smoothness feature vector and the spot feature vector are used to construct a covariance matrix , which is expressed as follows: The covariance matrix is constructed; The obtained fusion feature vector is input into the skin quality classification model, and based on the decoding of the key parameters of the fusion feature vector in the model, the following parameters are obtained: Among them, the spot density enhancement value ; Smoothness principal component intensity , wherein is half of the grid number; Covariance weight ; The above parameters are calculated respectively, as shown below: ; ; ; According to the above decision logic, when , it is marked as pathological skin quality, and the user is output with the judgment and recognition result of pathological skin quality after calculation and decision analysis; it should be noted that in an embodiment, , it is marked as pathological skin quality, corresponding to being set as the first layer of decision; , and , outputting oily skin, corresponding to being set as the second layer of decision; , and , and , outputting photoaging skin, corresponding to being set as the third layer of decision; none of the above classifications, outputting neutral / mixed skin, corresponding to being set as the third layer of decision, in this embodiment, based on the decoded spot density enhancement value , smoothness principal component intensity , and covariance weight In the analysis and identification of the user's skin type, the spot density enhancement value , smoothness principal component intensity and the covariance weight are compared with the threshold value. It can be understood that if the spot density enhancement value , the user's skin type is pathological skin, and the remaining parameters are no longer compared. Conversely, if the spot density enhancement value , further comparison is made in sequence, and the user's skin type is output. Based on the above, in an embodiment of the present application, the fusion feature vector calculated based on the smoothness feature vector and the spot density feature vector is input into the skin classification model, and based on the obtained fusion adjustment vector, the key parameters including the spot density enhancement value , smoothness principal component intensity and covariance weight are extracted and decoded. Finally, the size comparison of the spot density enhancement value , smoothness principal component intensity and covariance weight parameters with the threshold value is used to determine the user's skin type in layers. The decoding based on the smoothness principal component intensity can filter noise, and since the smoothness principal component intensity fuses the multi-grid entropy value distribution feature, it is more sensitive than the smoothness mean value.

[0024] In an embodiment, the smoothness feature extraction in S2 includes: dividing the image into 8x8 grids; calculating the entropy values of the gray level co-occurrence matrix in 0°, 45°, 90° and 135° directions for each grid: ; wherein, is the occurrence probability of the gray value ; when , it is determined as a high sebum secretion area.

[0025] As described above, according to the input and processing of the user's standardized image , the facial image is divided into 8x8 grids, corresponding to 64 sub-regions. Then, the GLCM entropy values of the right cheek area are calculated. Assuming that all the calculated entropy values are as follows: ; and among them, according to the rule of , the area of is defined as a high sebum secretion area. As described above, ​The corresponding area is the right cheek, and the entropy value of the right cheek area is less than 2.0, and it is determined that the right cheek is a high sebum secretion area.

[0026] In an embodiment, in the S2, the spot density The U-Net network is improved by: The network adds an SE attention module in the encoder; The loss function is calculated according to the formula: ; Output: .

[0027] It can be understood that the original image uploaded by the user, after preprocessing, the standardized image In the calculation of the spot density feature vector, it is assumed that the number of region spot pixels is 8532, and the total number of region skin pixels is 85300, then according to the calculation, as follows: ; In an embodiment, in the S3, the covariance matrix The calculation is based on: ; Wherein, is the number of samples; is the smoothness feature vector of the th sample, is the spot density of the th sample, is the global mean of the smoothness feature, the global mean of the spot density; denotes the smoothness deviation vector, denotes the spot density deviation scalar.

[0028] Embodiment two: In an embodiment, the S1 further comprises: S11: performing multi-level image quality diagnosis on the standardized image , including pose angle detection, dynamic blur evaluation and illumination uniformity analysis, if The image quality meets the standard, the smoothness feature vector and the spot feature vector are extracted; Otherwise, real-time guidance correction is started.

[0029] In an embodiment, in the S11, the method of pose angle detection is: Use a face key point detection model to locate the coordinates of the user's nose tip and both eye pupils in the standardized image ; Calculate the Euler angle deviation : ; wherein, is the current face normal vector, is the frontal reference vector; when , the pose is determined to be unqualified.

[0030] In an embodiment, the method for dynamic blur evaluation in S11 is: calculating Laplacian variance : ; wherein, is the image Laplacian operator response, is the pixel point coordinate; is the mean value of the Laplacian response; represents local deviation; when , the dynamic blur is determined to be over-standard.

[0031] In an embodiment, the method for illumination uniformity analysis in S11 is: segmenting the face into four regions of forehead, left cheek, right cheek, and chin; calculating the average luminance in HSV space of each region ; if or , the illumination is determined to be unqualified.

[0032] In an embodiment, the method for real-time guidance correction in S11 is: simultaneously, when , a directional arrow is displayed; when , a yellow flashing warning is displayed; when the illumination is uneven, an adjustment light prompt is displayed; five frames of images are continuously captured, and the frame image with the Euler angle deviation , and is automatically selected.

[0033] In the skin classification of the user, according to the original image input by the user based on the mobile terminal, the smoothness feature vector and the spot feature vector can be extracted by preprocessing the original image, but the original image is not required, and it can be understood that, in the skin classification, the most important thing is whether the original image meets the standard, if the shooting angle of the original image deviates too much from the normal angle, or the original image is blurred, the light is poor, etc., which will affect the smoothness feature vector and the spot feature vector, resulting in unreliable skin classification result; on this basis, in an embodiment of the present application, when the user shoots the face image based on the mobile terminal, the user needs to be guided to help the user shoot the original image that meets the requirements, which is to avoid the flow of face images that do not meet the requirements into the system, causing storage pressure, specifically: When the user shoots the face image based on the mobile terminal, the face angle, dynamic blur and light uniformity of the shooting image need to be analyzed, assuming that the face angle in the original image is right 15.8°, and the threshold , then according to the comparison, the face angle of the user in the original image deviates too much, which is determined as unqualified posture, which can also be understood as, when the user shoots the first face image, uploads the system, and the system displays the unqualified posture and feedback to the user's mobile terminal, in order to avoid causing storage pressure, on this basis, the unqualified face original image also needs to be deleted, and then according to the guidance, the user re-shoots the face image, if the face image, recorded as the second face image, is uploaded to the system, and detected due to the user's hand shake, , then according to the analysis, the second face image of the user is blurred due to hand shake, which loses the real face state of the user, so even if the original image is used for spot density calculation and skin classification, it cannot truly reflect the real skin type of the user, on this basis, the system will also feedback to the user and give suggestions, in addition, it also includes the light uniformity analysis of each region of the user's face, assuming that the user uploads the third face image again, and according to the analysis, the right cheek brightness of the third face image is , and the left cheek brightness is , after calculation , then according to the analysis, the light uniformity of each region of the user's face is not up to standard, according to the above, through multi-level image quality diagnosis, including posture angle detection, dynamic blur evaluation and light uniformity analysis, the original face image uploaded by the user can be screened, and modification and guidance suggestions are given according to the original face image uploaded by the user; On the basis of the above, in order to help the user to assist in shooting the required front face image, and in the case of ensuring uniform illumination, an embodiment of the present application further comprises, according to the user mobile terminal real-time shooting process, the user can be guided by the App shooting action, it can be foreseen that the direction arrow, the flashing light and the adjustment light prompt the user current posture, the dynamic blur or the illumination uniformity aspect of the deficiency, reduces the user to repeat the number of times of shooting the face image, reduces the system to delete the action of the original image which does not meet the requirements; In addition, further, when the face image shot by the user meets the requirements, at least 5 frames of images are continuously captured, and the optimal frame is selected from them as the original image available to the user.

[0034] The basic principles, main features and advantages of the present application are shown and described above. Those skilled in the art should understand that the present application is not limited to the above-mentioned embodiments, and the above-mentioned embodiments and descriptions in the specification are only to illustrate the principles of the present application. Without departing from the spirit and scope of the present application, various changes and improvements can be made to the present application, and these changes and improvements all fall within the scope of the present application. The scope of protection of the present application is defined by the appended claims and their equivalents.

Claims

1. A skin quality classification method integrating smoothness and spot recognition, characterized by: The following steps are involved: S1: Obtain the user's skin image through the mobile terminal, and output a standardized image after preprocessing with adaptive histogram equalization and non-local mean filtering. ; S2: Based on standardized image Parallel extraction: Smoothness eigenvector ,in, is the entropy value of the gray-level co-occurrence matrix; Blob feature vector , blob density calculated based on the U-Net segmentation network ; S3: Constructing the covariance matrix , and generate the fusion feature vector ,in: ； S4: Fusion feature vector Input the skin type classification model and output the skin type classification results.

2. The skin quality classification method integrating smoothness and spot recognition according to claim 1, characterized in that: In S4, the method for outputting the skin quality classification result is: from Decode key parameters, including spot density enhancement value , smoothness principal component intensity and covariation weights ; Among them, if , it is marked as pathological skin; like ,and , then the output is oily skin; like ,and ,and , then the output is photoaged skin; Otherwise, output normal / combination skin.

3. The skin quality classification method integrating smoothness and spot recognition according to claim 2, characterized in that: In S2, the smoothness feature extraction includes: Will Divide into 8×8 grid; Calculate the entropy of the gray-level co-occurrence matrix at 0°, 45°, 90°, and 135° for each grid: ； in, Grayscale value The probability of occurrence; when When the skin is thicker than the skin around the neck, it is judged as a high sebum secretion area.

4. The skin quality classification method integrating smoothness and spot recognition according to claim 3, characterized in that: In S2, the spot density By improving the U-Net network calculation: The network adds a SE attention module to the encoder; The loss function is calculated according to the formula: ； Output: .

5. The skin quality classification method integrating smoothness and spot recognition according to claim 4, characterized in that: In S3, the covariance matrix The calculation basis of include: ； in, is the number of samples; For the The smoothness feature vector of samples, For the The spot density of the sample, is the global mean of the smoothness feature, Global mean of spot density; represents the smoothness deviation vector, Scalar representing the blob density deviation.

6. The skin quality classification method integrating smoothness and spot recognition according to claim 5, characterized in that: Said S1 further comprises: S11: Standardized image Perform multi-level image quality diagnosis, including posture angle detection, dynamic blur assessment, and illumination uniformity analysis. If the image quality meets the standard, the smoothness feature vector and the speckle feature vector are extracted; Otherwise, start real-time guided correction.

7. The skin quality classification method integrating smoothness and spot recognition according to claim 6, characterized in that: In S11, the method for posture angle detection is: Use facial keypoint detection model to locate the standardized image The coordinates of the user's nose tip and pupils; Calculating Euler angle deviation : ； in, is the current facial normal vector, is the normal reference vector; when , the posture is judged to be unqualified.

8. The skin quality classification method integrating smoothness and spot recognition according to claim 7, characterized in that: In S11, the method of dynamic fuzzy evaluation is: Calculate the Laplace variance : ； in, is the image Laplacian response, is the pixel coordinate; is the Laplace response mean; Indicates local deviation; when , it is determined that the dynamic blur exceeds the standard.

9. The skin quality classification method integrating smoothness and spot recognition according to claim 8, characterized in that: In S11, the method for analyzing the uniformity of illumination is: Segment the face into four regions: forehead, left cheek, right cheek, and chin; Calculate the average brightness of each area in HSV space ; like or The lighting is judged to be unqualified.

10. The skin quality classification method integrating smoothness and spot recognition according to claim 9, characterized in that: In S11, the method for real-time guidance correction is: At the same time, when , a directional arrow is displayed; when When the alarm is on, a yellow flashing warning is displayed; When the lighting is uneven, a prompt to adjust the lighting is displayed; Capture 5 frames of image continuously and automatically select Euler angle deviation ,and Frame image.

Images