Skin detection method, electronic device, storage medium, and product

Abstract

The application provides a skin detection method, an electronic device, a storage medium and a product. The method comprises: acquiring a target image of a skin region to be detected; wherein the skin region to be detected comprises a facial skin region; determining a position region of a defect on the target image and a defect category to which the defect belongs; removing a background skin region corresponding to the position region according to the defect category, to obtain a defect region corresponding to the position region; and determining a boundary line of the defect according to pixel values of each pixel point in the defect region. The embodiment of the application improves the accuracy of facial defect detection.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of image processing, and particularly relates to a skin detection method, an electronic device, a computer readable storage medium and a computer program product. BACKGROUND

[0002] Facial skin detection is a research direction in the field of computer vision, and has wide applications in portrait beautification and face tracking. Using artificial intelligence (AI) technology to detect the state of facial skin has become a new trend.

[0003] In the related art, when facial skin detection is performed, whether a facial flaw exists and a position frame of the flaw can be detected. The position frame detected by the related art is generally a rectangular frame, the area of the rectangular frame is relatively large, and there are many non-flaw areas, and the accuracy is low.

[0004] Therefore, how to accurately perform facial flaw detection is a technical problem to be solved at present. SUMMARY

[0005] The present application provides a skin detection method, an electronic device, a computer readable storage medium and a computer program product to at least solve the problem of low accuracy of facial flaw detection in the related art. The technical solutions of the present application are as follows:

[0006] According to a first aspect of an embodiment of the present application, a skin detection method is provided, comprising:

[0007] obtaining a target image of a skin region to be detected; wherein the skin region to be detected comprises a facial skin region;

[0008] determining a position region of a flaw on the target image and a flaw category to which the flaw belongs;

[0009] removing a background skin region corresponding to the position region according to the flaw category, to obtain a flaw region corresponding to the position region;

[0010] determining a boundary line of the flaw according to pixel values of each pixel point in the flaw region.

[0011] Optionally, the removing the background skin region corresponding to the position region according to the flaw category, to obtain the flaw region corresponding to the position region, comprises:

[0012] converting a local image of a position region corresponding to the flaw of the flaw category into a target image of a color space corresponding to the flaw category according to the flaw category;

[0013] According to a color component of a target channel in the color space and a target screening threshold, a background skin region corresponding to the position region is filtered to obtain a corresponding flaw region; wherein the target channel is a color channel corresponding to the flaw category.

[0014] Optionally, the target screening threshold is determined according to the following steps:

[0015] For each flaw of a flaw category, a local image of a position region corresponding to the flaw is converted into a target image in a color space corresponding to the flaw category;

[0016] A color component of each pixel point in the target image in the target channel is extracted;

[0017] An average value of the color component corresponding to each pixel point in the target image is determined;

[0018] The average value is taken as a screening threshold of the flaw of the flaw category.

[0019] Optionally, the determination of the boundary line of the flaw region according to the pixel value of each pixel point in the flaw region comprises:

[0020] According to the pixel value of each pixel point in the flaw region, an outer contour line of a maximum connected region corresponding to the flaw region is determined;

[0021] The outer contour line of the maximum connected region is taken as the boundary line of the corresponding flaw.

[0022] Optionally, the determination of the position region of the flaw on the target image and the flaw category to which the flaw belongs comprises:

[0023] The position region of each flaw on the target image and the flaw category to which the flaw belongs are determined by a deep learning detection model.

[0024] Optionally, the method further comprises:

[0025] According to a pixel region size of a target region surrounded by the boundary line and / or according to a color difference value of the color component of each pixel point in the target region in the target channel and the color component of each pixel point in the surrounding region in the target channel, a flaw level of the flaw is determined.

[0026] Optionally, the color difference value is calculated by the following process, comprising:

[0027] determining a first color component mean value of each pixel point in the target region in the target channel according to the color component of each pixel point in the target region in the target channel; and determining a second color component mean value of each pixel point in the surrounding region in the target channel according to the color component of each pixel point in the surrounding region in the target channel;

[0028] determining a color difference value between the first color component mean value and the second color component mean value as the color difference value.

[0029] Optionally, the method further comprises:

[0030] determining a color difference value between the color component of each pixel point in the target region in the target channel and the color component of each pixel point in the surrounding region in the target channel, and / or determining a region size of the target region surrounded by the boundary line.

[0031] if the color difference value is smaller than or equal to a first target threshold value, and / or the region size is smaller than or equal to a second target threshold value, determining that the defect corresponding to the target region is a false detection defect, and deleting the false detection defect.

[0032] According to a second aspect of the embodiments of the present application, an electronic device is provided, comprising:

[0033] a processor;

[0034] a memory for storing instructions executable by the processor;

[0035] The processor is configured to execute the instructions to implement the skin detection method as described above.

[0036] According to a third aspect of the embodiments of the present application, a computer readable storage medium is provided, when instructions in the computer readable storage medium are executed by a processor of an electronic device, the electronic device can execute the skin detection method as described above.

[0037] According to a fourth aspect of the embodiments of the present application, a computer program product is provided, comprising a computer program or instructions, which, when executed by a processor, implement the skin detection method as described above.

[0038] The technical solutions provided by the embodiments of the present application at least bring the following beneficial effects:

[0039] In the embodiment of the present application, a target image of a skin region to be detected is acquired, wherein the skin region to be detected includes a facial skin region; a position region where a flaw on the target image is located and a flaw category to which the flaw belongs are determined; a background skin region corresponding to the position region is removed according to the flaw category, so as to obtain a flaw region corresponding to the position region; and a boundary line of the flaw is determined according to pixel values of each pixel point in the flaw region. That is, in the embodiment of the present application, the position region where the flaw on the target image is located and the category to which the flaw belongs are determined, the background skin region corresponding to the position region is removed according to the flaw category, so as to obtain the flaw region corresponding to the position region, and the boundary line of the flaw is determined according to the pixel values of each pixel point in the flaw region, that is, the preliminary position is further accurately processed through a series of subsequent operations, and some non-flaw regions are removed, so that the accuracy of the finally positioned flaw region is higher, that is, the accuracy of facial flaw detection is improved.

[0040] It should be understood that the general description above and the following detailed description are only exemplary and explanatory, and cannot limit the present application. BRIEF DESCRIPTION OF DRAWINGS

[0041] The accompanying drawings incorporated in and forming a part of the specification, illustrate embodiments consistent with the present application and, together with the description, serve to explain the principles of the application; they should not be regarded in an restrictive manner.

[0042] Figure 1 is a flowchart of a skin detection method according to an exemplary embodiment.

[0043] Figure 2A is an RGB original image of a pimple region according to an exemplary embodiment.

[0044] Figure 2B is an intensity image of the A component of the LAB space converted from the RGB original image of the pimple region according to an exemplary embodiment.

[0045] Figure 3 is another flowchart of a skin detection method according to an exemplary embodiment.

[0046] Figure 4 is a block diagram of a skin detection device according to an exemplary embodiment.

[0047] Figure 5 is a block diagram of a removing module according to an exemplary embodiment.

[0048] Figure 6 is a block diagram of a second determining module according to an exemplary embodiment.

[0049] Figure 7 is a block diagram of an electronic device according to an exemplary embodiment.

[0050] Figure 8 is a block diagram of an apparatus with skin detection according to an exemplary embodiment. DETAILED DESCRIPTION

[0051] In order to make the ordinary person skilled in the art better understand the technical solutions of the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings.

[0052] It should be noted that the terms "first", "second", and the like in the specification and claims of the present application and the above-described drawings are used to distinguish similar objects, and do not necessarily have to describe a specific order or sequence. It should be understood that the data thus used can be interchanged under appropriate circumstances, so that the embodiments of the present application described herein can be implemented in an order other than those illustrated or described herein. The embodiments described in the following exemplary embodiments do not represent all the embodiments consistent with the present application. Rather, they are merely examples of devices and methods consistent with some aspects of the present application as detailed in the appended claims.

[0053] In recent years, important progress has been made in the research of computer vision, deep learning, machine learning, image processing, image recognition and other technologies based on artificial intelligence. Artificial intelligence is a new science and technology that studies and develops theories, methods, technologies and application systems for simulating and extending human intelligence. Artificial intelligence is a comprehensive discipline involving chips, big data, cloud computing, the Internet of Things, distributed storage, deep learning, machine learning, neural networks and many other technology categories. Computer vision, as an important branch of artificial intelligence, is specifically to let the machine recognize the world. Computer vision technology usually includes face recognition, liveness detection, fingerprint recognition and anti-fraud verification, biometric recognition, face detection, pedestrian detection, object detection, pedestrian recognition, image processing, image recognition, image semantic understanding, image retrieval, character recognition, video processing, video content recognition, three-dimensional reconstruction, virtual reality, augmented reality, simultaneous localization and mapping (SLAM), computational photography, robot navigation and positioning, etc. With the research and progress of artificial intelligence technology, this technology has been applied in many fields, such as security and protection, city management, traffic management, building management, park management, face passage, face attendance, logistics management, warehouse management, robots, intelligent marketing, computational photography, mobile imaging, cloud services, smart home, wearable devices, unmanned vehicles, autonomous driving, intelligent medical treatment, face payment, face unlocking, fingerprint unlocking, face and certificate verification, smart screen, smart TV, camera, mobile Internet, network live broadcast, beauty, makeup, medical cosmetology, intelligent temperature measurement, etc.

[0054] Figure 1 This is a flowchart illustrating a skin detection method according to an exemplary embodiment, such as... Figure 1 As shown, this skin detection method is not limited to use on terminals or servers, and the method includes:

[0055] In step 101, a target image of the skin region to be detected is obtained; wherein, the skin region to be detected includes the facial skin region.

[0056] In this step, the skin area to be detected can include facial skin, but it can also include other skin areas, such as the skin area on the neck. In this embodiment, the target image is a facial image, which can be acquired through a terminal or acquisition device. Of course, the facial image in this embodiment can be a facial image of a user of any age and gender. Typically, to improve detection accuracy, a high-definition, unedited target image can be used, such as an RGB color image. RGB represents the three color channels: red (R), green (G), and blue (B). Various colors can be obtained by varying these three color channels and superimposing them. Each color can be visually synthesized using different proportions of red, green, and blue. It should be noted that each pixel in a color image is represented by different proportions of RGB; such an image is called an RGB image.

[0057] In step 102, the location region of the blemish on the target image and the blemish category to which the blemish belongs are determined.

[0058] The blemishes mentioned in step 102 can be any type of blemish such as pimples, freckles, or moles.

[0059] In step 103, the background skin area corresponding to the location area is removed according to the defect category to obtain the defect area corresponding to the location area.

[0060] In this step, after determining the location of the blemish, the background skin area within that area is removed, resulting in the blemish area corresponding to that location. In other words, removing some non-blemish areas surrounding the blemish narrows down the range of the identified blemish area, improving the accuracy of blemish location.

[0061] In step 104, the boundary line of the defect is determined based on the pixel value of each pixel in the defect area.

[0062] In the embodiment of the present application, the target image of the skin area to be detected is obtained, the position area and the category of the defect on the target image are determined, the background skin area corresponding to the position area is removed according to the defect category, the defect area corresponding to the position area is obtained, and the boundary line of the defect is determined according to the pixel value of each pixel point in the defect area. That is, the position of the preliminary obtained defect is accurately processed through a series of operations in the embodiment of the present application, some non-defect areas around the defect are removed, the accuracy of the finally positioned defect area is higher, and the accuracy of the facial defect detection is improved.

[0063] Optionally, in another embodiment, the position area of each defect on the target image and the defect category to which the defect belongs can be predicted by a deep learning detection model on the basis of the above-mentioned embodiment. The deep learning detection model in this embodiment is a trained model, and the training process includes:

[0064] 11) Labeling each category of defect and the position area of the defect on each facial image;

[0065] A plurality of sample images corresponding to the target skin area are obtained. For example, if the target skin area is a facial skin area, the obtained sample images are images containing a face. If the target skin area is a neck skin area, the obtained sample images are images containing a neck. Of course, for different target skin areas, the regions contained in the obtained sample images are different. After obtaining the sample images, the positions of the defects on the sample images are labeled with bounding boxes, such as the positions of pimples, freckles, moles, and closed pores. The bounding boxes cannot overlap with each other. In specific implementation, the positions of defects such as pimples, freckles, moles, and closed pores on each sample image can be labeled manually. For each sample image, the labeling area and the labeling category of each defect need to be labeled. The specific labeling process is a well-known technology in the field, and will not be described here.

[0066] 12) Constructing a training set according to the facial images of each category of defect and the position area of the defect after labeling; wherein each sample image in the training set has corresponding coordinates of the labeling area and the category of the bounding box;

[0067] In this step, after labeling, a training set S is constructed, each image x i in the training set S has corresponding labeling y i , y i represents a set of labeling areas and labeling categories, and y i = {b i1 , b i2 , b i3 , b i4, … b in} for any element b i in Y ij , b ij = (w1, h1, w2, h2, label), where w1, h1, w2, h2 represent the minimum value coordinates of the width, the minimum value coordinates of the height, the maximum value coordinates of the width, and the maximum value coordinates of the height of the labeled region in sequence, and label represents the category of the labeled region, such as acne, spots, moles, and closed pores.

[0068] 13) training a deep learning detection model using the training set, so that the deep learning detection model learns the mapping relationship between the position region of the defect on the input sample image and the defect category, and generates a trained deep learning detection model.

[0069] In this step, the data in the labeled training set is used to train a deep learning detection model, that is, the deep learning detection model learns a mapping relationship through training, where f represents the parameters of the deep learning detection model, and in this embodiment, the deep learning detection model can select Faster-rcnn, YOLOv5, etc. Through training, the deep learning detection model can automatically learn the association between the input image and various defect positions. That is, a high-definition sample picture is input into the trained deep learning detection model, and the deep learning detection model can output the position and category of the acne, spots, moles, and closed pores on the sample picture.

[0070] Therefore, in this embodiment, the position region of the facial defect and the category of the defect can be predicted by the trained deep learning detection model. Generally, the output of the deep learning detection model can be represented by a set p = {b1, b2, …, b i ,}, where b i = (x 1i , y 1i , x 2i , y 2i , l i ), where x 1i , y 1i , x 2i , y 2i , l i represent the minimum value coordinates of the horizontal axis, the minimum value coordinates of the vertical axis, the maximum value coordinates of the horizontal axis, the maximum value coordinates of the vertical axis, and the category coordinates of the labeled box, respectively.

[0071] In the embodiment, the trained deep learning detection model is used to predict the position region of each defect on the target image and the defect category to which the defect belongs, so that the position region of each defect on the target image and the category of the defect can be quickly identified, and the identification efficiency of the defect position is improved.

[0072] Optionally, in another embodiment, based on the above-mentioned embodiment, the removing the background skin region corresponding to the position region according to the defect category to obtain the defect region corresponding to the position region comprises: first converting the local image of the position region corresponding to the defect of the defect category into a target image of the color space corresponding to the defect category according to the defect category; then, filtering out the background skin region corresponding to the position region according to the color component of the target channel in the color space and the target screening threshold to obtain the corresponding defect region; wherein the target channel is the color channel corresponding to the defect category.

[0073] In this step, the traditional image processing algorithm can be used to convert each defect region of different categories into the corresponding color space of the defect, that is, the facial image (such as an RGB picture) is converted into a picture form of different color domains according to the color characteristics of different types of defects, and then the color components corresponding to various defects are extracted, so that the defect region (referred to as foreground region) and the corresponding background skin region (referred to as background region) can be effectively distinguished, and the background skin region and the defect region are distinguished more obviously.

[0074] In this embodiment, corresponding or similar processing is performed on acne, freckles, moles, and closed pores. Since different types of defects have different colors, for example, acne generally appears red, freckles generally appear yellow, moles generally appear black, and closed pores generally appear white, that is, the color appearance of acne, freckles, moles, and closed pores is quite different, and it is difficult to process through an RGB picture. Therefore, in this embodiment, the local image of the position region corresponding to the defect is converted into a target image of the color space corresponding to the defect category according to the color characteristics of the defect. Therefore, for each category of defect predicted, there is a corresponding position region. In general, the defect only occupies part of the position region, and the other part is occupied by the background skin. Since the color of the background skin is closer to the normal skin color, the visual effect is weaker than that of the defect region. For example, the normal skin around the acne is not as red as the acne, and the normal skin around the freckle is not as yellow as the freckle. According to the color characteristics of different defects, the RGB picture is converted into a picture form of different color domains (for example, the RGB picture of the mole is converted into a GRAY form, and the RGB picture of the acne is converted into a LAB form, etc.).

[0075] In an alternative embodiment, the target image can be used to determine the target screening threshold of the target image when performing skin detection based on the target image. Of course, other images of the same flaw type can also be used for determination.

[0076] In this embodiment, the background skin region corresponding to the position region can be filtered out by the target screening threshold, thereby obtaining the corresponding flaw region. As shown in Figure 2A and Figure 2B , Figure 2A is an RGB original image of a pimple region according to an exemplary embodiment, Figure 2B is an intensity map of the RGB original image of the pimple region converted into the A component of the LAB space according to an exemplary embodiment;

[0077] In this embodiment, assuming that the target image to be detected is an RGB picture, the following examples of moles and closures, pimples and color spots are used to illustrate, but in actual applications, it is not limited thereto.

[0078] Since moles and closures represent black and white components, the RGB picture can be converted into GRAY form. The darker the color, the greater the black component, and the lighter the color, the greater the white component. The color of the mole is obviously darker than the surrounding skin color, and the color of the closure is obviously whiter than the surrounding skin color.

[0079] Since pimples represent red components, the RGB picture can be converted into a corresponding color space, i.e. LAB form, wherein the A channel represents the red component of the pimple. The greater the red component value, the greater the red component of the pimple pixel point. It can be determined that the red component of the pimple region is obviously higher than the surrounding skin.

[0080] Since color spots represent yellow components, the RGB picture can also be converted into LAB form, wherein the B channel represents the yellow component of the color spot. The greater the yellow component value, the greater the yellow component of the color spot pixel point. It can be determined that the yellow component of the color spot region is obviously higher than the surrounding skin.

[0081] In this application, for each category of flaw region, the corresponding background skin region is filtered out by the pre-determined target screening threshold, thereby obtaining the corresponding flaw region.

[0082] In this step, after converting the image of the position region corresponding to different flaws into the color space corresponding to the flaw category, the corresponding background skin region needs to be filtered out by the pre-determined target screening threshold, thereby obtaining the corresponding flaw region.

[0083] In the embodiment, the target screening threshold K is used to distinguish the blemish area and the background skin area. Due to the influence of the shooting angle, exposure intensity, light condition, face shadow, skin color change and the like of each face image, even on the face image of the same face, the same blemish has slight difference in color depth and shade degree at different parts of the face (for example, the color spot is in the T zone and in the left cheek), so it is difficult to screen the blemish area by using the absolute screening threshold. Therefore, the relative screening threshold is needed to distinguish the blemish area and the background skin area. This is because even the same type of blemish on the same face has obvious difference in blemish color and surrounding skin color.

[0084] Optionally, in an optional embodiment, the target screening threshold can be set artificially according to experience or be determined based on the image of the corresponding blemish type. The target screening threshold specifically includes:

[0085] In the embodiment, the position area corresponding to different types of blemish is converted into the target image of the color space corresponding to the blemish category, then the color component of each pixel point in the target image in the target channel is extracted, and the average value of the color component corresponding to each pixel point in the target image is determined, and the average value is taken as the screening threshold of the blemish of the blemish category. In the embodiment, the corresponding background skin area is filtered by using the relative screening threshold, so that the corresponding blemish area is obtained. The accuracy of the face blemish area detection is improved, and the interference factors such as light, angle and skin color can be effectively overcome.

[0086] Optionally, in another embodiment, before the corresponding background skin area is filtered to obtain the corresponding blemish area, the screening threshold is determined in advance. The specific determination process includes:

[0087] For each blemish category, the local image of the position area corresponding to the blemish is converted into the target image of the color space corresponding to the blemish category, the color component of each pixel point in the target image in the target channel is extracted, the average value of the color component corresponding to each pixel point in the target image is determined, and the average value is taken as the screening threshold of the blemish of the blemish category. That is, the pictures of different categories of blemish areas are converted into the picture format of the corresponding color space, the color component of the corresponding blemish is extracted, the average value of the color component of each category of blemish is calculated, and finally the average value of each category of blemish is taken as the screening threshold of the corresponding category of blemish area.

[0088] For the convenience of understanding, the color spot is taken as an example for illustration in the embodiment.

[0089] First, the picture of the rectangular frame position (w1, h1, w2, h2) of the color spot output by the deep learning detection model is converted into LAB format, i.e., spot_area lab = RGB2LAB(spot_rect rgb ), spot_rect represents the local picture enclosed by the rectangular frame, and then the screening threshold K = mean(spot_rect lab [ :, :, 2]) is calculated, i.e., the target screening threshold. In this embodiment, through a large number of tests, the screening threshold K is set to the average value of the yellow component of the color spot area, which can effectively distinguish the defect area and the background skin area and has high robustness. By analogy, the screening threshold of the nevus and the closed mouth is the average value of the grayscale image, the screening threshold of the acne is the average value of the red component, etc.

[0090] Optionally, in another embodiment, which is based on the above-mentioned embodiment, the method further comprises:

[0091] determining the outer contour line of the maximum connected region corresponding to the defect region according to the pixel value of each pixel point in the defect region; and then taking the outer contour line of the maximum connected region as the boundary line of the corresponding defect.

[0092] In this step, first, after threshold screening, a binary image can be obtained for each defect region, and a maximum connected region algorithm is used to determine the largest connected region in each type of defect region. One maximum connected region algorithm includes: starting from any non-zero pixel point, judging whether the adjacent pixel points of the upper, lower, left, right and diagonal lines of the pixel point are non-zero, if non-zero, judging whether they belong to the same region as the current pixel point, taking these new pixel points as the boundary, repeating the above process until no new adjacent pixel points are generated, and after such processing, a binary image will generate several regions, and the region with the most pixel points is selected as the maximum connected region.

[0093] Secondly, the largest connected region in each type of defect region is taken as the outer contour line of the corresponding defect region.

[0094] In this embodiment, for each rectangular frame, only one maximum connected domain is retained, and other smaller connected domains and discrete points are removed, and the outer contour line of the maximum connected region is the accurate boundary line of the defect region, and the area of the maximum connected region is the area of the defect region.

[0095] In the embodiment of the present application, a target image of a skin region to be detected is obtained, wherein the skin region to be detected includes a facial skin region; a position region of a flaw on the target image and a flaw category to which the flaw belongs are determined; a background skin region corresponding to the position region is removed according to the flaw category, so as to obtain a flaw region corresponding to the position region; and a boundary line of the flaw is determined according to pixel values of each pixel point in the flaw region. That is, in the embodiment of the present application, the position region of the flaw on the target image and the category to which the flaw belongs are determined, the background skin region corresponding to the position region is removed according to the flaw category, so as to obtain the flaw region corresponding to the position region, and the boundary line of the flaw is determined according to the pixel values of each pixel point in the flaw region. By using the above technical solution provided in the embodiment, the outer contour line of the flaw region of different categories in the target image can be accurately obtained, and the accuracy of facial flaw detection is improved.

[0096] Further, in another embodiment, the method further includes: drawing the determined boundary lines of the flaws of different categories back to the facial image.

[0097] In this step, the boundary lines of the flaws of different categories obtained by the regression algorithm can be drawn back to the facial image. The boundary of the flaw position recognized by the region boundary regression is an irregular shape containing only the flaw. Therefore, the flaw contour line obtained by the regression in the embodiment of the present application is more accurate. It should be noted that the regression algorithm in the embodiment is a well-known technology for those skilled in the art, and will not be described here.

[0098] In the embodiment of the present application, after the boundary lines of the flaws of different categories are determined, the boundary lines of the flaws of different categories are drawn back to the facial image by the regression algorithm. The boundary information of the skin flaws is provided on the facial image, and more accurate reference is provided for skin care professionals.

[0099] Also see Figure 3 is another flowchart of a skin detection method according to an exemplary embodiment, the method comprising:

[0100] Step 401: obtaining a target image of a skin region to be detected, wherein the skin region to be detected includes a facial skin region;

[0101] Step 402: determining a position region of a flaw on the target image and a flaw category to which the flaw belongs;

[0102] Step 403: removing a background skin region corresponding to the position region according to the flaw category, so as to obtain a flaw region corresponding to the position region;

[0103] Step 404: determining the boundary line of the flaw according to the pixel value of each pixel point in the flaw region;

[0104] It should be noted that steps 401 to 404 are the same as steps 101 to 104, and the specific details are described above, which will not be repeated here.

[0105] Step 405: determining the flaw level of the flaw according to the area size of the target region surrounded by the boundary line and / or according to the color difference value of the color component of each pixel point in the target channel in the target region and the color component of each pixel point in the target channel in the surrounding region.

[0106] In this embodiment, one way is to determine the flaw level of the flaw by the area size of the region corresponding to the boundary line of the flaw region (which can be calculated by the size of the area of each pixel point in the region). The above-mentioned boundary line surrounded target region is larger, which means that the flaw is more serious. In this embodiment, the area size is determined based on the pixel point, that is, the number of pixel points in the region surrounded by the boundary line of the flaw region represents the area size.

[0107] Another way is to determine the flaw level of the flaw by the color difference value of the color component of each pixel point in the target channel in the target region and the color component of each pixel point in the target channel in the surrounding skin region. In this embodiment, the color difference value is calculated by the following process: determining the first color component mean value of each pixel point in the target channel in the target region according to the color component of each pixel point in the target channel in the target region; and determining the second color component mean value of each pixel point in the target channel in the surrounding region according to the color component of each pixel point in the target channel in the surrounding region; then, the difference between the first color component mean value and the second color component mean value is determined as the color difference value, and then the severity level can also be determined according to the size of the color difference value. The larger the color difference value is, the more serious the flaw is, and the higher the severity level is.

[0108] Specifically, the color difference value Δcolor = mean(blem_area X )-mean(skin_area X ) of the average color component of each kind of flaw region and the average color component of the surrounding skin region of the flaw region can be calculated, wherein blem_area X represents the value of the flaw region in the X component, and skin_area XThe value of the surrounding skin area in the X component (for the nevus, the X component represents the gray scale, and so on) can be used to determine the severity level of the defect according to the size of the difference value, and the severity level of each defect is pre-classified. It should be noted that for different types of defects such as acne, pigmentation, closed pores, and nevus, the numerical distribution of each color component is different, and the color difference of the gray scale is obviously larger than the red component and the yellow component of LAB, and the severity level of the defect needs to be determined according to the visual effect of the actual picture.

[0109] In this embodiment, the severity level of the defect can be determined by any of the above methods, and the severity level of different types of defects is pre-classified, for example, pigmentation is divided into three levels, light, heavy, and severe, and the severity level of the pigmentation can be determined by any of the above methods. Light, heavy, or severe. Of course, for some types of defects, the severity level needs to be determined by the above two methods in turn.

[0110] In an optional embodiment, the severity level of the defect can also be determined based on the area size and color difference of the target area surrounded by the boundary line; for example, when the area size reaches a certain size threshold and the color difference reaches a certain difference threshold, the severity level of the defect is determined to be severe.

[0111] Of course, in some optional embodiments, the classification can also be divided into levels such as level one, level two, and level three.

[0112] Optionally, in another embodiment, the method can further include:

[0113] First, determine the color difference between the color component of each pixel point in the target channel in the target area surrounded by the boundary line and the color component of each pixel point in the surrounding area; and / or, determine the area size of the target area surrounded by the boundary line; if the size of the color difference is less than or equal to a first target threshold, and / or the area size is less than or equal to a second target threshold, the defect corresponding to the target area is determined to be a false detection defect, and the false detection defect is deleted.

[0114] In specific implementation, the area size of the target area can be used to determine the false detection defect, or the color difference can be used to determine the false detection defect, or the area size and the color difference of the target area can be used to determine the false detection defect.

[0115] In practice, if Δcolor (color difference) is less than a certain target threshold, the blemish area can be considered to have no obvious visual features compared to the surrounding skin area, and therefore can be judged as a false detection, and the falsely detected blemish area can be removed. This corrects the detection result of the blemish area. In practical applications, the minimum target threshold for acne, pigmentation, closed comedones, and moles in the red component, yellow component, and grayscale image can be set to 2, 10, 15, and 20. It should be noted that the target threshold setting is related to the image acquisition method and the image pixel quality, and there is no single set of parameters that can be applied to all situations.

[0116] In this embodiment, by calculating the difference between the defective area and the background skin in a specific color component, the severity level of the defect can be determined accordingly. At the same time, defective areas with a difference less than a certain target threshold can be removed, making the final defect detection result more accurate.

[0117] It should be noted that, for the sake of simplicity, the method embodiments are all described as a series of actions. However, those skilled in the art should understand that this disclosure is not limited to the described order of actions, because according to this application, some steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should also understand that the embodiments described in the specification are all preferred embodiments, and the actions involved are not necessarily essential to this application.

[0118] Figure 4 This is a block diagram illustrating a skin detection device according to an exemplary embodiment. (Refer to...) Figure 4 The device includes an acquisition module 501, a first determination module 502, a removal module 503, and a second determination module 504, wherein...

[0119] The acquisition module 501 is used to acquire a target image of a skin region to be detected; wherein, the skin region to be detected includes a facial skin region;

[0120] The first determining module 502 is used to determine the location area of ​​the blemish on the target image and the blemish category to which the blemish belongs;

[0121] The removal module 503 is used to remove the background skin area corresponding to the location area according to the blemish category, so as to obtain the blemish area corresponding to the location area;

[0122] The second determining module 504 is used to determine the boundary line of the defect based on the pixel value of each pixel in the defect area.

[0123] Optionally, in another embodiment, based on the above embodiment, the removal module 503 includes: a conversion module 601 and a filtering module 602, the structural diagram of which is shown below.Figure 5 As shown in the figure,

[0124] The conversion module 601 is configured to convert a local image of a position region corresponding to a flaw of a flaw category into a target image in a color space corresponding to the flaw category according to the flaw category.

[0125] The filtering module 602 is configured to filter out a background skin region corresponding to the position region to obtain a corresponding flaw region according to a color component of a target channel in the color space and a target screening threshold value, wherein the target channel is a color channel corresponding to the flaw category.

[0126] Optionally, in another embodiment, the embodiment is based on the above-mentioned embodiment, and the device further comprises a third determination module, specifically comprising a space conversion module, an extraction module, a mean value determination module and a screening threshold value determination module, wherein,

[0127] The space conversion module is configured to convert a local image of a position region corresponding to a flaw of each flaw category into a target image in a color space corresponding to the flaw category.

[0128] The extraction module is configured to extract a color component of each pixel point in the target image in the target channel.

[0129] The mean value determination module is configured to determine an average value of the color component corresponding to each pixel point in the target image.

[0130] The screening threshold value determination module is configured to take the average value as a screening threshold value of the flaw of the flaw category.

[0131] Optionally, in another embodiment, the embodiment is based on the above-mentioned embodiment, and the second determination module 504 comprises an outer contour line determination module 701 and a boundary line determination module 702, and a structural schematic diagram is as shown in the figure, Figure 6 As shown in the figure,

[0132] The outer contour line determination module 701 is configured to determine an outer contour line of a maximum connected region corresponding to the flaw region according to a pixel value of each pixel point in the flaw region.

[0133] The boundary line determination module 702 is configured to take the outer contour line of the maximum connected region as a boundary line of the corresponding flaw.

[0134] Optionally, in another embodiment, the embodiment is based on the above-mentioned embodiment, and the first determination module is specifically configured to determine the position region of each flaw on the target image and the flaw category to which the flaw belongs by a deep learning detection model.

[0135] Optionally, in another embodiment, which is based on the above-mentioned embodiments, the fourth determining module is configured to determine the flaw level of the flaw according to a size of the target region surrounded by the boundary line and / or according to a color difference value between the color component of each pixel in the target region in a target channel and the color component of each pixel in the surrounding region in the target channel.

[0136] Optionally, in another embodiment, which is based on the above-mentioned embodiments, the device further comprises a fifth determining module, which specifically comprises:

[0137] a component determining module configured to determine a first color component mean value of each pixel in the target region in the target channel according to the color component of each pixel in the target region in the target channel, and determine a second color component mean value of each pixel in the surrounding region in the target channel according to the color component of each pixel in the surrounding region in the target channel;

[0138] a color difference value calculating module configured to determine the difference between the first color component mean value and the second color component mean value as the color difference value.

[0139] Optionally, in another embodiment, which is based on the above-mentioned embodiments, the device further comprises:

[0140] a sixth determining module configured to determine the color difference value between the color component of each pixel in the target region in the target channel and the color component of each pixel in the surrounding region in the target channel, and / or determine the size of the target region surrounded by the boundary line.

[0141] a flaw deleting module configured to determine that the flaw corresponding to the target region is a false detection flaw and delete the false detection flaw if the size of the color difference value is less than or equal to a first target threshold value and / or the size of the target region is less than or equal to a second target threshold value.

[0142] The skin detection device provided in the embodiments of the present application can implement all the method steps of the above-mentioned skin detection method, and thus the specific implementation functions of each module included in the device can refer to the above-mentioned method embodiments, which will not be described here again.

[0143] The embodiments of the present application also provide an electronic device, which comprises:

[0144] a processor;

[0145] a memory for storing instructions executable by the processor;

[0146] wherein the processor is configured to execute the instructions to implement the skin detection method as described above.

[0147] The embodiment of the present application further provides a computer readable storage medium, when instructions in the computer readable storage medium are executed by a processor of an electronic device, the electronic device can execute the skin detection method as described above.

[0148] The embodiment of the present application further provides a computer program product, comprising a computer program or instructions, when the computer program or instructions are executed by a processor, the skin detection method as described above is realized.

[0149] As to the apparatus in the above embodiment, the specific manners in which various modules perform operations have been described in detail in the embodiment of the method, and thus will not be described in detail here.

[0150] In the exemplary embodiment, a computer readable storage medium comprising instructions, for example, a memory comprising instructions, is further provided, and the above method can be executed by a processor of an apparatus to complete the above method. Optionally, the storage medium can be a non-transitory computer readable storage medium, for example, the non-transitory computer readable storage medium can be a ROM, a random access memory (RAM), a CD-ROM, a magnetic tape, a floppy disk, and an optical data storage device, etc.

[0151] Figure 7 is a block diagram of an electronic device 800 according to an exemplary embodiment. For example, the electronic device 800 can be a mobile terminal or a server, and the embodiment of the present application takes the electronic device as a mobile terminal for example. For example, the electronic device 800 can be a mobile phone, a computer, a digital broadcast terminal, a messaging device, a game console, a tablet device, a medical device, a fitness device, a personal digital assistant, etc.

[0152] Referring to Figure 7 , the electronic device 800 can include one or more of the following components: a processing component 802, a memory 804, a power supply component 806, a multimedia component 808, an audio component 810, an input / output (I / O) interface 812, a sensor component 814, and a communication component 816.

[0153] The processing component 802 usually controls overall operations of the electronic device 800, such as operations associated with displaying, making phone calls, data communications, camera operations, and recording operations. The processing component 802 can include one or more processors 820 to execute instructions to complete all or part of steps of the above method. Further, the processing component 802 can include one or more modules to facilitate the interaction between the processing component 802 and other components. For example, the processing component 802 can include a multimedia module to facilitate the interaction between the multimedia component 808 and the processing component 802.

[0154] The memory 804 is configured to store various types of data to support the operation of the electronic device 800. Examples of such data include instructions for any application or method operating on the electronic device 800, contact data, phonebook data, messages, pictures, videos, etc. The memory 804 can be implemented by any type of volatile or nonvolatile memory, or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read only memory (EEPROM), erasable programmable read only memory (EPROM), programmable read only memory (PROM), read only memory (ROM), magnetic memory, flash memory, magnetic disc or optical disc.

[0155] The power supply component 806 supplies power for various components of the electronic device 800. The power supply component 806 can include a power management system, one or more power supplies, and other components associated with generating, managing, and distributing power for the electronic device 800.

[0156] The multimedia component 808 includes a screen providing an output interface between the electronic device 800 and a user. In some embodiments, the screen can include a liquid crystal display (LCD) and a touch panel (TP). If the screen includes a touch panel, the screen can be implemented as a touch screen to receive an input signal from a user. The touch panel includes one or more touch sensors to sense a touch, a slide, and a gesture on the touch panel. The touch sensor can not only sense a boundary of a touching or a sliding action, but also detect duration and pressure related to the touching or sliding action. In some embodiments, the multimedia component 808 includes a front camera and / or a back camera. The front camera and / or the back camera can receive external multimedia data when the device 800 is in an operating mode, such as a shooting mode or a video mode. Each of the front and back cameras can be a fixed optical lens system or have a focal length and optical zoom capability.

[0157] The audio component 810 is configured to output and / or input an audio signal. For example, the audio component 810 includes a microphone (MIC) configured to receive an external audio signal when the electronic device 800 is in an operating mode, such as a call mode, a recording mode, and a voice recognition mode. The received audio signal can be further stored in the memory 804 or transmitted via the communication component 816. In some embodiments, the audio component 810 also includes a speaker for outputting an audio signal.

[0158] The I / O interface 812 provides an interface between the processing component 802 and peripheral interface modules, which can be a keyboard, a click wheel, a button, etc. The buttons can include, but are not limited to, a home button, a volume button, a start button, and a lock button.

[0159] The sensor component 814 includes one or more sensors for providing various state assessments for the electronic device 800. For example, the sensor component 814 can detect an open / closed position of the device 800, relative positioning of components, such as a display and a keypad of the electronic device 800, a change in position of the electronic device 800 or a component of the electronic device 800, presence or absence of user contact with the electronic device 800, orientation or acceleration / deceleration / g-force and temperature of the electronic device 800. The sensor component 814 can include an optical sensor for detecting ambient light, a proximity sensor for detecting nearby objects without any physical touch, a CMOS or CCD image sensor for use in imaging applications, or an acceleration sensor, a gyroscope sensor, a magnetic sensor, a pressure sensor, or a temperature sensor in some embodiments.

[0160] The communication component 816 is configured to facilitate wired or wireless communication between the electronic device 800 and other devices. The electronic device 800 can access a wireless network based on a communication standard, such as WiFi, a cellular network (e.g., 2G, 3G, 4G, or 5G), or a combination thereof. In an example embodiment, the communication component 816 receives broadcast signals or broadcast-related information from an external broadcast management system via a broadcast channel. In an example embodiment, the communication component 816 further includes a Near Field Communication (NFC) module to facilitate short-range communication. For example, the NFC module can be implemented based on Radio Frequency Identification (RFID) techniques, infrared data association (IrDA) techniques, ultra-wideband (UWB) techniques, Bluetooth (BT) techniques, and other techniques.

[0161] In an example embodiment, the electronic device 800 can be implemented using one or more application-specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field programmable gate arrays (FPGAs), controllers, micro-controllers, microprocessors, or other electronic elements to perform the above-described skin detection method.

[0162] In an example embodiment, a non-transitory computer-readable storage medium including instructions, such as the memory 804 including instructions, is also provided, which can be executed by the processor 820 of the electronic device 800 to implement the above-described skin detection method. For example, the non-transitory computer-readable storage medium can be a ROM, a random access memory (RAM), a CD-ROM, a magnetic tape, a floppy disc, and an optical data storage device, etc.

[0163] In an example embodiment, a computer program product is also provided, which when the instructions in the computer program product are executed by the processor 820 of the electronic device 800, causes the electronic device 800 to perform the above-described illustrated skin detection method.

[0164] Figure 8 is a block diagram of an apparatus 900 for skin detection according to an example embodiment. For example, the apparatus 900 can be provided as a server. Referring to Figure 8 , the apparatus 900 includes a processing component 922, which further includes one or more processors, and a memory resource represented by the memory 932 for storing instructions, such as an application program, executable by the processing component 922. The application program stored in the memory 932 can include one or more than one module each corresponding to a set of instructions. In addition, the processing component 922 is configured to execute the instructions to perform the above-described method.

[0165] The apparatus 900 can also include a power supply component 926 configured to perform power management of the apparatus 900, a wired or wireless network interface 950 configured to connect the apparatus 900 to a network, and an input output (I / O) interface 958. The apparatus 900 can operate based on an operating system stored in the memory 932, such as Windows Server™, Mac OS X™, Unix™, Linux™, FreeBSD™, or the like.

[0166] Other embodiments of the application will be apparent to those skilled in the art from consideration of the specification and practice of the application disclosed herein. It is intended that the specification and examples be considered as exemplary only, with the true scope and spirit of the application being indicated by the following claims.

[0167] It is to be understood that the application is not limited to the precise construction described in the specification and shown in the drawings, and that various modifications and changes can be made by those skilled in the art without departing from the scope of the application. The scope of the application is to be defined by the claims appended hereto.

Claims

1. A skin detection method characterized by, The method comprises: acquiring a target image of a skin region to be detected, wherein the skin region to be detected comprises a facial skin region; determining a position region of a flaw on the target image and a flaw category to which the flaw belongs, wherein the flaw category comprises acne, a color spot, a mole, and a closed pore; removing a background skin region corresponding to the position region according to the flaw category to obtain a flaw region corresponding to the position region, comprising: converting a local image of the position region corresponding to the flaw of the flaw category into a target image of a color space corresponding to the flaw category according to the flaw category; filtering out the background skin region corresponding to the position region according to a color component of a target channel in the color space and a target screening threshold to obtain a corresponding flaw region, wherein the target channel is a color channel corresponding to the flaw category; determining a boundary line of the flaw according to pixel values of each pixel point in the flaw region; wherein the target screening threshold is determined based on an image of a corresponding flaw category, and the target screening threshold is determined according to the following steps: for each flaw of a flaw category, converting a local image of a position region corresponding to the flaw into a target image of a color space corresponding to the flaw category; extracting a color component of each pixel point in the target image in the target channel; determining an average value of the color component corresponding to each pixel point in the target image; taking the average value as a screening threshold of the flaw of the flaw category.

2. The skin detection method of claim 1, wherein, The determination of the boundary line of the flaw region according to the pixel values of each pixel point in the flaw region comprises: determining an outer contour line of a maximum connected region corresponding to the flaw region according to the pixel values of each pixel point in the flaw region; taking the outer contour line of the maximum connected region as the boundary line of the corresponding flaw.

3. The skin detection method according to any one of claims 1 to 2, characterized in that, The determination of the position region of the flaw on the target image and the flaw category to which the flaw belongs comprises: determining the position region of each flaw on the target image and the flaw category to which the flaw belongs by a deep learning detection model.

4. The skin detection method according to any one of claims 1 to 2, characterized in that, The method further comprises: determining a flaw level of the flaw according to a region size of a target region surrounded by the boundary line and / or according to a color difference value of a color component of each pixel point in the target region in a target channel and a color component of each pixel point in a surrounding region in the target channel.

5. The skin detection method of claim 4, wherein, The color difference value is calculated by the following process, comprising: determining a first color component average value of each pixel point in the target region in the target channel according to the color component of each pixel point in the target region in the target channel, and determining a second color component average value of each pixel point in the surrounding region in the target channel according to the color component of each pixel point in the surrounding region in the target channel; determining a difference value between the first color component average value and the second color component average value as the color difference value.

6. The skin detection method according to any one of claims 1 to 2, characterized in that, The method further comprises: determining the color difference value of the color component of each pixel point in the target region in the target channel and the color component of each pixel point in the surrounding region in the target channel, and / or determining the region size of the target region surrounded by the boundary line. If the color difference value is less than or equal to a first target threshold value, and / or the region size is less than or equal to a second target threshold value, it is determined that the defect corresponding to the target region is a false detection defect, and the false detection defect is deleted.

7. An electronic device, comprising: Comprise: a processor; a memory for storing instructions executable by the processor; wherein the processor is configured to execute the instructions to implement the skin detection method of any one of claims 1 to 6.

8. A computer-readable storage medium, characterized in that, When the instructions in the computer readable storage medium are executed by the processor of the electronic device, the electronic device is enabled to perform the skin detection method of any one of claims 1 to 6.

9. A computer program product comprising computer programs or instructions, characterized in that, The computer program or instructions, when executed by the processor, implement the skin detection method of any one of claims 1 to 6.

Images