A method and related device for detecting skin color

Abstract

The embodiment of the present invention relates to the field of face detection technology, and in particular to a method and related device for detecting skin color, which first extracts a facial skin area in an image to obtain a facial skin area image, and then projects the facial skin area image into a preset three-dimensional color space to obtain a facial skin three-dimensional color space model, so that the color distinction of each pixel in the facial skin three-dimensional color space model is higher than the color distinction of each pixel in the facial skin area image, which is conducive to the subsequent acquisition of multiple valid pixel points corresponding to the skin area under normal light according to the facial skin three-dimensional color space model, and finally, the skin color of the face is determined according to the multiple valid skin pixel points. That is, the method is to screen out multiple valid skin pixel points that can truly reflect the skin color under normal light to determine the skin color of the face, which can effectively eliminate the influence of environmental factors such as uneven illumination on the skin color, so that the detected skin color is more accurate.

Description

Technical Field

[0001] The embodiments of the present invention relate to the technical field of face detection, and in particular to a method for detecting skin color and a related device. Background Art

[0002] With the rapid development of mobile communication technology and the improvement of people's living standards, various smart terminals have been widely used in people's daily work and life, making people more and more accustomed to using APP and other software, resulting in more and more APPs with functions such as beauty selfie and photo skin test. Therefore, people's demand for personal image design is increasing rapidly, such as quickly and accurately obtaining the user's facial skin color, selecting the appropriate foundation color, makeup and accessories for the user, etc.

[0003] Currently, existing facial skin color detection algorithms mainly determine facial skin color by matching the color of the face with the color of a skin color template. This method is difficult to adapt to different environments and will be affected by factors such as light and brightness, resulting in low accuracy in facial skin color detection. Summary of the invention

[0004] The main technical problem solved by the embodiment of the present invention is to provide a method for detecting skin color, which can accurately detect the skin color of a face.

[0005] In order to solve the above technical problems, in a first aspect, an embodiment of the present invention provides a method for detecting skin color, comprising:

[0006] Acquire an image including a face;

[0007] Extracting a facial skin region according to the image to obtain a facial skin region image;

[0008] Projecting the facial skin region image into a preset three-dimensional color space to obtain a three-dimensional color space model of the facial skin, wherein the color distinction of each pixel point in the three-dimensional color space model of the facial skin is higher than the color distinction of each pixel point in the facial skin region image;

[0009] According to the facial skin three-dimensional color space model, a plurality of valid skin pixel points are obtained, wherein the valid skin pixel points are pixel points corresponding to the facial skin area under normal light;

[0010] The skin color of the face is determined according to the multiple valid skin pixels.

[0011] In some embodiments, the step of acquiring a plurality of valid skin pixels according to the facial skin three-dimensional color space model includes:

[0012] Using a preset clustering algorithm to learn each pixel point in the three-dimensional color space model of the facial skin, and obtaining a target cluster center and a target cluster radius, wherein the target cluster center is the center of the target cluster where the multiple valid skin pixel points are located, and the target cluster radius is the radius of the target cluster where the multiple valid skin pixel points are located;

[0013] The multiple valid skin pixel points are obtained according to the target cluster center and the target cluster radius.

[0014] In some embodiments, the learning of each pixel point in the three-dimensional color space model of facial skin using a preset clustering algorithm to obtain a target cluster center and a target cluster radius includes:

[0015] Calculating the distance between each pixel point in the three-dimensional color space model of the facial skin and each current cluster center of a plurality of current cluster centers, wherein the number of the current cluster centers is equal to the number of preset skin color categories;

[0016] According to the distance, determine the minimum clustering loss that meets the preset constraints when the facial skin three-dimensional color space model takes the current clustering centers as clustering centers, and calculate multiple new clustering centers based on the minimum clustering loss, wherein the clustering loss of the facial skin three-dimensional color space model is the sum of membership loss and variance loss; the membership loss is equal to the sum of the membership degrees corresponding to the current clustering centers, the membership degree corresponding to a current clustering center is equal to the sum of the products corresponding to the pixels, the product corresponding to a pixel is the probability that the pixel belongs to the current clustering center, and the product of the distance between the pixel and the current clustering center; the variance loss is the sum of the variances corresponding to the current clustering centers, and a variance is the variance of each pixel belonging to a current clustering center;

[0017] Updating the multiple current cluster centers to the multiple new cluster centers, returning to the step of calculating the distance between each pixel point in the three-dimensional color space model of facial skin and each current cluster center, until the minimum cluster loss is less than a preset threshold, thereby obtaining multiple cluster centers of the three-dimensional color space model of facial skin;

[0018] The target cluster center and the target cluster radius are determined according to the multiple cluster centers and the facial skin three-dimensional color space model.

[0019] In some embodiments, determining the target cluster center and the target cluster radius according to the multiple cluster centers and the facial skin three-dimensional color space model includes:

[0020] According to the first cluster center and the distribution of pixels in the three-dimensional color space model of facial skin, the mean value of each pixel belonging to the first category is obtained, wherein the first cluster center is any cluster center among the multiple cluster centers, the first category is a preset skin color category represented by the first cluster center, and each pixel belonging to the first category is a pixel point in the three-dimensional color space model of facial skin that is within a range defined by a sphere radius with the first cluster center as the center of the sphere and a first cluster radius corresponding to the first cluster center as the radius of the sphere;

[0021] Determine the variance of each pixel belonging to the first category according to the mean;

[0022] Determine the target cluster center according to the mean corresponding to each cluster center and the variance corresponding to each cluster center;

[0023] The target cluster radius is determined according to the target cluster center and the facial skin three-dimensional color space model.

[0024] In some embodiments, acquiring the plurality of valid skin pixels according to the target cluster center and the target cluster radius includes:

[0025] Among the pixel points in the three-dimensional color space model of the facial skin, the pixel points within a range determined by a sphere centering on the target cluster center and a sphere radius being a preset distance are determined as the multiple valid skin pixel points, wherein the preset distance is the target cluster radius multiplied by a preset percentage.

[0026] In some embodiments, extracting the facial skin area according to the image to obtain the facial skin area image includes:

[0027] Acquire a non-facial skin area in the image according to a facial key point algorithm;

[0028] The pixel values ​​of the pixels corresponding to the non-facial skin area are replaced with preset pixel values ​​to obtain the facial skin area image.

[0029] In some embodiments, before acquiring a plurality of valid skin pixels according to the facial skin three-dimensional color space model, the method further includes:

[0030] Counting the probability of occurrence of the pixel value of each pixel point in the three-dimensional color space model of facial skin;

[0031] Remove the pixel points corresponding to the pixel values ​​whose probability is less than the preset probability threshold.

[0032] In some embodiments, determining the skin color of the face according to the plurality of valid skin pixels includes:

[0033] Calculating the sum of the Euclidean distances between the pixel values ​​of the plurality of valid skin pixels and the pixel values ​​of a target preset skin color category, wherein the target preset skin color category is any one of the preset skin color categories;

[0034] The preset skin color category corresponding to the minimum sum of Euclidean distances is determined as the skin color of the human face.

[0035] In order to solve the above technical problems, in a second aspect, an embodiment of the present invention provides an electronic device, including:

[0036] at least one processor, and

[0037] a memory communicatively coupled to the at least one processor, wherein:

[0038] The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the method described in the first aspect above.

[0039] To solve the above technical problems, in a third aspect, an embodiment of the present invention provides a non-transitory computer-readable storage medium, wherein the non-transitory computer-readable storage medium stores computer-executable instructions, and the computer-executable instructions are used to enable an electronic device to execute the method described in the first aspect above.

[0040] Beneficial effects of the embodiments of the present invention: Different from the prior art, the method for detecting skin color provided by the embodiments of the present invention first extracts the facial skin area in the image to obtain the facial skin area image, and then projects the facial skin area image into a preset three-dimensional color space to obtain a facial skin three-dimensional color space model, so that the color distinction of each pixel in the facial skin three-dimensional color space model is higher than the color distinction of each pixel in the facial skin area image, thereby facilitating the distinction of pixels under different light conditions, and facilitating the subsequent acquisition of multiple valid pixel points corresponding to the skin area under normal light according to the facial skin three-dimensional color space model, and finally, determining the skin color of the face according to the multiple valid skin pixel points. That is, the method is to screen out multiple valid skin pixel points that can truly reflect the skin color under normal light to determine the skin color of the face, which can effectively eliminate the influence of environmental factors such as uneven illumination on the skin color, so that the detected skin color is more accurate. BRIEF DESCRIPTION OF THE DRAWINGS

[0041] One or more embodiments are exemplarily described by pictures in the corresponding drawings, and these exemplified descriptions do not constitute limitations on the embodiments. Elements with the same reference numerals in the drawings represent similar elements, and unless otherwise stated, the figures in the drawings do not constitute proportional limitations.

[0042] Figure 1 A schematic diagram of an operating environment of a method for detecting skin color provided by an embodiment of the present invention;

[0043] Figure 2 A schematic diagram of the structure of an electronic device provided by an embodiment of the present invention;

[0044] Figure 3 A schematic diagram of a process for detecting skin color provided by an embodiment of the present invention;

[0045] Figure 4 for Figure 3 A schematic diagram of a sub-process of step S22 in the method shown;

[0046] Figure 5 A facial skin area image provided by an embodiment of the present invention;

[0047] Figure 6 for Figure 3 A schematic diagram of a sub-process of step S24 in the method shown;

[0048] Figure 7 for Figure 6 A schematic diagram of a sub-process of step S241 in the method shown;

[0049] Figure 8 for Figure 7 A schematic diagram of a sub-process of step S2414 in the method shown;

[0050] Fig. 9 A schematic diagram of a process for detecting skin color provided by an embodiment of the present invention;

[0051] Fig.10 for Figure 3 A schematic diagram of a sub-process of step S25 in the method shown;

[0052] Fig.11 A schematic diagram of preset skin color categories provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0053] The present invention is described in detail below in conjunction with specific embodiments. The following embodiments will help those skilled in the art to further understand the present invention, but are not intended to limit the present invention in any form. It should be noted that, for those of ordinary skill in the art, several variations and improvements may be made without departing from the concept of the present invention. These all belong to the protection scope of the present invention.

[0054] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0055] It should be noted that, if there is no conflict, the various features in the embodiments of the present invention can be combined with each other, all within the scope of protection of the present application. In addition, although the functional modules are divided in the device schematic diagram and the logical order is shown in the flow chart, in some cases, the steps shown or described can be performed in a different order than the module division in the device or the flow chart. In addition, the words "first", "second", "third", etc. used herein do not limit the data and execution order, but only distinguish the same items or similar items with basically the same functions and effects.

[0056] Unless otherwise defined, all technical and scientific terms used in this specification have the same meaning as those commonly understood by those skilled in the art of the present invention. The terms used in this specification and in the specification of the present invention are only for the purpose of describing specific embodiments and are not intended to limit the present invention. The term "and / or" used in this specification includes any and all combinations of one or more of the related listed items.

[0057] In addition, the technical features involved in the various embodiments of the present invention described below can be combined with each other as long as they do not conflict with each other.

[0058] Figure 1 Schematic diagram of the operating environment of the method for detecting skin color provided by an embodiment of the present invention. Figure 1 , including an electronic device 10 and an image acquisition device 20, wherein the electronic device 10 and the image acquisition device 20 are communicatively connected.

[0059] The communication connection can be a wired connection, such as a fiber optic cable, or a wireless communication connection, such as a WIFI connection, a Bluetooth connection, a 4G wireless communication connection, a 5G wireless communication connection, and the like.

[0060] The image acquisition device 20 is used to acquire an image including a human face. The image acquisition device 20 may be a terminal capable of capturing images, such as a mobile phone, tablet computer, video recorder or camera with a capturing function.

[0061] The electronic device 10 is a device that can run according to a program and automatically and quickly process massive amounts of data. It is usually composed of a hardware system and a software system, such as a computer, a smart phone, etc. The electronic device 10 can be a local device that is directly connected to the image acquisition device 20; it can also be a cloud device, such as a cloud server, a cloud host, a cloud service platform, a cloud computing platform, etc. The cloud device is connected to the image acquisition device 20 through a network, and the two are connected in communication through a predetermined communication protocol. In some embodiments, the communication protocol can be TCP / IP, NETBEUI, IPX / SPX, etc.

[0062] It is understandable that the image acquisition device 20 and the electronic device 10 may also be integrated together as an integrated device, such as a computer or smart phone with a camera.

[0063] The electronic device 10 receives the image including the face sent by the image acquisition device 20, and determines the skin color of the face.

[0064] In the above Figure 1 Based on this, another embodiment of the present invention provides an electronic device 10, see Figure 2 , is a hardware structure diagram of an electronic device 10 provided in an embodiment of the present invention, specifically, as Figure 2 As shown, the electronic device 10 includes at least one processor 11 and a memory 12 ( Figure 2 (a bus connection and a processor are used as an example).

[0065] The processor 11 is used to provide computing and control capabilities to control the electronic device 10 to perform corresponding tasks, for example, to control the electronic device 10 to perform any one of the methods for detecting skin color provided in the following embodiments of the invention.

[0066] It can be understood that the processor 11 can be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it can also be a digital signal processor (DSP), an application specific integrated circuit (ASIC), a field programmable gate array (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components.

[0067] The memory 12 is a non-transitory computer-readable storage medium, which can be used to store non-transitory software programs, non-transitory computer executable programs and modules, such as program instructions / modules corresponding to the method for detecting skin color in the embodiment of the present invention. The processor 11 can implement the method for detecting skin color in any of the following method embodiments by running the non-transitory software programs, instructions and modules stored in the memory 12. The memory 12 may include a high-speed random access memory, and may also include a non-transitory memory, such as at least one disk storage device, a flash memory device, or other non-transitory solid-state storage device. In some embodiments, the memory 12 may also include a memory remotely arranged relative to the processor, and these remote memories may be connected to the processor via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network and a combination thereof.

[0068] The following is a detailed description of the method for detecting skin color provided by an embodiment of the present invention. Figure 3 , the method S20 includes but is not limited to the following steps:

[0069] S21: Acquire an image including a face.

[0070] S22: Extracting the facial skin area according to the image to obtain a facial skin area image.

[0071] S23: Projecting the facial skin area image into a preset three-dimensional color space to obtain a three-dimensional color space model of the facial skin, wherein the color distinction of each pixel point in the three-dimensional color space model of the facial skin is higher than the color distinction of each pixel point in the facial skin area image.

[0072] S24: Acquire a plurality of valid skin pixel points according to the facial skin three-dimensional color space model, wherein the valid skin pixel points are pixel points corresponding to the facial skin area under normal light.

[0073] S25: Determine the skin color of the face according to the multiple valid skin pixels.

[0074] The image includes a face and can be acquired by the above-mentioned image acquisition device. For example, the image can be an ID photo or a selfie captured by the image acquisition device. It can be understood that the image includes a face and a background, wherein the pixels of the facial features and the background area of ​​the face will interfere with the subsequent determination of multiple valid skin pixels. Therefore, in order to obtain multiple valid skin pixels that can accurately reflect the skin color of the face and reduce the interference of the facial features and the background area, the image is subjected to facial feature recognition, and the facial skin area can be extracted to obtain a facial skin area image. It can be understood that the facial skin area image refers to an image that is not interfered by features such as facial features or background. That is, after determining the facial skin area, other areas in the image can be deleted or the pixels of this area can be processed to eliminate interference.

[0075] In some embodiments, see Figure 4 , the step S22 specifically includes:

[0076] S221: Acquire a non-facial skin area in the image according to a facial key point algorithm.

[0077] S222: Replace the pixel values ​​of the pixels corresponding to the non-facial skin area with preset pixel values ​​to obtain the facial skin area image.

[0078] According to the human face key point algorithm, several key points of the human face can be located, including points in the eyebrows, eyes, nose, mouth, facial contour and other areas. Therefore, according to these key points, the facial skin area and the non-facial skin area can be determined. Among them, the non-facial skin area is the facial features area and the background area.

[0079] For ease of processing, Figure 5 As shown, the pixel values ​​of the pixels corresponding to the non-facial skin area are replaced with preset pixel values ​​to obtain the facial skin area image. The preset pixel value is a pixel value that is artificially set and has a large difference from the pixel value of the skin color. For example, the preset pixel value can be 0, so that the color of the non-facial skin area in the facial skin area image can be clearly distinguished from the color of the facial skin area, and the boundary is clear, thereby reducing the interference with the facial skin color.

[0080] It is understandable that the pixel points corresponding to each skin area in the facial skin area image are all color values ​​that the facial skin may reflect in different scenes. Therefore, the color distinction of each pixel point in the facial skin area image is relatively small.

[0081] In order to increase the color distinction of each pixel, the facial skin area image is projected into a preset three-dimensional color space to obtain a three-dimensional color space model of the facial skin, wherein the color distinction of each pixel in the three-dimensional color space model of the facial skin is higher than the color distinction of each pixel in the facial skin area image.

[0082] It can be understood that the facial skin area image can be an RGB color space, and the facial skin area image is converted from the RGB color space to the preset three-dimensional color space, thereby obtaining a facial skin three-dimensional color space model with higher color distinction. In some embodiments, the preset three-dimensional color space can be a CrCbCg color space, and the three components of the CrCbCg color space can be calculated by the following formula.

[0083]

[0084] Based on the distribution characteristics of the pixels of the facial skin area in the facial skin three-dimensional color space model, a plurality of effective skin pixels can be obtained, and the effective skin pixels are the pixels corresponding to the facial skin area under normal light. Normal light refers to the light brightness that can expose the image normally, to distinguish it from weak light and strong light.

[0085] It is understandable that, ideally, the pixel values ​​of the pixels in the facial skin area should be roughly the same, and should be concentrated together in the facial skin three-dimensional color space. However, during the photo shooting process, due to the influence of light and angle, it is easy to have some skin areas with weak light and some skin areas with strong light. For example, the left face has weak light, the right face has normal light, and the forehead has strong light, which leads to uneven color of the facial skin area, which is reflected in the fact that the pixels will be distributed in multiple clusters in the three-dimensional color space. For example, the pixels corresponding to the left face with weak light are roughly concentrated together to form a cluster, the pixels corresponding to the forehead with strong light are roughly concentrated together to form a cluster, and the pixels corresponding to the right face with normal light are roughly concentrated together to form a cluster. It is worth noting that in the facial skin three-dimensional color space model, the distribution characteristics of each cluster are different. Therefore, according to the distribution characteristics of each cluster in the facial skin three-dimensional color space model, each pixel in the cluster corresponding to normal light can be determined and used as a valid skin pixel.

[0086] Finally, the skin color of the face is determined based on the multiple valid skin pixels. Since the multiple valid skin pixels are skin pixels under normal light, they can reflect the real skin color of the face, thus effectively eliminating the influence of environmental factors such as uneven lighting on the skin color, making the skin color detection result accurate.

[0087] In this embodiment, the method for detecting skin color first extracts the facial skin area in the image to obtain the facial skin area image, and then projects the facial skin area image into a preset three-dimensional color space to obtain a facial skin three-dimensional color space model, so that the color distinction of each pixel in the facial skin three-dimensional color space model is higher than the color distinction of each pixel in the facial skin area image, thereby facilitating the distinction of pixels under different light conditions, and facilitating the subsequent acquisition of multiple valid pixel points corresponding to the skin area under normal light according to the facial skin three-dimensional color space model, and finally, determining the skin color of the face according to the multiple valid skin pixel points. That is, the method is to screen out multiple valid skin pixel points that can truly reflect the skin color under normal light to determine the skin color of the face, which can effectively eliminate the influence of environmental factors such as uneven illumination on the skin color, so that the detected skin color is more accurate.

[0088] In some embodiments, see Figure 6 , the step S24 specifically includes:

[0089] S241: Using a preset clustering algorithm to learn each pixel point in the three-dimensional color space model of the facial skin, and obtaining a target cluster center and a target cluster radius, wherein the target cluster center is the center of the target cluster where the multiple valid skin pixel points are located, and the target cluster radius is the radius of the target cluster where the multiple valid skin pixel points are located.

[0090] S242: Acquire the plurality of valid skin pixel points according to the target cluster center and the target cluster radius.

[0091] Each pixel point in the three-dimensional color space model of the facial skin is equivalent to a learning sample. The learning sample is unsupervisedly learned by a preset clustering algorithm, and each pixel point in the learning sample is classified. In some embodiments, the preset clustering algorithm can be a K-means clustering algorithm, which defines the number of groups as K and performs automatic grouping. Specifically, (1) K centers of gravity are defined. In the initial stage, these centers of gravity are random; (2) each pixel point is assigned to one of the K centers of gravity to form a cluster, that is, each pixel point is assigned to the cluster corresponding to the center of gravity closest to itself; (3) the center of each cluster is obtained by calculating the average position of all pixels in each cluster, and the center of gravity is moved to the center of the cluster corresponding to itself, and the center of each cluster is used as the center of gravity. Repeat steps (2) and (3) until the position of the center of gravity no longer changes significantly in each iteration, that is, the preset clustering algorithm converges.

[0092] According to the distribution characteristics of the pixels in each cluster, a target cluster that can reflect the skin color under normal light is determined. The multiple valid skin pixels are the pixels in the target cluster. Thus, the target cluster center and the target cluster radius can be obtained according to the target cluster. The target cluster center is the average position of all the pixels in the target cluster, and the target cluster radius is the distance between the pixel points close to the outermost layer in the target cluster and the average position.

[0093] Thus, the multiple valid skin pixel points can be obtained according to the target cluster center and the target cluster radius. For example, a plurality of pixel points with the target cluster center as the center of the sphere and within the target cluster radius are selected as the multiple valid skin pixel points, so that the multiple valid skin pixel points can more accurately reflect the skin color under normal light.

[0094] In this embodiment, each pixel point in the three-dimensional color space model of the facial skin is learned using a preset clustering algorithm to generate various clusters, and based on the characteristics of each cluster, a target cluster center and a target cluster radius are obtained. Finally, based on the target cluster center and the target cluster radius, the multiple valid skin pixel points are obtained, so that the multiple valid skin pixel points can more accurately reflect the skin color under normal light.

[0095] In some embodiments, see Figure 7 , the step S241 specifically includes:

[0096] S2411: Calculating the distance between each pixel point in the three-dimensional color space model of the facial skin and each current cluster center of a plurality of current cluster centers, wherein the number of the current cluster centers is equal to the number of the preset skin color categories.

[0097] S2412: Determine, based on the distance, a minimum clustering loss that satisfies preset constraints when the facial skin three-dimensional color space model takes the current clustering centers as clustering centers, and calculate multiple new clustering centers based on the minimum clustering loss.

[0098] Among them, the clustering loss of the facial skin three-dimensional color space model is the sum of the membership loss and the variance loss; the membership loss is equal to the sum of the membership degrees corresponding to the current cluster centers, and the membership degree corresponding to a current cluster center is equal to the sum of the products corresponding to the pixels, and the product corresponding to a pixel is the probability that the pixel belongs to the current cluster center, and the product of the distance between the pixel and the current cluster center; the variance loss is the sum of the variances corresponding to the current cluster centers, and a variance is the variance of each pixel belonging to a current cluster center.

[0099] S2413: Update the multiple current cluster centers to the multiple new cluster centers, return to execute the step of calculating the distance between each pixel point in the facial skin three-dimensional color space model and each current cluster center, until the minimum clustering loss is less than a preset threshold, and obtain multiple cluster centers of the facial skin three-dimensional color space model.

[0100] S2414: Determine the target cluster center and the target cluster radius according to the multiple cluster centers and the facial skin three-dimensional color space model.

[0101] In this embodiment, if the number of the preset skin color categories is c, the number of the current cluster centers is also c, that is, there are c clusters in the model. For a facial skin area image of size M*N, the corresponding facial skin three-dimensional color space model has M*N pixels. The pixel x in the model is calculated. i (i=1,2,3,...,M*N) are respectively the distances from the current cluster center mj(j=1,2,...,c). This distance can reflect the degree of overlap between each current cluster center and each cluster. It can be understood that the greater the degree of overlap between each current cluster center and the center position of each cluster, the smaller the distance; the smaller the degree of overlap between each current cluster center and the center position of each cluster, the larger the distance.

[0102] Based on this, in this embodiment, the degree of overlap between each current cluster center and the center position of each cluster in the three-dimensional color space model of facial skin is evaluated by cluster loss. Specifically, the cluster loss is the sum of membership loss and variance loss. Among them, the membership loss is equal to the sum of the membership degrees corresponding to the current cluster centers, and the membership degree corresponding to a current cluster center is equal to the sum of the products corresponding to the pixels, and the product corresponding to a pixel point is the probability that the pixel point belongs to the current cluster center, and the product of the distance between the pixel point and the current cluster center. For example, if a pixel point x i The probability of belonging to the current cluster center is u j (x i ), where i = 1, 2, 3, ..., M*N, j = 1, 2, ..., c, then the product corresponding to the pixel point xi is (xi-mj)*u j (x i ), it can be understood that, in some embodiments, in order to avoid the positive and negative sign influence caused by the distance, (x i -m j )*u j (x i ) is transformed, for example, it is transformed into (x i -m j )2 *u j (x i ) or (x i -m j ) 2 *u j (x i ) b Etc., thus, the clustering loss Among them, σ j represents the variance of the jth cluster. That is, by calculating the probability of each pixel belonging to a current cluster center and the distance from the current cluster center, the membership degree of each pixel and each cluster is reflected, thereby reflecting the membership loss of each cluster in the entire facial skin three-dimensional color space model. It can be seen that calculating the membership loss from the two aspects of distance and membership probability makes the membership loss more accurate.

[0103] The variance loss is the sum of the variances corresponding to the current cluster centers, and one variance is the variance of each pixel belonging to a current cluster center. For example, for the j-th cluster, including W pixels, W<M*N, then the variance of the j-th cluster is the variance of these W pixels belonging to the j-th cluster center, which can be calculated by the following formula:

[0104]

[0105] Among them, δ j represents the variance of the jth cluster, t=1,2,3...W,x t j is the t-th pixel among the W pixels of the j-th cluster, is the sum of the probabilities that the overall pixel points belong to the current cluster centers, is the sum of the probabilities that the j-th pixel belongs to the j-th cluster center.

[0106] Therefore, the above clustering loss reflects the degree of overlap between the current cluster centers and the center positions of various clusters from the two aspects of membership loss and variance loss, which can enable the preset clustering algorithm to optimize learning in the direction of small membership loss and variance loss, which is conducive to improving the accuracy of each cluster center output by the preset clustering algorithm.

[0107] Based on the above clustering loss, when the current clustering centers are used as clustering centers, the distance between each pixel and each of the current clustering centers is combined, and the minimum clustering loss is calculated through preset constraints to obtain the minimum clustering loss, and multiple new clustering centers are calculated based on the minimum clustering loss. It can be understood that in the above clustering loss, each distance can be calculated, and the probability that the pixel belongs to the current clustering center is grasped as a whole. It can be understood that for the pixel point x i, the sum of the probabilities of belonging to each current cluster center is 1, so the sum of the probabilities of the M*N pixels in the three-dimensional color space model of the facial skin belonging to each current cluster center is M*N, that is, Thus, we can As a preset constraint condition of the clustering loss, the clustering loss is derived to obtain the derivative function L' of the clustering loss, and the minimum clustering loss is calculated based on the derivative function L'. It can be understood that when calculating the minimum clustering loss, the derivative function L'=0, so that multiple new clustering centers m can be calculated. j '.

[0108] Thus, the multiple current cluster centers m j Update to the multiple new cluster centers m j ', that is, the new cluster center m j 'Substitute the above clustering loss formula, recalculate the minimum clustering loss, and continuously iterate and update until the minimum clustering loss is less than the preset threshold, which means that the accuracy of each cluster center at this time meets the preset requirements, and each cluster center is close to the center position of each cluster to the greatest extent. Therefore, each cluster center at this time is used as the multiple cluster centers of the three-dimensional color space model of facial skin.

[0109] Furthermore, the target cluster center and the target cluster radius can be determined from the multiple cluster centers according to the distribution of the clusters to which the multiple cluster centers belong in the three-dimensional color space model of the facial skin. In some embodiments, a cluster with uniform distribution can be selected as the target cluster, and the center of the target cluster can be used as the target cluster center, and the radius of the target cluster can be used as the target cluster radius.

[0110] In some embodiments, see Figure 8 , the step S2414 specifically includes:

[0111] S24141: Obtain a mean value of each pixel belonging to the first category according to the first cluster center and the pixel distribution in the three-dimensional color space model of the facial skin.

[0112] Among them, the first cluster center is any cluster center among the multiple cluster centers, the first category is a preset skin color category represented by the first cluster center, and the pixel points belonging to the first category are pixel points in the three-dimensional color space model of facial skin, which are within a range limited by the first cluster center as the sphere center and the first cluster radius corresponding to the first cluster center as the sphere radius.

[0113] S24142: Determine the variance of each pixel point belonging to the first category according to the mean.

[0114] S24143: Determine the target cluster center according to the means corresponding to the cluster centers and the variances corresponding to the cluster centers.

[0115] S24144: Determine the target cluster radius according to the target cluster center and the facial skin three-dimensional color space model.

[0116] It is understandable that in the three-dimensional color space model of facial skin, the pixels in the target cluster corresponding to normal light are evenly distributed, while there are abnormal pixels affected by light in the clusters corresponding to strong or weak light, so that the pixels in the clusters corresponding to strong or weak light are unevenly distributed. In order to find the optimal cluster that can represent the skin color under normal light, the mean and variance of each pixel in each cluster are calculated, and the cluster with the smallest mean and variance is determined as the target cluster that can most truly reflect the skin color.

[0117] Specifically, after obtaining the multiple cluster centers, for any cluster center among the multiple cluster centers, that is, the first cluster center P, the preset skin color category represented by the first cluster center is the first category, that is, the preset skin color category reflected by the cluster to which the first cluster center belongs is the first category. It can be understood that if k clusters are obtained after clustering each pixel point in the three-dimensional color space model of the facial skin, the first category is the preset skin category reflected by any cluster among the K clusters. It can be understood that K is less than the number of preset skin color categories (k<c).

[0118] Thus, according to the first cluster center P and the pixel distribution in the three-dimensional color space model of the facial skin, each pixel belonging to the first category can be obtained, that is, the entire cluster to which the first cluster center P belongs can be obtained. Specifically, after locating the first cluster center P and the cluster to which it belongs, the pixel points within the range defined by the first cluster center as the sphere center and the first cluster radius corresponding to the first cluster center as the sphere radius can be obtained, that is, each pixel belonging to the first category can be obtained, and then the pixel values ​​of each pixel belonging to the first category are averaged to obtain the mean. Furthermore, according to the mean, the variance of each pixel belonging to the first category is determined.

[0119] Therefore, for the j-th cluster, the calculation formulas for its mean and variance are as follows:

[0120]

[0121]

[0122] Among them, S j represents the mean of the j-th cluster, μ jrepresents the variance of the jth cluster, K is the number of clusters obtained after clustering (k<c), j=1,2,...,k, x i j Represents the i-th pixel in the j-th cluster.

[0123] It is understandable that the above steps S24141-S24144 describe the calculation process of the mean and variance of each pixel point corresponding to any one of the K clusters (K cluster centers), and the calculation process of the mean and variance of each pixel point corresponding to each cluster is similar. It is understandable that based on the k clusters in the three-dimensional color space model of facial skin, K groups (mean, variance) can be obtained.

[0124] Therefore, the target cluster, that is, the target cluster center, can be determined according to the K group (mean, variance). After obtaining the target cluster and the target cluster center, the target cluster radius can be determined according to the distribution of the target cluster in the three-dimensional color space model of the facial skin. For example, the distance from the pixel point close to the outer layer of the target cluster to the target cluster center is used as the target cluster radius.

[0125] In this embodiment, based on the characteristics of each pixel under normal light, the cluster with the smallest mean and variance is used as the pixel sample that can truly reflect the skin color under normal light to determine the skin color of the face. This can effectively eliminate the impact of environmental factors such as uneven lighting on the skin color, making the determined skin color more accurate.

[0126] In some embodiments, step S242 specifically includes:

[0127] S2421: Among the pixel points in the three-dimensional color space model of the facial skin, the pixel points within a range determined by a sphere centered on the target cluster center and having a sphere radius of a preset distance are determined as the multiple valid skin pixel points, wherein the preset distance is the cluster radius multiplied by a preset percentage.

[0128] In order to make the effective skin pixels more accurate, the pixels in the outer layer of the target cluster are removed to avoid the inaccuracy of the outer layer pixels interfering with the effective skin pixels. Specifically, the pixels within the range determined by the target cluster center as the sphere center and the sphere radius as the preset distance are determined as the multiple effective skin pixels, wherein the preset distance is the cluster radius multiplied by the preset percentage. For example, the pixels within the range determined by the target cluster center as the sphere center and the sphere radius as 2 / 3 of the target cluster radius can be determined as the multiple effective skin pixels.

[0129] In this embodiment, the pixels in the target cluster are screened and the pixels in the outer layer of the target cluster are removed to avoid interference of the inaccuracy of the outer pixels on the valid skin pixels. This can improve the accuracy of the multiple valid skin pixels, thereby making the detected skin color more accurate.

[0130] In some embodiments, see Fig. 9 , before step S23, further comprising:

[0131] S26: Counting the probability of occurrence of the pixel value of each pixel point in the three-dimensional color space model of facial skin;

[0132] S27: Remove pixel points corresponding to pixel values ​​whose probabilities are less than a preset probability threshold.

[0133] In this embodiment, before obtaining the multiple valid pixel points, first, the distribution of pixel values ​​of the pixel points in the three-dimensional color space model of the facial skin is counted, that is, the probability of the occurrence of the pixel value is counted. If the probability of a certain pixel value is less than the preset probability threshold, it means that the pixel value is not common, and the pixel values ​​of the facial skin are generally close, and a region in the facial skin corresponds to multiple pixel values. Therefore, it can be explained that the pixel value with a probability less than the preset probability threshold is an abnormal value, so that the abnormal pixel points corresponding to the pixel value with a probability less than the preset probability threshold are removed, which can reduce the interference of the abnormal pixel points on the three-dimensional color space model of the facial skin, making the three-dimensional color space model of the facial skin cleaner.

[0134] In some embodiments, see Fig.10 , the step S25 specifically includes:

[0135] S251: Calculating the sum of the Euclidean distances between the pixel values ​​of the plurality of valid skin pixels and the pixel values ​​of a target preset skin color category, wherein the target preset skin color category is any one of the preset skin color categories;

[0136] S252: Determine the preset skin color category corresponding to the minimum sum of Euclidean distances as the skin color of the face.

[0137] For example, for N valid skin pixels a i (i=1,2,..,N), such as Fig.11 The 36 preset skin color categories shown are respectively calculated by summing the Euclidean distances between the N valid skin pixels and the target preset skin color category to obtain the corresponding Euclidean distance sum. Based on the target preset skin color category being any one of the 36 preset skin color categories, 36 Euclidean distance sums are obtained. Therefore, the preset skin color category corresponding to the smallest Euclidean distance sum among the 36 Euclidean distance sums is determined as the skin color of the face.

[0138] In this embodiment, the preset skin color category corresponding to the smallest sum of Euclidean distances is determined as the skin color of the face, so that the preset skin color category closest to the real skin color of the face is used as the face skin color, making the face skin color more accurate.

[0139] In summary, the method for detecting skin color provided by the embodiment of the present invention first extracts the facial skin area in the image to obtain the facial skin area image, and then projects the facial skin area image into a preset three-dimensional color space to obtain a facial skin three-dimensional color space model, so that the color distinction of each pixel in the facial skin three-dimensional color space model is higher than the color distinction of each pixel in the facial skin area image, thereby facilitating the distinction of pixels under different light conditions, and facilitating the subsequent acquisition of multiple valid pixel points corresponding to the skin area under normal light according to the facial skin three-dimensional color space model, and finally, determining the skin color of the face according to the multiple valid skin pixel points. That is, the method is to screen out multiple valid skin pixel points that can truly reflect the skin color under normal light to determine the skin color of the face, which can effectively eliminate the influence of environmental factors such as uneven illumination on the skin color, so that the detected skin color is more accurate.

[0140] One embodiment of the present invention further provides a non-transitory computer-readable storage medium, wherein the non-transitory computer-readable storage medium stores computer-executable instructions, and the computer-executable instructions are used to enable an electronic device to execute the method for detecting skin color in any of the above embodiments.

[0141] An embodiment of the present invention provides a computer program product, including a computer program stored on a non-volatile computer-readable storage medium, wherein the computer program includes program instructions. When the program instructions are executed by a computer, the computer executes the method for detecting skin color in any of the above embodiments.

[0142] It should be noted that the device embodiments described above are merely illustrative, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0143] Through the description of the above implementation methods, ordinary technicians in this field can clearly understand that each implementation method can be implemented by means of software plus a general hardware platform, and of course, it can also be implemented by hardware. Ordinary technicians in this field can understand that all or part of the processes in the above-mentioned embodiment method can be completed by instructing related hardware through a computer program, and the program can be stored in a computer-readable storage medium. When the program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, the storage medium can be a disk, an optical disk, a read-only memory (ROM) or a random access memory (RAM), etc.

[0144] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit them. Under the concept of the present invention, the technical features in the above embodiments or different embodiments may also be combined, the steps may be implemented in any order, and there are many other changes in different aspects of the present invention as described above, which are not provided in detail for the sake of simplicity. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they may still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for detecting skin color, characterized in that: include: Acquire an image including a face; Extracting a facial skin region according to the image to obtain a facial skin region image; The facial skin area image is projected into a preset three-dimensional color space to obtain a three-dimensional color space model of the facial skin, wherein the color distinction of each pixel in the three-dimensional color space model of the facial skin is higher than the color distinction of each pixel in the facial skin area image; according to the three-dimensional color space model of the facial skin, a plurality of valid skin pixel points are obtained, wherein the valid skin pixel points are pixel points corresponding to the facial skin area under normal light, and the normal light refers to the light intensity that can normally expose the image; according to the plurality of valid skin pixel points, the skin color of the face is determined; The step of obtaining a plurality of valid skin pixel points according to the facial skin three-dimensional color space model comprises: Using a preset clustering algorithm to learn each pixel point in the three-dimensional color space model of the facial skin, a target cluster center and a target cluster radius are obtained, wherein the target cluster center is the center of a target cluster where a plurality of valid skin pixel points are located, and the target cluster radius is the radius of the target cluster where the plurality of valid skin pixel points are located; Acquire the plurality of valid skin pixel points according to the target cluster center and the target cluster radius; The acquiring the plurality of valid skin pixel points according to the target cluster center and the target cluster radius comprises: Among the pixel points in the three-dimensional color space model of the facial skin, the pixel points within a range determined by a sphere centering on the target cluster center and a sphere radius being a preset distance are determined as the multiple valid skin pixel points, wherein the preset distance is the target cluster radius multiplied by a preset percentage.

2. The method according to claim 1, characterized in that The learning of each pixel point in the three-dimensional color space model of the facial skin using a preset clustering algorithm to obtain a target cluster center and a target cluster radius includes: Calculating the distance between each pixel point in the three-dimensional color space model of the facial skin and each current cluster center of a plurality of current cluster centers, wherein the number of the current cluster centers is equal to the number of preset skin color categories; According to the distance, determine the minimum clustering loss that meets the preset constraints when the facial skin three-dimensional color space model takes the current clustering centers as clustering centers, and calculate multiple new clustering centers based on the minimum clustering loss, wherein the clustering loss of the facial skin three-dimensional color space model is the sum of membership loss and variance loss; the membership loss is equal to the sum of the membership degrees corresponding to the current clustering centers, the membership degree corresponding to a current clustering center is equal to the sum of the products corresponding to the pixels, the product corresponding to a pixel is the probability that the pixel belongs to the current clustering center, and the product of the distance between the pixel and the current clustering center; the variance loss is the sum of the variances corresponding to the current clustering centers, and a variance is the variance of each pixel belonging to a current clustering center; Updating the multiple current cluster centers to the multiple new cluster centers, returning to the step of calculating the distance between each pixel point in the three-dimensional color space model of facial skin and each current cluster center, until the minimum cluster loss is less than a preset threshold, thereby obtaining multiple cluster centers of the three-dimensional color space model of facial skin; The target cluster center and the target cluster radius are determined according to the multiple cluster centers and the facial skin three-dimensional color space model.

3. The method according to claim 2, characterized in that Determining the target cluster center and the target cluster radius according to the multiple cluster centers and the facial skin three-dimensional color space model includes: According to the first cluster center and the distribution of pixels in the three-dimensional color space model of facial skin, the mean value of each pixel belonging to the first category is obtained, wherein the first cluster center is any cluster center among the multiple cluster centers, the first category is a preset skin color category represented by the first cluster center, and each pixel belonging to the first category is a pixel point in the three-dimensional color space model of facial skin that is within a range defined by a sphere radius with the first cluster center as the center of the sphere and a first cluster radius corresponding to the first cluster center as the radius of the sphere; Determine the variance of each pixel belonging to the first category according to the mean; Determine the target cluster center according to the mean corresponding to each cluster center and the variance corresponding to each cluster center; The target cluster radius is determined according to the target cluster center and the facial skin three-dimensional color space model.

4. The method according to any one of claims 1 to 3, characterized in that: Extracting the facial skin area according to the image to obtain a facial skin area image includes: Acquire a non-facial skin area in the image according to a facial key point algorithm; The pixel values ​​of the pixels corresponding to the non-facial skin area are replaced with preset pixel values ​​to obtain the facial skin area image.

5. The method according to any one of claims 1 to 3, characterized in that: Before acquiring a plurality of valid skin pixel points according to the facial skin three-dimensional color space model, the method further includes: Counting the probability of occurrence of the pixel value of each pixel point in the three-dimensional color space model of facial skin; Remove the pixel points corresponding to the pixel values ​​whose probability is less than the preset probability threshold.

6. The method according to any one of claims 1 to 3, characterized in that: The determining the skin color of the face according to the plurality of valid skin pixels comprises: Calculating the sum of the Euclidean distances between the pixel values ​​of the plurality of valid skin pixels and the pixel values ​​of a target preset skin color category, wherein the target preset skin color category is any one of the preset skin color categories; The preset skin color category corresponding to the minimum sum of Euclidean distances is determined as the skin color of the human face.

7. An electronic device, characterized in that: include: at least one processor, and a memory communicatively coupled to the at least one processor, wherein: The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the method according to any one of claims 1 to 6.

8. A non-transitory computer-readable storage medium, characterized in that: The non-transitory computer-readable storage medium stores computer-executable instructions, and the computer-executable instructions are used to enable an electronic device to execute the method according to any one of claims 1 to 6.

Images