A method, apparatus, device, and storage medium for makeup color recognition

Abstract

This application proposes a makeup color recognition method, apparatus, device, and storage medium. The method includes: acquiring a user's makeup image; identifying the color tone category of a preset part of the user's face in the makeup image; and determining the user's makeup color tone based on the color tone category corresponding to the preset part. This application extracts the color tone of at least one part of the user's face and automatically identifies the user's makeup color tone using the extracted color tone. By combining makeup elements such as skin tone, eyeshadow, and lipstick, it extracts only the color information of the parts that have a significant impact on the makeup color, excluding the influence of other makeup effects on the makeup color tone, thus increasing the uniformity and accuracy of makeup color tone recognition. Moreover, it can accurately identify the user's makeup color tone through image processing alone, without the need for deep learning models, resulting in low computational load, low cost, reduced server processing pressure, and reduced algorithm dependence on hardware resources and human investment costs.

Description

Technical Field

[0001] This application belongs to the field of image processing technology, specifically relating to a makeup color recognition method, apparatus, device, and storage medium. Background Technology

[0002] Makeup has become an essential part of many people's daily lives. Eyeshadow, blush, and lipstick, among other color cosmetics, play a crucial role in shaping the overall color scheme of the face. In fields like virtual makeup and beauty, it's necessary to recognize the user's makeup colors.

[0003] Related technologies typically train deep learning models to identify makeup colors on a user's face in facial images. However, facial images are private user data, making it difficult to collect large datasets. Furthermore, model training requires significant computational resources, resulting in high costs. The model's accuracy is inversely proportional to its real-time performance; deep learning models that meet real-time requirements often have low detection accuracy. Summary of the Invention

[0004] This application proposes a makeup color recognition method, apparatus, device, and storage medium to identify the color tones of preset parts of a user's face and automatically determine the user's makeup color tone using the identified color tones. It can accurately identify the user's makeup color tone through image processing alone, without the need for deep learning models, resulting in low computational load, low cost, reduced server processing pressure, and reduced reliance on hardware resources and human resources.

[0005] The first aspect of this application proposes a method for makeup color recognition, including:

[0006] Obtain the user's makeup image;

[0007] Identify the color tone category of the preset area of ​​the user's face in the makeup image;

[0008] The user's makeup color tone is determined based on the color tone category corresponding to the preset part.

[0009] In some embodiments of this application, identifying the hue category of the color of a preset area of ​​the user's face in the makeup image includes:

[0010] Detect the facial key points corresponding to the makeup image;

[0011] Based on the facial key points, obtain the facial region image corresponding to the makeup image;

[0012] Identify the hue category of the color of a preset part of the user's face in the face region image.

[0013] In some embodiments of this application, identifying the hue category of the color of a preset part of the user's face in the face region image includes:

[0014] Based on the facial key points, extract the target region image corresponding to the preset part from the facial region image;

[0015] Obtain the dominant color of the pixels in the target region image;

[0016] Based on the primary color of the pixel, determine the hue category to which the color of the preset part belongs.

[0017] In some embodiments of this application, the preset location includes facial skin; the step of extracting the target region image corresponding to the preset location from the facial region image based on the facial key points includes:

[0018] Based on the facial key points, extract the facial image from the facial region image;

[0019] Based on the facial key points, the eyebrow region, eye region, and mouth region are extracted from the facial image to obtain the target region image corresponding to the facial skin.

[0020] In some embodiments of this application, the preset region includes an eyeshadow area; the step of extracting the target region image corresponding to the preset region from the face region image based on the facial key points includes:

[0021] Based on the eye key points included in the facial key points, the eye image is extracted from the facial region image;

[0022] The eye image is subjected to image dilation processing a preset number of times;

[0023] Based on the key eye points included in the key eye points, the eye area is removed from the expanded eye area to obtain the target area image corresponding to the eyeshadow area.

[0024] In some embodiments of this application, the preset location includes the mouth; the step of extracting the target region image corresponding to the preset location from the face region image based on the facial key points includes:

[0025] Interpolation is performed on the upper lip key point, lower lip key point, upper lip key point, and lower lip key point, which are included in the facial key points, to obtain the upper lip key point, lower lip key point, upper lip key point, and lower lip key point interpolation.

[0026] Based on the key points of the upper lip, the interpolation points of the upper lip, the key points of the lower lip, and the interpolation points of the lower lip, the upper lip image is extracted from the face region image;

[0027] Based on the key points of the upper edge of the lower lip, the interpolation points of the upper edge of the lower lip, the key points of the lower edge of the lower lip, and the interpolation points of the lower edge of the lower lip, the lower lip image is extracted from the face region image;

[0028] The upper lip image and the lower lip image are stitched together to form a target region image corresponding to the mouth.

[0029] In some embodiments of this application, determining the hue category of the preset region's color based on the pixel's primary color includes:

[0030] Convert the color space of the pixel's primary color to the preset color space corresponding to the preset part;

[0031] Based on the color channel values ​​of the pixel's primary color in the preset color space, determine the hue range to which each color channel value belongs;

[0032] The color category corresponding to the color range is determined as the color category to which the preset part belongs.

[0033] In some embodiments of this application, the preset location includes at least one of facial skin, eye shadow area, and mouth area; the preset color space corresponding to the facial skin includes the LAB color space, and the preset color spaces corresponding to the eye shadow area and mouth area include the HSV color space.

[0034] In some embodiments of this application, the preset area includes one or more areas selected from facial skin, eyeshadow area, and mouth area. Determining the user's makeup color tone based on the color category corresponding to the preset area includes:

[0035] Determine the color coefficient corresponding to the hue category for each preset part;

[0036] The user's makeup color coefficient is calculated based on the preset weight corresponding to each preset part and the color coefficient.

[0037] The color category corresponding to the makeup color coefficient is determined as the user's makeup color tone.

[0038] In some embodiments of this application, obtaining the facial region image corresponding to the makeup image based on the facial key points includes:

[0039] Based on the facial key points, the makeup image and the facial key points are rotated and corrected.

[0040] Based on the corrected facial key points, an image containing the facial region is extracted from the corrected makeup image;

[0041] The image containing the face region is scaled to a preset size to obtain the face region image corresponding to the makeup image.

[0042] In some embodiments of this application, the step of rotating and correcting the makeup image and the facial key points based on the facial key points includes:

[0043] Based on the left eye key point and the right eye key point included in the facial key points, the coordinates of the left eye center and the right eye center are determined respectively.

[0044] Based on the coordinates of the left eye center and the right eye center, determine the rotation angle and the coordinates of the rotation center point corresponding to the makeup image;

[0045] Based on the rotation angle and the coordinates of the rotation center point, the makeup image and the key facial points are rotated and corrected.

[0046] In some embodiments of this application, the step of extracting an image containing a facial region from the corrected makeup image based on the corrected facial key points includes:

[0047] Based on the corrected facial key points, the facial region contained in the corrected makeup image is cropped.

[0048] In some embodiments of this application, based on the corrected facial key points, image cropping is performed on the facial region contained in the corrected makeup image, including:

[0049] Determine the minimum x-coordinate value, minimum y-coordinate value, maximum x-coordinate value, and maximum y-coordinate value from the corrected facial key points;

[0050] Based on the minimum x-coordinate value, the minimum y-coordinate value, the maximum x-coordinate value, and the maximum y-coordinate value, determine the cropping box corresponding to the face region in the corrected makeup image;

[0051] Based on the cropping frame, an image containing the face region is cropped from the corrected makeup image.

[0052] In some embodiments of this application, it also includes:

[0053] Enlarge the cropping frame by a preset factor;

[0054] Based on the magnified cropping frame, an image containing the face region is cropped from the corrected makeup image.

[0055] In some embodiments of this application, it also includes:

[0056] Based on the size of the image containing the face region and the preset size, the corrected facial key points are scaled and translated.

[0057] In some embodiments of this application, it also includes:

[0058] Detect whether the makeup image contains at least one complete human face image;

[0059] If so, then perform the operation of identifying the color tone category of the preset part of the user's face in the makeup image;

[0060] If not, a prompt message is sent to the user's terminal, prompting the user to provide a makeup image that includes at least one complete face image.

[0061] A second aspect of this application provides a makeup color recognition device, the device comprising:

[0062] The acquisition module is used to acquire the user's makeup image;

[0063] The recognition module is used to identify the color tone category of the preset part of the user's face in the makeup image;

[0064] The determination module is used to determine the user's makeup color tone based on the color tone category corresponding to the preset part.

[0065] An embodiment of the third aspect of this application provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the method described in the first aspect above.

[0066] An embodiment of the fourth aspect of this application provides a computer-readable storage medium having a computer program stored thereon, the program being executed by a processor to implement the method described in the first aspect above.

[0067] The technical solutions provided in this application embodiment have at least the following technical effects or advantages:

[0068] In this embodiment, the color tone category of a preset part of the user's makeup image is identified, and then the user's makeup tone is determined based on the color tone category corresponding to the preset part. The user's face undergoes color tone extraction for at least one part, and the extracted color tone is used to automatically identify the user's makeup tone. Furthermore, by combining makeup elements such as skin tone, eyeshadow, and lipstick (one or more dimensions), only the color information of the parts that have a significant impact on the makeup color is extracted, excluding the influence of other makeup effects on the makeup tone, thus increasing the uniformity and accuracy of makeup tone recognition. Moreover, the user's makeup tone can be accurately identified solely through image processing, without the need for deep learning models, resulting in low computational load, low cost, reduced server processing pressure, and reduced reliance on hardware resources and human resources.

[0069] Additional aspects and advantages of this application will be set forth in part in the description which follows, and in part will be obvious from the description, or may be learned by practice of this application. Attached Figure Description

[0070] Various other advantages and benefits will become apparent to those skilled in the art upon reading the following detailed description of preferred embodiments. The accompanying drawings are for illustrative purposes only and are not intended to limit the scope of this application. Furthermore, the same reference numerals denote the same parts throughout the drawings.

[0071] In the attached diagram:

[0072] Figure 1 A flowchart of a makeup color recognition method provided in an embodiment of this application is shown;

[0073] Figure 2 A schematic diagram showing the rotation angle of the solved image provided in an embodiment of this application is illustrated;

[0074] Figure 3 A schematic diagram illustrating two coordinate system transformations provided in an embodiment of this application is shown;

[0075] Figure 4 This illustration shows a structural schematic diagram of a makeup color recognition device according to an embodiment of this application;

[0076] Figure 5 This illustration shows a schematic diagram of the structure of an electronic device according to an embodiment of this application;

[0077] Figure 6 A schematic diagram of a storage medium provided in one embodiment of this application is shown. Detailed Implementation

[0078] Exemplary embodiments of this application will now be described in more detail with reference to the accompanying drawings. While exemplary embodiments of this application are shown in the drawings, it should be understood that this application may be implemented in various forms and should not be limited to the embodiments set forth herein. Rather, these embodiments are provided to enable a more thorough understanding of this application and to fully convey the scope of this application to those skilled in the art.

[0079] It should be noted that, unless otherwise stated, the technical or scientific terms used in this application shall have the ordinary meaning as understood by one of ordinary skill in the art to which this application pertains.

[0080] The following description, in conjunction with the accompanying drawings, describes a makeup color recognition method, apparatus, device, and storage medium according to embodiments of this application.

[0081] Eyeshadow, blush, lipstick, and other makeup elements play a crucial role in the overall color tone of facial makeup. In fields like virtual makeup and beauty treatments, it's essential to recognize the user's makeup colors. Related technologies typically train deep learning models to identify makeup colors in user facial images. However, facial images are private user data, making it difficult to collect large datasets. Furthermore, model training requires significant computational resources, resulting in high costs. The accuracy of the model is inversely proportional to real-time performance; deep learning models that meet real-time requirements often have low detection accuracy.

[0082] Based on this, embodiments of this application provide a makeup color recognition method. This method identifies the hue category of a user-preset part of a makeup image, and then determines the user's makeup hue based on the hue category corresponding to the preset part. It can accurately identify the user's makeup hue through image processing alone, without the need for deep learning models, resulting in low computational load, low cost, and reduced server processing pressure.

[0083] See Figure 1 The method specifically includes the following steps:

[0084] Step 901: Obtain the user's makeup image.

[0085] The user's makeup image includes at least one facial image. This makeup image can be a single image or any frame from the user's makeup video. In this embodiment, the executing entity is a server. The user's mobile phone or computer, or other user terminal, has a client installed that matches the makeup color recognition service provided by the server. This client has an interface for submitting makeup images. When a user needs to identify the makeup color tone corresponding to a certain makeup look, the user submits the makeup image through this interface.

[0086] Specifically, when the client detects that a user clicks on this interface, it displays a makeup color recognition interface, which may include a camera button and / or a local file upload interface. If the client detects that the user clicks the camera button, it uses the user's terminal camera to capture a makeup image or video. The captured makeup image or video is then sent to the server. If the client detects that the user clicks the local file upload interface, it displays a list of local folders, allowing the user to select the desired makeup image or video. The client then sends the user-selected makeup image or video to the server.

[0087] The server receives makeup images or videos sent by user terminals. If the server receives a makeup video from a user terminal, it uses the currently received frame image as the makeup image to be identified.

[0088] In other embodiments of this application, after obtaining the user's makeup image, the server further detects whether the makeup image contains at least one complete face image. If the makeup image contains one or more complete face images, the makeup tone corresponding to each complete face image in the makeup image is identified according to the method provided in this embodiment. If it is detected that the makeup image does not contain any face images, or that all the face images contained are incomplete, a prompt message is sent to the user's terminal. The user's terminal receives and displays the prompt message to prompt the user to provide a makeup image containing at least one complete face image. For example, the prompt message could be "Please ensure that the makeup image contains at least one complete face."

[0089] After obtaining the user's makeup image through this step, the server determines and identifies the makeup tone of the user's face in the makeup image through the following steps 902 and 903.

[0090] Step 902: Identify the color tone category of the preset parts of the user's face in the makeup image.

[0091] The makeup image includes at least one complete face image. Since the process of identifying the makeup tone of each complete face image is the same, this embodiment of the application will use the process of identifying the makeup tone of a complete face image as an example for explanation. For the at least one complete face image included in the makeup image, any one of the complete face images is selected as the user face to be processed.

[0092] The server identifies the color tone category of a preset part of the user's face through the following steps S1-S3:

[0093] S1: Detect facial landmarks corresponding to the makeup image.

[0094] The server is configured with a pre-trained detection model for detecting facial landmarks, which provides an interface service for facial landmark detection. After the server obtains the user's makeup image, it calls the facial landmark detection interface service and uses the detection model to identify all facial landmarks of the user's face in the makeup image.

[0095] The identified facial landmarks include key points on the user's facial contours, as well as key points on the mouth, nose, eyes, eyebrows, and other parts of the face. The number of identified facial landmarks can be up to 106.

[0096] S2: Based on facial key points, obtain the facial region image corresponding to the makeup image.

[0097] The server obtains the face region image corresponding to the current user's face through the following steps S20-S22:

[0098] S20: Rotate and correct the makeup image and facial key points based on facial key points.

[0099] Specifically, based on the facial keypoints corresponding to the user's face, including the left and right eye keypoints, the center coordinates of the left and right eyes are determined respectively. All left-eye keypoints in the left eye region and all right-eye keypoints in the right eye region are determined from the facial keypoints. The average of the x-coordinates and y-coordinates of all determined left-eye keypoints is taken, and a coordinate is formed by combining the average of the x-coordinates and y-coordinates corresponding to the left eye. This coordinate is then determined as the center coordinate of the left eye. The center coordinates of the right eye are determined in the same way.

[0100] Then, based on the coordinates of the left and right eye centers, determine the rotation angle and the coordinates of the rotation center point of the makeup image. For example... Figure 2 As shown, the horizontal difference (dx) and vertical difference (dy) between the left and right eye center coordinates are calculated, along with the length (d) of the line connecting the left and right eye center coordinates. Based on the length (d), horizontal difference (dx), and vertical difference (dy), the angle (θ) between the line and the horizontal direction is calculated; this angle (θ) is the rotation angle of the makeup image. Then, the coordinates of the center point of the line connecting the left and right eye center coordinates are calculated; this midpoint coordinate is the rotation center point of the makeup image.

[0101] Based on the calculated rotation angle and rotation center point coordinates, the makeup image and facial key points are rotated and corrected. Specifically, the rotation angle and rotation center point coordinates are input into a preset function for calculating the image's rotation matrix. This preset function can be the OpenCV function `cv2.getRotationMatrix2D()`. The rotation matrix corresponding to the makeup image is obtained by calling this preset function. Then, the product of the makeup image and the rotation matrix is ​​calculated to obtain the corrected makeup image. The operation of correcting the makeup image using the rotation matrix can also be accomplished by calling the OpenCV function `cv2.warpAffine()`.

[0102] For facial landmarks, each landmark needs to be corrected individually to correspond with the corrected makeup image. This correction requires two coordinate system transformations: first, transforming the coordinate system originating from the top left corner of the makeup image to the bottom left corner; then, further transforming the bottom left corner coordinate system to a coordinate system with the aforementioned rotation center point as its origin. Figure 3 As shown. After two coordinate system transformations, the key points of each face are transformed using the following formula (1) to complete the rotation correction of the key points of the face.

[0103]

[0104] In formula (1), x0 and y0 are the abscissa and ordinate of the key points of the face before rotation correction, respectively, x and y are the abscissa and ordinate of the key points of the face after rotation correction, and θ is the rotation angle mentioned above.

[0105] The corrected makeup image and facial key points are based on the entire image. The entire image not only contains the user's facial information, but also other redundant image information. Therefore, the facial region needs to be cropped in the corrected image through the following step S21.

[0106] S21: Based on the corrected facial key points, extract the image containing the facial region from the corrected makeup image.

[0107] Based on the corrected facial landmarks, the region containing the user's face in the corrected makeup image is cropped. First, the minimum, minimum, maximum, and maximum x-coordinate values ​​are determined from the corrected facial landmarks. Then, based on these values, a cropping box corresponding to the face region in the corrected makeup image is determined. Specifically, the minimum and minimum x-coordinate values ​​form a coordinate point, which is used as the top-left vertex of the cropping box corresponding to the face region. The maximum and maximum x-coordinate values ​​form another coordinate point, which is used as the bottom-right vertex of the cropping box. The position of the cropping box is determined based on these top-left and bottom-right vertices in the corrected makeup image, and the image within this cropping box, i.e., the image containing the user's face, is extracted from the corrected makeup image.

[0108] In other embodiments of this application, to ensure that the entire facial region of the user is captured and to avoid significant errors in subsequent makeup progress detection due to incomplete capture, the capture frame can be enlarged by a preset factor, such as 1.15 or 1.25. This application does not limit the specific value of the preset factor; it can be set according to requirements in practical applications. After enlarging the capture frame by the preset factor, the image within the enlarged capture frame is extracted from the corrected makeup image, thereby capturing an image containing the complete facial region of the user.

[0109] S22: Scale the image containing the face region to a preset size to obtain the face region image corresponding to the makeup image.

[0110] After extracting the image containing the user's face from the makeup image using the above method, the image containing the face is scaled to a preset size to obtain the face image corresponding to the makeup image. This preset size can be 390×390 or 400×400, etc. This embodiment does not limit the specific value of the preset size; it can be set according to requirements in practical applications.

[0111] To adapt the facial key points to the scaled facial region image, after scaling the cropped image containing the facial region to a preset size, the corrected facial key points need to be scaled and translated based on the original size of the image containing the facial region and the preset size. Specifically, based on the original size of the image containing the facial region and the preset size to which the image needs to be scaled, the translation direction and distance of each facial key point are determined. Then, according to the corresponding translation direction and distance, each facial key point is translated, and the coordinates of each translated facial key point are recorded.

[0112] The facial region image is obtained from the makeup image using the above method, and facial key points are adapted to the obtained facial region image through operations such as rotation correction and translation scaling. Then, step S3 is used to identify the color tone category of the preset parts of the user's face.

[0113] S3: Identify the hue category of the preset part of the user's face in the face region image.

[0114] Specifically, the color category corresponding to the preset part is identified through the following steps S30-S32:

[0115] S30: Based on facial key points, extract the target region image corresponding to the preset part from the facial region image.

[0116] In this embodiment of the application, the preset location may include one or more of the following areas: facial skin, eye shadow area, mouth area, etc.

[0117] For facial skin, firstly, based on all facial landmarks of the user's face, the facial image is extracted from the facial region image to remove hair and background areas. Then, based on all eyebrow landmarks included in the facial landmarks, the eye area is extracted from the facial image. Finally, based on all mouth landmarks included in the facial landmarks, the mouth area is extracted from the facial image. This results in a target region image containing only the facial skin, which facilitates subsequent identification of facial skin tone, eliminates interference from colors in other areas, and improves the accuracy of facial skin tone identification.

[0118] For the eyeshadow area, firstly, based on the eye key points included in the facial key points, the eye image is extracted from the facial region image. This eye image includes the upper and lower eyelid areas and the eye area. Eyeshadow makeup is applied to the upper and lower eyelid areas, and the color of the sclera and iris in the eye area can interfere with the color recognition of the eyeshadow. Therefore, the eye area needs to be removed. First, the obtained eye image undergoes a preset number of image dilation processes to ensure that the dilated eye image includes the eyeshadow area. The preset number of processes can be 3 or 4, etc. Then, based on the eye key points included in the eye key points, the eye area is extracted from the dilated eye region to obtain the target area image corresponding to the eyeshadow. This facilitates subsequent identification of the eyeshadow hue, eliminates the interference of eye color on eyeshadow hue recognition, and improves the accuracy of eyeshadow hue recognition.

[0119] For the mouth, firstly, the upper lip upper edge keypoint, upper lip lower edge keypoint, lower lip upper edge keypoint, and lower lip lower edge keypoint are determined from the facial keypoints corresponding to the user's face. Linear interpolation is then performed on the upper lip upper edge keypoint to obtain multiple upper lip upper edge interpolation points. Interpolation is also performed on the upper lip lower edge keypoint to obtain multiple upper lip lower edge interpolation points. Based on the upper lip upper edge keypoint, upper lip upper edge interpolation points, upper lip lower edge keypoint, and upper lip lower edge interpolation points, the upper lip image is extracted from the face region image.

[0120] Specifically, connecting multiple key points along the upper edge of the upper lip to multiple interpolation points along the upper edge of the upper lip sequentially yields a smoother curve, which is the boundary line of the upper lip. Similarly, connecting multiple key points along the lower edge of the upper lip to multiple interpolation points along the lower edge of the upper lip yields a smoother curve, which is the boundary line of the lower edge of the upper lip. The area enclosed by the boundary lines of the upper and lower edges of the upper lip is the upper lip region. Extracting this upper lip region from the face region image yields the upper lip image.

[0121] Linear interpolation is performed on the key points along the upper edge of the lower lip to obtain multiple interpolation points along the upper and lower edges of the lower lip. Linear interpolation is also performed on the key points along the lower edge of the lower lip to obtain multiple interpolation points along the lower edge of the lower lip. Based on these key points, interpolation points, and interpolation points, the lower lip image is extracted from the face region image.

[0122] Specifically, connecting multiple key points along the upper edge of the lower lip to multiple interpolation points along the upper edge of the lower lip sequentially yields a smoother curve, which is the boundary line of the upper edge of the lower lip. Similarly, connecting multiple key points along the lower edge of the lower lip to multiple interpolation points along the lower edge of the lower lip sequentially yields a smoother curve, which is the boundary line of the lower edge of the lower lip. The area enclosed by the boundary lines of the upper and lower edges of the lower lip is the lower lip region. Extracting this lower lip region from the face region image yields the lower lip image.

[0123] The obtained upper lip and lower lip images are stitched together to form the target region image corresponding to the mouth. This removes the area between the upper and lower lips, especially when the mouth is open, and removes the image of the internal oral cavity area, preventing the area between the upper and lower lips from affecting the recognition of lip color and improving the accuracy of recognizing lipstick shades.

[0124] This application does not impose any special limitation on the preset part, and the preset part can also be other parts. Based on the facial key points, the target area image corresponding to the preset part is extracted from the face area image so as to identify the color of the preset part from the target area image.

[0125] S31: Obtain the dominant color of the pixels in the target region image.

[0126] After obtaining the target region image corresponding to the preset part through the above step S30, the main color tone of the target region image is extracted to determine the main color of the pixels in the target region image. Specifically, the target region image read by OpenCV is converted from the BGRA color space to the RGBA color space that conforms to the PIL reading format. Then, the get_palette(image) function is called to create a color palette, and the median cut algorithm is used to gather similar colors to obtain the main color of the pixels in the target region image.

[0127] S32: Determine the hue category of the preset part based on the main pixel color of the target area image.

[0128] The primary color of a pixel in the target area image includes the values ​​of the R, G, and B color channels. Different preset areas have their own characteristics in terms of makeup color tones. For example, facial skin makeup tones may include different shades such as pink, fair, natural, wheat, bronze, and dark. It is difficult to determine the color category corresponding to the primary color of a pixel for lighter shades like pink, fair, and natural using the RGB color space. Similarly, lipstick tones may include nude pink, red, orange, and reddish-brown. Shades with subtle differences, such as pink and red, are difficult to distinguish using the RGB color space. Therefore, this application embodiment pre-configures preset color spaces corresponding to different preset areas, making it easier to distinguish different makeup tones for those areas. Furthermore, this application embodiment pre-configures multiple color categories corresponding to different preset areas, as well as color ranges corresponding to each color category. The color range corresponding to a color category includes the value range of each color channel of the color belonging to that color category in the corresponding preset color space.

[0129] After obtaining the dominant pixel color of the target area image in step S31, the color space of the dominant pixel color of the target area image is converted from RGB to the preset color space corresponding to the preset part. Then, based on the color channel values ​​of the dominant pixel color in the preset color space, the hue range to which each color channel value belongs is determined. The hue category corresponding to this hue range is determined as the hue category to which the color of the preset part belongs.

[0130] In this embodiment, the preset location includes at least one of the following: facial skin, eyeshadow area, and mouth area. The preset color space corresponding to the facial skin can include the LAB color space, which consists of one luminance channel and two color channels, with each color represented by three values: L, A, and B. Here, L represents luminance, A represents the component from green to red, and B represents the component from blue to yellow. Lighter colors can also be accurately distinguished in the LAB color space. The preset color space corresponding to the eyeshadow area and mouth area can include the HSV color space, which can also accurately distinguish colors with small hue differences.

[0131] Therefore, in this step, for the facial skin area, the primary color of the pixels in the target region image corresponding to the facial skin is converted from the RGB color space to the LAB color space. Based on the L, A, and B channel values ​​of the primary color of the facial skin pixels in the LAB color space, the tonal range to which the L, A, and B channel values ​​belong is determined. The tonal category corresponding to this tonal range is then determined as the tonal category to which the user's skin tone belongs.

[0132] For the eyeshadow area, the primary pixel color of the target region image corresponding to the eyeshadow is converted from the RGB color space to the HSV color space. Based on the H, S, and V channel values ​​of the primary pixel color corresponding to the eyeshadow in the HSV color space, the tonal range of the H, S, and V channel values ​​is determined. The tonal category corresponding to this tonal range is determined as the tonal category of the eyeshadow color for this user's face.

[0133] For the mouth area, the primary color of the pixels in the target region image corresponding to the mouth is converted from the RGB color space to the HSV color space. Based on the H, S, and V channel values ​​of the primary color of the pixels corresponding to the mouth in the HSV color space, the tonal range of the H, S, and V channel values ​​is determined. The tonal category corresponding to this tonal range is then determined as the tonal category of the lipstick color for that user's face.

[0134] Step 903: Determine the user's makeup color tone based on the color tone category corresponding to the preset parts.

[0135] In this embodiment, the preset areas are one or more areas, and color coefficients corresponding to each tone category are pre-configured. For example, the color coefficient for a fair tone can be 0.2, the color coefficient for a brownish-orange tone can be 0.3, and so on. After identifying the tone category corresponding to each preset area through the above steps, the color coefficient corresponding to the tone category of each preset area is determined. Based on the preset weight and color coefficient corresponding to each preset area, the user's makeup color coefficient is calculated. Specifically, the product of the preset weight and color coefficient corresponding to each preset area is calculated, and then the products corresponding to each preset area are summed. The sum is used as the user's makeup color coefficient.

[0136] This application embodiment also pre-configures color coefficient ranges corresponding to different makeup color tone categories. After determining the user's makeup color coefficient using the above method, the color coefficient range to which the makeup color coefficient belongs is determined, and the color tone category corresponding to this color coefficient range is determined as the user's makeup color tone. This makeup color tone can be considered as the main color tone of the user's facial makeup.

[0137] As an example, the preset areas include facial skin, eyeshadow, and lips. After identifying the skin tone, eyeshadow tone, and lipstick tone, the color coefficients corresponding to the skin tone, eyeshadow tone, and lipstick tone are determined respectively. Assuming the color coefficient for a "fair" skin tone is 0.2, the color coefficient for a "brownish-orange" eyeshadow tone is 0.3, and the color coefficient for a "nude pink" lipstick tone is 0.5, and the preset weights for facial skin, eyeshadow, and lips are all 0.33, then 0.33*0.2 + 0.33*0.3 + 0.33*0.5 = 0.33, meaning the user's makeup color coefficient is 0.33. Assuming the nude pink tone corresponds to a color coefficient range of 0-10, the user's makeup tone is determined to be nude pink.

[0138] As other examples, in some embodiments of this application, the preset area may only include facial skin and the mouth, combining facial skin tone and lipstick color to identify the user's makeup tone. Alternatively, the preset area may only include facial skin and eyeshadow, combining facial skin tone and eyeshadow color to identify the user's makeup tone. Alternatively, the preset area may only include eyeshadow and the mouth, combining lipstick color and eyeshadow color to identify the user's makeup tone. Alternatively, the preset area may only include facial skin, identifying the user's makeup tone through facial skin tone. Alternatively, the preset area may only include eyeshadow, identifying the user's makeup tone through eyeshadow color. Alternatively, the preset area may only include the mouth, identifying the user's makeup tone through lipstick color.

[0139] In this embodiment, the color tone category of a preset part of the user's makeup image is identified, and then the user's makeup tone is determined based on the color tone category corresponding to the preset part. The user's face undergoes color tone extraction for at least one part, and the extracted color tone is used to automatically identify the user's makeup tone. Furthermore, by combining makeup elements such as skin tone, eyeshadow, and lipstick (one or more dimensions), only the color information of the parts that have a significant impact on the makeup color is extracted, excluding the influence of other makeup effects on the makeup tone, thus increasing the uniformity and accuracy of makeup tone recognition. Moreover, the user's makeup tone can be accurately identified solely through image processing, without the need for deep learning models, resulting in low computational load, low cost, reduced server processing pressure, and reduced reliance on hardware resources and human resources.

[0140] This application also provides a makeup color recognition device, which is used to perform the makeup color recognition method provided in any of the above embodiments. Figure 4 As shown, the device includes:

[0141] Module 100 is used to acquire the user's makeup image;

[0142] The recognition module 200 is used to identify the color tone category of a preset part of the user's face in a makeup image;

[0143] The determination module 300 is used to determine the user's makeup color tone based on the color tone category corresponding to the preset parts.

[0144] The recognition module 200 is used to detect facial key points corresponding to the makeup image; obtain the facial region image corresponding to the makeup image based on the facial key points; and identify the color tone category of the preset part of the user's face in the facial region image.

[0145] The recognition module 200 is used to extract a target region image corresponding to a preset part from a face region image based on facial key points; obtain the main color of the pixels in the target region image; and determine the hue category of the preset part based on the main color of the pixels.

[0146] The preset area includes facial skin; the recognition module 200 is used to extract the facial image from the facial region image based on facial key points; and to remove the eyebrow region, eye region and mouth region from the facial image based on facial key points to obtain the target region image corresponding to the facial skin.

[0147] The preset area includes the eye shadow area; the recognition module 200 is used to extract the eye image from the face region image based on the eye key points included in the face key points; perform image dilation processing on the eye image a preset number of times; and extract the eye region from the dilated eye region based on the eye key points included in the eye key points to obtain the target region image corresponding to the eye shadow area.

[0148] The preset area includes the mouth; the recognition module 200 is used to interpolate the upper lip key points, lower lip key points, lower lip key points, and lower lip key points of the facial key points, respectively, to obtain the upper lip key point, lower lip key point, upper lip key point, and lower lip key point interpolation points; based on the upper lip key points, upper lip key points, lower lip key points, and lower lip key points, the upper lip image is extracted from the facial region image; based on the lower lip key points, lower lip key points, and lower lip key points, the lower lip image is extracted from the facial region image; the upper lip image and the lower lip image are stitched together to form the target region image corresponding to the mouth.

[0149] The determination module 300 is used to convert the color space of the pixel's main color to the preset color space corresponding to the preset part; determine the hue range to which each color channel value belongs based on the color channel values ​​of the pixel's main color in the preset color space; and determine the hue category corresponding to the hue range as the hue category to which the color of the preset part belongs.

[0150] The preset areas include at least one of the following: facial skin, eyeshadow, and lips; the preset color space corresponding to facial skin includes the LAB color space, and the preset color spaces corresponding to eyeshadow and lips include the HSV color space.

[0151] The preset areas include one or more areas among facial skin, eyeshadow, and mouth. The determination module 300 is used to determine the color coefficient corresponding to the color category of each preset area; calculate the user's makeup color coefficient based on the preset weight and color coefficient corresponding to each preset area; and determine the color category corresponding to the makeup color coefficient as the user's makeup color.

[0152] The recognition module 200 is used to rotate and correct the makeup image and the facial key points based on the facial key points; to extract an image containing the face region from the corrected makeup image based on the corrected facial key points; and to scale the image containing the face region to a preset size to obtain the face region image corresponding to the makeup image.

[0153] The recognition module 200 is used to determine the center coordinates of the left eye and the center coordinates of the right eye based on the left eye key points and the right eye key points included in the facial key points; determine the rotation angle and rotation center point coordinates of the makeup image based on the left eye center coordinates and the right eye center coordinates; and perform rotation correction on the makeup image and the facial key points based on the rotation angle and rotation center point coordinates.

[0154] The recognition module 200 is used to extract the facial region contained in the corrected makeup image based on the corrected facial key points.

[0155] The recognition module 200 is used to determine the minimum horizontal coordinate value, minimum vertical coordinate value, maximum horizontal coordinate value, and maximum vertical coordinate value from the corrected facial key points; determine the cropping box corresponding to the face region in the corrected makeup image based on the minimum horizontal coordinate value, minimum vertical coordinate value, maximum horizontal coordinate value, and maximum vertical coordinate value; and extract the image containing the face region from the corrected makeup image based on the cropping box.

[0156] The recognition module 200 is also used to enlarge the cropping box by a preset factor; based on the enlarged cropping box, it extracts the image containing the face region from the corrected makeup image.

[0157] The recognition module 200 is also used to scale and translate the corrected facial key points according to the size of the image containing the face region and the preset size.

[0158] The device also includes: a complete face detection module, used to detect whether the makeup image contains at least one complete face image; if so, it performs the operation of identifying the color tone category of a preset part of the user's face in the makeup image; if not, it sends a prompt message to the user's terminal, the prompt message being used to prompt the user to provide a makeup image containing at least one complete face image.

[0159] The makeup color recognition device and the makeup color recognition method provided in the above embodiments of this application are based on the same inventive concept and have the same beneficial effects as the methods adopted, run or implemented by the applications stored therein.

[0160] This application also provides an electronic device for performing the above-described makeup color recognition method. Please refer to... Figure 5 This illustrates a schematic diagram of an electronic device provided by some embodiments of this application. For example... Figure 5As shown, the electronic device 11 includes: a processor 1100, a memory 1101, a bus 1102, and a communication interface 1103. The processor 1100, the communication interface 1103, and the memory 1101 are connected via the bus 1102. The memory 1101 stores a computer program that can run on the processor 1100. When the processor 1100 runs the computer program, it executes the makeup color recognition method provided in any of the foregoing embodiments of this application.

[0161] The memory 1101 may include high-speed random access memory (RAM) or non-volatile memory, such as at least one disk storage device. Communication between this device network element and at least one other network element is achieved through at least one communication interface 1103 (which can be wired or wireless), such as the Internet, wide area network, local area network, metropolitan area network, etc.

[0162] Bus 1102 can be an ISA bus, PCI bus, or EISA bus, etc. The bus can be divided into an address bus, a data bus, a control bus, etc. The memory 1101 is used to store programs. After receiving an execution instruction, the processor 1100 executes the program. The makeup color recognition method disclosed in any of the foregoing embodiments of this application can be applied to the processor 1100, or implemented by the processor 1100.

[0163] The processor 1100 may be an integrated circuit chip with signal processing capabilities. In implementation, each step of the above method can be completed by the integrated logic circuitry in the hardware of the processor 1100 or by instructions in software form. The processor 1100 may be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it may also be a digital signal processor (DSP), an application-specific integrated circuit (ASIC), an off-the-shelf programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components. It can implement or execute the methods, steps, and logic block diagrams disclosed in the embodiments of this application. The general-purpose processor may be a microprocessor or any conventional processor. The steps of the methods disclosed in the embodiments of this application can be directly embodied in the execution of a hardware decoding processor, or executed by a combination of hardware and software modules in the decoding processor. The software modules may reside in random access memory, flash memory, read-only memory, programmable read-only memory, electrically erasable programmable memory, registers, or other mature storage media in the art. The storage medium is located in memory 1101. Processor 1100 reads the information in memory 1101 and, in conjunction with its hardware, completes the steps of the above method.

[0164] The electronic device provided in this application embodiment and the makeup color recognition method provided in this application embodiment are based on the same inventive concept and have the same beneficial effects as the methods they adopt, operate or implement.

[0165] This application also provides a computer-readable storage medium corresponding to the makeup color recognition method provided in the foregoing embodiments. Please refer to... Figure 6 The computer-readable storage medium shown is an optical disc 30, on which a computer program (i.e., a program product) is stored. When the computer program is run by a processor, it executes the makeup color recognition method provided in any of the foregoing embodiments.

[0166] It should be noted that examples of the computer-readable storage medium may also include, but are not limited to, phase-change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other optical and magnetic storage media, which will not be elaborated here.

[0167] The computer-readable storage medium provided in the above embodiments of this application and the makeup color recognition method provided in the embodiments of this application are based on the same inventive concept and have the same beneficial effects as the methods adopted, run or implemented by the applications stored therein.

[0168] It should be noted that:

[0169] Numerous specific details are set forth in the specification provided herein. However, it will be understood that embodiments of this application may be practiced without these specific details. In some instances, well-known structures and techniques have not been shown in detail so as not to obscure the understanding of this specification.

[0170] Similarly, it should be understood that, for the sake of brevity and to aid in understanding one or more of the various inventive aspects, in the above description of exemplary embodiments of this application, various features of this application are sometimes grouped together in a single embodiment, figure, or description thereof. However, this disclosure should not be construed as reflecting a schematic diagram in which the claimed application requires more features than expressly recited in each claim. Rather, as reflected in the following claims, inventive aspects lie in fewer than all features of a single foregoing disclosed embodiment. Therefore, the claims following the detailed description are hereby expressly incorporated into that detailed description, wherein each claim itself is a separate embodiment of this application.

[0171] Furthermore, those skilled in the art will understand that although some embodiments described herein include certain features but not others included in other embodiments, combinations of features from different embodiments are intended to be within the scope of this application and form different embodiments. For example, in the following claims, any of the claimed embodiments can be used in any combination.

[0172] The above description is merely a preferred embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the technical scope disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A makeup color identification method, characterized by, The method comprises: acquiring a makeup image of a user; identifying a color tone category to which a color of a preset part of a face of the user in the makeup image belongs, comprising: performing rotation correction on the makeup image and a face key point corresponding to the makeup image according to the face key point; cutting an image containing a face region from the makeup image after correction according to the face key point after correction; scaling the image containing the face region to a preset size to obtain a face region image corresponding to the makeup image; performing scaling and translation processing on the face key point after correction according to a size of the image containing the face region and the preset size; identifying the color tone category to which the color of the preset part of the face of the user in the face region image belongs; determining a makeup color tone of the user according to the color tone category corresponding to the preset part.

2. The method of claim 1, wherein, Before the face region image corresponding to the makeup image is acquired according to the face key point, the method further comprises: detecting the face key point corresponding to the makeup image.

3. The method of claim 1, wherein, The identifying of the color tone category to which the color of the preset part of the face of the user in the face region image belongs comprises: cutting a target region image corresponding to the preset part from the face region image according to the face key point; acquiring a pixel main color of the target region image; determining the color tone category to which the color of the preset part belongs according to the pixel main color.

4. The method of claim 3, wherein, The preset part comprises face skin; the cutting of the target region image corresponding to the preset part from the face region image according to the face key point comprises: cutting a face image from the face region image according to the face key point; excluding a brow region, an eye region and a mouth region from the face image according to the face key point to obtain a target region image corresponding to face skin.

5. The method of claim 3, wherein, The preset part comprises an eye shadow part; the cutting of the target region image corresponding to the preset part from the face region image according to the face key point comprises: cutting an eye image from the face region image according to an eye key point included in the face key point; performing image dilation processing on the eye image for a preset number of times; excluding an eye region from the eye image after the dilation processing according to an eye key point included in the eye key point to obtain a target region image corresponding to the eye shadow part.

6. The method of claim 3, wherein, The preset part comprises a mouth; the cutting of the target region image corresponding to the preset part from the face region image according to the face key point comprises: respectively interpolating an upper lip upper edge key point, an upper lip lower edge key point, a lower lip upper edge key point and a lower lip lower edge key point included in the face key point to obtain an upper lip upper edge interpolation point, an upper lip lower edge interpolation point, a lower lip upper edge interpolation point and a lower lip lower edge interpolation point; cutting an upper lip image from the face region image according to the upper lip upper edge key point, the upper lip upper edge interpolation point, the upper lip lower edge key point and the upper lip lower edge interpolation point; According to the lower lip upper edge key point, the lower lip upper edge interpolation point, the lower lip lower edge key point and the lower lip lower edge interpolation point, a lower lip image is cropped from the face region image; The upper lip image and the lower lip image are spliced into a target region image corresponding to a mouth.

7. The method of claim 3, wherein, The method further comprises: Converting the color space of the pixel main color to a preset color space corresponding to the preset part; According to the color channel values of the pixel main color in the preset color space, determining a tonal interval to which the color channel values belong; Determining the tonal category corresponding to the tonal interval as the tonal category to which the color of the preset part belongs.

8. The method of claim 7, wherein, The preset part includes at least one of facial skin, eye shadow part and mouth, the preset color space corresponding to the facial skin includes LAB color space, and the preset color space corresponding to the eye shadow part and the mouth includes HSV color space.

9. The method according to any one of claims 1 to 8, characterized in that, The preset part includes one or more of facial skin, eye shadow part and mouth, and the method further comprises: Determining a color coefficient corresponding to the tonal category of each preset part; According to a preset weight corresponding to each preset part and the color coefficient, calculating a makeup color coefficient of the user; Determining the tonal category corresponding to the makeup color coefficient as the makeup tonal category of the user.

10. The method of claim 1, wherein, The method further comprises: According to the left eye key point and the right eye key point included in the face key point, respectively determining a left eye center coordinate and a right eye center coordinate; According to the left eye center coordinate and the right eye center coordinate, determining a rotation angle and a rotation center point coordinate corresponding to the makeup image; According to the rotation angle and the rotation center point coordinate, performing rotation correction on the makeup image and the face key point.

11. The method of claim 1, wherein, The method further comprises: According to the corrected face key point, performing image cropping on the face region included in the corrected makeup image.

12. The method of claim 11, wherein, The method further comprises: Determining a minimum horizontal coordinate value, a minimum vertical coordinate value, a maximum horizontal coordinate value and a maximum vertical coordinate value from the corrected face key point; According to the minimum horizontal coordinate value, the minimum vertical coordinate value, the maximum horizontal coordinate value and the maximum vertical coordinate value, determining a cropping frame corresponding to the face region in the corrected makeup image; According to the cropping frame, cropping an image containing the face region from the corrected makeup image.

13. The method of claim 12, wherein, The method further comprises: Enlarging the cropping frame by a preset multiple; According to the enlarged cropping frame, cropping an image containing the face region from the corrected makeup image.

14. The method according to any one of claims 1 to 8, characterized in that, The method further comprises: Detecting whether the makeup image contains at least one complete face image; If yes, the operation of identifying a color tone category to which a color of a preset part of a face of the user in the makeup image belongs is performed; If no, prompt information is sent to a terminal of the user, the prompt information being used to prompt the user to provide a makeup image containing at least one complete face image.

15. A makeup look color identification device, characterized in that, The device comprises: an acquisition module configured to acquire a makeup image of a user; an identification module configured to identify a color tone category to which a color of a preset part of a face of the user in the makeup image belongs, specifically configured to: perform rotation correction on the makeup image and face key points corresponding to the makeup image according to the face key points; extract an image containing a face region from the makeup image after correction according to the face key points after correction; scale the image containing the face region to a preset size to obtain a face region image corresponding to the makeup image; perform scaling and translation processing on the face key points after correction according to a size of the image containing the face region and the preset size; and identify a color tone category to which a color of a preset part of a face of the user in the makeup image belongs. a determination module configured to determine a makeup color tone of the user according to the color tone category corresponding to the preset part.

16. An electronic device comprising a memory, a processor, and a computer program stored on the memory and executable on the processor, wherein, The processor runs the computer program to implement the method of any one of 1-14.

17. A computer readable storage medium having stored thereon a computer program, characterized in that, The program is executed by the processor to implement the method of any one of 1-14.

Images