Method, system, device and storage medium for removing eye fat from a human face image

Abstract

This invention belongs to the field of image processing technology, specifically relating to a method, system, device, and storage medium for removing eye fat from facial images. The method includes: acquiring an original image containing a user's face; obtaining a mask image of the eyelid region based on facial key points detected by face detection; statistically analyzing the skin tone in the eye region of the mask image to obtain the average brightness of the skin tone and calculating the weight of skin tone light and shadow smoothness in the eye region; performing eye contour segmentation on the eyelid portion of the image to obtain the segmentation result image; converting it into a binary image for connected component detection; combining the facial feature point set to determine and filter out the eyelid contour to obtain an initial contour line; parameterizing the initial contour line; extracting discrete point sets for quadratic curve fitting to obtain a new contour line; and deforming the image to obtain a result image of the adjusted eye contour. This invention can quickly refine eye images, greatly improving the efficiency of image processing and the quality of eye refinement.

Description

Technical Field

[0001] This invention belongs to the field of image processing technology, specifically relating to a method, system, device, and storage medium for removing eye fat from facial images. Background Technology

[0002] Comprehensive eye surgery is one of the most popular procedures in the medical aesthetics industry, including eyelid fat removal and double eyelid surgery. The development and use of beauty apps have given users more options. On the one hand, they can use software to simulate and view the effects of their eye surgery; on the other hand, the beautification effects on portraits are increasingly favored by users who enjoy photo editing. Eyelid fat removal mainly addresses issues such as uneven eye shape, puffy eyelids, and excessive swelling, and can improve the appearance of many puffy eyes. The design of eyelid fat removal solutions directly affects the user experience and the revenue of related software. Current portrait eyelid fat removal solutions have the following drawbacks:

[0003] (1) Using professional software (such as PS) to achieve eye retouching requires a high learning cost and is complicated to operate. It is too difficult for ordinary users and cannot meet the needs of intelligence and simplicity for eye retouching.

[0004] (2) Existing image eye retouching algorithms can enlarge the eyes and overlay double eyelid effects, but they do not improve the problems of eye swelling and drooping eyelids, and cannot meet the increasingly demanding eye retouching needs of customers. Summary of the Invention

[0005] To address the shortcomings of current portrait eye fat removal solutions, this invention provides a method, system, device, and storage medium for removing eye fat from facial images. It utilizes an innovative portrait eye light and shadow smoothing algorithm combined with an eye shape contour adjustment algorithm, enabling even those without professional photo editing knowledge to quickly and precisely retouch eye images.

[0006] This invention is achieved using the following technical solution:

[0007] A method for removing eye fat from a human face image, comprising the following steps:

[0008] Obtain the original image containing the user's face, and obtain the mask image of the upper eyelid region based on the facial key points obtained from face detection;

[0009] The average brightness of the skin color in the eye area of ​​the mask image is obtained by statistical analysis, and the weight of the skin color light and shadow smoothness in the eye area is calculated.

[0010] Perform eye contour segmentation on the eyelid portion of the image to obtain the segmentation result image;

[0011] The obtained segmentation result image is converted into a binary image, connected component detection is performed, and the eyelid contour is selected by combining the facial feature point set to obtain the initial contour line.

[0012] The initial contour line is parameterized, a discrete point set is extracted and a quadratic curve is fitted to obtain a new contour line. The points on the new contour line are used as the coordinates of the points on the initial contour line after deformation. The image is then deformed to obtain the result image of the eye shape contour adjustment.

[0013] As a further aspect of the present invention, a method for obtaining a mask image of the upper eyelid region includes the following steps:

[0014] The feature point set of the face in the original image is obtained through face detection. The original image containing the user's face is labeled as follows: ;

[0015] Find the feature point set of the eyelid area using an index. The feature point set Take the bounding box to obtain the image of the eyelid portion, denoted as . ;

[0016] Based on the obtained images of the eyelids The system determines whether the area is skin; skin areas are the ones that need processing, while non-skin areas have all channels set to 0. The resulting image is the eye image. ;

[0017] Eye image The image is converted to a single-channel image and binarized. The skin portion of the processing area is set to 255 channels, while the non-skin portion, which does not need processing, is set to 0 channels, resulting in a mask image. .

[0018] As a further aspect of the present invention, a method for statistically analyzing the skin tone in the eye region of the mask image, obtaining the average brightness of the skin tone, and calculating the weight of the skin tone light and shadow smoothness in the eye region includes the following steps:

[0019] Image of the eye Upper mask image The pixels in the specified area are statistically analyzed to obtain skin color statistics. The average value of these skin color statistics is then used to obtain the eyelid portion of the image. Average brightness of the area ;

[0020] Using a weighted formula for average skin brightness, Adjust the skin tone in the affected area;

[0021] Using skin tone adjustment formulas, we obtain skin tone after smoothing out the light and shadow.

[0022] Save the resulting image of the smoothed light and shadow in the upper eyelid area as... .

[0023] As a further aspect of the present invention, the weighted formula for the average brightness of skin tone is:

[0024]

[0025] in, The brightness of the current pixel. Adjust the weight for skin tone.

[0026] As a further aspect of the present invention, the skin tone adjustment formula is as follows:

[0027]

[0028] in, This is the currently adjusted color. The color within the neighborhood. Adjust the weight of skin tone for colors in the neighborhood.

[0029] As a further aspect of the present invention, the method for statistically analyzing the skin tone in the eye region of the mask image to obtain the average brightness of the skin tone includes:

[0030] The RGB values ​​of each pixel in the eye region of the original image are traversed and averaged to obtain the average color value. The average brightness is obtained by using an RGB and HSV conversion algorithm. The original image is composed of RGB channels.

[0031] As a further aspect of the present invention, a method for segmenting the eyelid portion of an image to obtain a segmentation result image includes the following steps:

[0032] Based on the Canny edge detection algorithm, the eyelid portion of the image is... The image is blurred, and then convolved with a Prewitt filter to obtain the derivative images in the horizontal and vertical directions, denoted as H and V.

[0033] For the current pixel of the eyelid region, the gradient direction and magnitude are calculated. When the magnitude exceeds a critical value, an edge is assigned, and the image inside the edge is the segmentation result image to be retained. .

[0034] As a further aspect of the present invention, the segmentation result image is converted into a binary image, connected component detection is performed, and the eyelid contour is selected by combining the facial feature point set to obtain the initial contour line.

[0035] Iterate through each pixel in the binary graph, and designate adjacent pixels of the same color as the same connected component, denoted as the connected component. This yields n connected components;

[0036] Based on the feature point set of the eyelid Feature points in Calculate the contour points of each connected component in turn. distance To obtain the minimum value The minimum value The outline of the corresponding connected region is the eyelid outline;

[0037] Each feature point Find the two nearest points on the connected region contour and use these two points as points on the upper eyelid contour line. All points on the upper eyelid contour line form the contour line. Connected components containing facial feature points Outline Specify the outline of the upper eyelid.

[0038] As a further aspect of the present invention, the outline Parameterization is performed, and a discrete point set is extracted with a step size of 0.1. A quadratic curve is then fitted to the point set to obtain a new contour line. and the new outline Points on the outline The coordinates of the points on the image after deformation are used to deform the image and obtain the result image of the eye shape contour adjustment.

[0039] The present invention also includes a facial image eye fat removal system, wherein the facial image eye fat removal system uses the aforementioned facial image eye fat removal method to remove fat from the eyes of the portrait and retouches the image. The facial image eye fat removal system includes an upper eyelid area light and shadow smoothing module and an eye contour adjustment module.

[0040] The upper eyelid region light and shadow smoothing module is used to process an acquired original image containing the user's face to obtain a result image of upper eyelid region light and shadow smoothing; the eye contour adjustment module is used to process the eyelid part image to obtain a result image of eye shape contour adjustment.

[0041] As a further embodiment of the present invention, the upper eyelid region light and shadow smoothing module includes a mask image generation unit and a weight calculation unit. The mask image generation unit is used to obtain the facial key points of the original image based on face detection to obtain a mask image of the upper eyelid region. The weight calculation unit is used to statistically analyze the skin color of the eye region where the mask image is located, obtain the average brightness of the skin color, and calculate the weight of the skin color light and shadow smoothing in the eye region.

[0042] As a further embodiment of the present invention, the eye contour adjustment module includes an image segmentation unit, a contour filtering unit, and a graphic contour deformation unit. The image segmentation unit is used to segment the eyelid portion of the image to obtain a segmentation result image. The contour filtering unit is used to convert the obtained segmentation result image into a binary image, perform connected component detection, and combine it with a set of facial feature points to filter out the eyelid contour and obtain an initial contour line. The graphic contour deformation unit is used to parameterize the obtained initial contour line, extract a discrete point set for quadratic curve fitting, obtain a new contour line, and use the points on the new contour line as the coordinates of the points on the initial contour line after deformation to deform the image and obtain the result image of the eye contour adjustment.

[0043] The present invention also includes an eye fat removal device, comprising a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the steps of an eye fat removal method for a human face image.

[0044] The present invention also includes a storage medium storing a computer program that, when executed by a processor, implements the steps of a method for removing eye fat from a human face image.

[0045] The technical solution provided by this invention has the following beneficial effects:

[0046] This invention relates to a method, system, device, and storage medium for removing eye fat from facial images. Utilizing an innovative human portrait eye lighting and shadow smoothing algorithm combined with an eye shape contour adjustment algorithm, it enables even those without professional retouching knowledge to quickly and accurately retouch eye images. In terms of speed, it far surpasses manual retouching methods while retaining the advantages of manual retouching, such as natural lighting and shadows, reduction of swelling in the eye fat and orbicularis oculi muscle, and no impact on the eyebrows and eyes. It also exhibits seamlessness in both single-person and multi-person environments and lighting conditions, significantly improving image processing efficiency and the quality of eye retouching. Furthermore, given the high risks and irreversible nature of cosmetic surgery, simulating eye fat removal surgery using image processing technology can assist consumers in their decision-making.

[0047] This invention discloses a method, system, device, and storage medium for removing eye fat from facial images. By using algorithms for smoothing light and shadow in the eyes and adjusting eye shape contours, it enables convenient and precise intelligent retouching of facial images with eye fat removal. Firstly, this invention significantly improves the efficiency of users' eye fat removal retouching, meeting their needs for intelligence and simplicity in eye retouching. Secondly, this invention can improve issues such as eye puffiness and drooping eyelids, satisfying clients' eye retouching requirements. Simultaneously, this invention meets the medical aesthetic requirements for eye fat removal in comprehensive medical eye care, allowing users to preview medical aesthetic eye fat removal solutions. Attached Figure Description

[0048] The accompanying drawings are provided to further illustrate the invention and form part of the specification. They are used in conjunction with embodiments of the invention to explain the invention and do not constitute a limitation thereof. In the drawings:

[0049] Figure 1 This is a flowchart of a method for removing eye fat from a human face image according to an embodiment of the present invention.

[0050] Figure 2 This is a schematic diagram illustrating the effect of smoothing the light and shadow around the eyes of a person in a face image before and after a method for removing fat from the eyes according to an embodiment of the present invention.

[0051] Figure 3 This is a schematic diagram illustrating the effect of adjusting the eye shape contour of a human face image before and after a method for removing fat from the eye area according to an embodiment of the present invention.

[0052] Figure 4 This is a system block diagram of a facial image fat removal system according to an embodiment of the present invention. Detailed Implementation

[0053] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the invention.

[0054] This invention provides a method, system, device, and storage medium for removing eye fat from facial images. Utilizing an innovative human portrait eye lighting and shadow smoothing algorithm combined with an eye shape contour adjustment algorithm, it allows even those without professional retouching knowledge to quickly and accurately retouch eye images. In terms of speed, it far surpasses manual retouching methods while retaining the advantages of manual retouching, such as natural lighting and shadows, reduction of swelling in the eye fat and orbicularis oculi muscle, and no impact on the eyebrows and eyes. It also works seamlessly in both single-person and multi-person environments and lighting conditions, significantly improving image processing efficiency and the quality of eye retouching. Furthermore, cosmetic surgery carries high risks and irreversible complications for users; simulating eye fat removal surgery using image processing technology can assist consumers in their decision-making.

[0055] The following will describe the specific embodiments.

[0056] like Figure 1 As shown, one embodiment of the present invention provides a method for removing eye fat from a human face image. This method retouches the eye area of ​​a portrait by removing fat and mainly includes two parts: smoothing the light and shadow in the upper eyelid area and adjusting the eye contour. Specifically, it includes the following steps:

[0057] S1. Obtain the original image containing the user's face, and obtain the mask image of the upper eyelid region based on the facial key points obtained from face detection.

[0058] In this embodiment, firstly, an original image containing the user's face is acquired. Based on the facial key points obtained from face detection, a mask image of the upper eyelid region is obtained.

[0059] In this embodiment, the method for obtaining a mask image of the upper eyelid region includes the following steps:

[0060] Step 11) Obtain the feature point set of the face in the original image through face detection. The original image containing the user's face is labeled as follows: ;

[0061] Step 12) Locate the feature point set of the eyelid region using the index. The feature point set Take the bounding box to obtain the image of the eyelid portion, denoted as . ;

[0062] Step 13) Based on the acquired eyelid images The system determines whether the area is skin; skin areas are the ones that need processing, while non-skin areas have all channels set to 0. The resulting image is the eye image. ;

[0063] Step 14) Transfer the eye image The image is converted to a single-channel image and binarized. The skin portion of the processing area is set to 255 channels, while the non-skin portion, which does not need processing, is set to 0 channels, resulting in a mask image. .

[0064] During processing, face detection obtains the feature point set of the face in the original image. Find the feature point set of the eyelid region using the index. Among them, face detection can currently obtain 130 feature points. The indexes of the feature points in the eyelid area, in clockwise order, are:

[0065] Left eye: 1, 0, 33, 41, 40, 39, 38, 71, 75, 55, 54, 53, 52, 51;

[0066] Right eye: 32, 31, 65, 64, 63, 62, 61, 85, 71, 50, 49, 48, 47, 46.

[0067] Retrieve directly by index according to a fixed order.

[0068] In this embodiment, feature point set When constructing the bounding box, a cube is used, and its four vertices are denoted as a, b, c, and d. The coordinates of all feature points on the upper eyelid are iterated, and the maximum and minimum values ​​in the x and y directions are taken as the coordinates of the two diagonal points of the bounding box, denoted as a( , ),b( , ), c( , ), d( , ).

[0069] In this embodiment, the acquired eyelid portion image The system determines whether the area is skin; skin areas are the ones that need processing, while non-skin areas have all channels set to 0. The resulting image is the eye image. In determining whether a part is skin, saturation and hue are used to distinguish it from the eyebrow and eye areas.

[0070] S2. Statistically analyze the skin color in the eye area of ​​the mask image to obtain the average brightness of the skin color and calculate the weight of the skin color light and shadow smoothness in the eye area.

[0071] In this embodiment, the skin color of the eye area in the original image is statistically analyzed to obtain the average brightness of the skin color, which is used to calculate the weight of the skin color light and shadow smoothness in the eye area.

[0072] Since the acquired original image consists of data composed of RGB channels, the RGB values ​​of each pixel in the eye region of the acquired original image are traversed, and the average value is obtained by accumulating them. The average color value is then obtained by using an RGB and HSV conversion algorithm.

[0073] In this embodiment, the method for statistically analyzing the skin tone in the eye region of the mask image, obtaining the average brightness of the skin tone, and calculating the weight of the skin tone light and shadow smoothness in the eye region includes the following steps:

[0074] Step 21) Image of the eye Upper mask image The pixels in the specified area are statistically analyzed to obtain skin color statistics. The average value of these skin color statistics is then used to obtain the eyelid portion of the image. Average brightness of the area ;

[0075] Step 22) Using the weighted formula for average skin tone brightness, The skin tone of the area is adjusted; the weighted formula for the average brightness of the skin tone is:

[0076]

[0077] in, The brightness of the current pixel. Adjust the weights for skin tone;

[0078] Step 23) Use the skin tone adjustment formula to obtain the skin tone after smoothing out the light and shadow; the skin tone adjustment formula is:

[0079]

[0080] in, This is the currently adjusted color. The color within the neighborhood. Adjust the weight of skin tone for colors in the neighborhood;

[0081] Step 24) Save the resulting image of the smoothed light and shadow in the upper eyelid area as... .

[0082] In this application, in addition to eliminating swelling in the upper eyelid area, eyelid fat removal can also address the drooping phenomenon caused by upper eyelid swelling. A simple smoothing effect cannot fully simulate the effect of eyelid fat removal surgery. It is necessary to smooth the upper eyelid contour, ensure a natural curvature, and reduce the area obscured by the pupil, thereby achieving a better fat removal effect.

[0083] See Figure 2 As shown, Figure 2 This is a schematic diagram illustrating the effect of this method on smoothing light and shadow in the eyes of a person. Figure 2 In the original image used for smoothing the lighting and shadows around the eyes in a portrait, there is swelling in the upper eyelid area. Figure 3 The image shows the effect of smoothing the light and shadow in a portrait eye area, eliminating puffiness in the upper eyelid region. By smoothing the light and shadow in the upper eyelid area, dark areas are brightened, and bright areas are darkened, resulting in a softer transition between light and shadow and eliminating the appearance of bulge caused by excess fat. See also Figure 3 In the original image and the final image, the swelling in the upper left eyelid area of ​​the user's face image is significantly eliminated after the eye lighting and shadow smoothing is applied.

[0084] S3. Perform eye contour segmentation on the eyelid portion of the image to obtain the segmentation result image.

[0085] In this embodiment, image segmentation technology is used to segment the eyelid portion of the image. Perform eye contour segmentation to obtain the segmentation result image. .

[0086] In this embodiment, the method for segmenting the eyelid portion of an image to obtain a segmentation result image includes the following steps:

[0087] Based on the Canny edge detection algorithm, the eyelid portion of the image is... The image is blurred, and then convolved with a Prewitt filter to obtain the derivative images in the horizontal and vertical directions, denoted as H and V.

[0088] For the current pixel of the eyelid region, the gradient direction and magnitude are calculated. When the magnitude exceeds a critical value, an edge is assigned, and the image inside the edge is the segmentation result image to be retained. .

[0089] In this application, the existing Canny edge detection algorithm is used to perform image edge detection. The image is blurred and then convolved with a Prewitt filter to obtain the derivative images in the horizontal and vertical directions, denoted as H and V, respectively. This is applied to the current pixel. Calculate the gradient direction and magnitude. If the magnitude exceeds a threshold, assign an edge. The image inside the edge is the segmentation result image you want to retain. .

[0090] S4. Convert the obtained segmentation result image into a binary image, perform connected component detection, and combine the facial feature point set to judge and filter out the eyelid contour to obtain the initial contour line.

[0091] In this embodiment, the segmentation result image is... Convert to a binary graph, perform connected component detection, and combine with a set of facial feature points. The eyelid contour is identified and filtered to obtain the outline. .

[0092] In this embodiment, the segmentation result image is converted into a binary image, connected component detection is performed, and the eyelid contour is selected by combining the facial feature point set to obtain the initial contour line.

[0093] Step 41) Traverse each pixel in the binary image and designate adjacent pixels of the same color as the same connected component, denoted as the connected component. This yields n connected components;

[0094] Step 42) Based on the feature point set of the eyelid region Feature points in Calculate the contour points of each connected component in turn. distance To obtain the minimum value The minimum value The outline of the corresponding connected region is the eyelid outline;

[0095] Step 43) Each feature point Find the two nearest points on the connected region contour and use these two points as points on the upper eyelid contour line. All points on the upper eyelid contour line form the contour line. Connected components containing facial feature points Outline Specify the outline of the upper eyelid.

[0096] In this embodiment, the process of connected component detection is as follows: traverse each pixel, and designate adjacent pixels with the same color (white or black) as the same connected component, denoted as the connected component. Finally, we can obtain n connected components.

[0097] In this embodiment, the purpose of connected component detection is that facial feature points are too sparse to be directly used as points for the eyelid contour. It is necessary to segment the eye area to extract the contour, but due to blemishes, eyebrows, hair, etc., on the upper eyelid, there may be more than one contour line. Therefore, it is necessary to detect connected components to identify those that contain facial feature points. The outline is specified as the outline of the upper eyelid.

[0098] From facial feature point set Feature points in Calculate the contour points of each connected component in turn. distance Find the minimum value The corresponding connected component's contour line is the eyelid contour. Then, for each feature point... Find the two nearest points on the contour of the connected region and use them as points on the upper eyelid contour line. These points form the contour. .

[0099] S5. Parameterize the obtained initial contour line, extract the discrete point set and perform quadratic curve fitting to obtain a new contour line, and use the points on the new contour line as the coordinates of the points on the initial contour line after deformation to deform the image and obtain the result image of eye shape contour adjustment.

[0100] In this embodiment, the outline is... Parameterization is performed, and a discrete point set is extracted with a step size of 0.1. A quadratic curve is then fitted to the point set to obtain a new contour line. and the new outline The point on The coordinates of the points on the image after deformation are used to deform the image and obtain the result image of the eye shape contour adjustment.

[0101] See Figure 3 As shown, Figure 3 This is a schematic diagram illustrating the effect of adjusting the eye shape contour of a person using this method. Figure 3The original image used for portrait eye contour adjustment showed drooping due to upper eyelid swelling, while... Figure 3 In the result image after adjusting the eye shape in the portrait, the upper eyelid contour is smoothed to ensure a natural arc and reduce the area of ​​the pupil that is covered, thereby achieving a better fat removal effect.

[0102] This invention provides a method for removing eye fat from facial images. By employing algorithms for smoothing eye lighting and adjusting eye contours, it enables convenient and precise intelligent retouching of facial images with eye fat removal. Firstly, this method significantly improves the efficiency of users' eye fat removal retouching, meeting their needs for intelligence and simplicity in eye retouching. Secondly, the method's eye retouching algorithm can improve issues such as eye puffiness and drooping eyelids, satisfying clients' eye retouching requirements. Simultaneously, this method aligns with the medical aesthetic requirements for eye fat removal in comprehensive eye care, allowing users to preview medical aesthetic eye fat removal solutions.

[0103] like Figure 4 As shown, in another embodiment of the present invention, a system for removing eye fat from a human face image is provided for retouching the eye area of ​​a human portrait. The system includes an upper eyelid region light and shadow smoothing module 800 and an eye contour adjustment module 900.

[0104] The upper eyelid region light and shadow smoothing module 800 is used to process an acquired original image containing a user's face to obtain a result image of upper eyelid region light and shadow smoothing; the eye contour adjustment module 900 is used to process the eyelid part image to obtain a result image of eye shape contour adjustment.

[0105] In this embodiment, the upper eyelid region light and shadow smoothing module 800 includes a mask image generation unit 801 and a weight calculation unit 802. The mask image generation unit 801 is used to obtain the facial key points of the original image based on face detection to obtain a mask image of the upper eyelid region. The weight calculation unit 802 is used to perform statistical analysis on the skin color of the eye region where the mask image is located, obtain the average brightness of the skin color, and calculate the weight of the skin color light and shadow smoothing in the eye region.

[0106] In this embodiment, the eye contour adjustment module 900 includes an image segmentation unit 901, a contour line filtering unit 902, and a graphic contour deformation unit 903. The image segmentation unit 901 is used to segment the eyelid portion of the image to obtain a segmentation result image. The contour line filtering unit 902 is used to convert the obtained segmentation result image into a binary image, perform connected component detection, and combine it with the facial feature point set to filter out the eyelid contour and obtain an initial contour line. The graphic contour deformation unit 903 is used to parameterize the obtained initial contour line, extract a discrete point set for quadratic curve fitting, obtain a new contour line, and use the points on the new contour line as the coordinates of the points on the initial contour line after deformation to deform the image and obtain the result image of the eye contour adjustment.

[0107] The eye fat removal system for facial images employs the steps of a facial image eye fat removal method as described in the aforementioned embodiment. Therefore, the operation process of the facial image eye fat removal system will not be described in detail in this embodiment.

[0108] In one embodiment of the present invention, an eye fat removal device is provided, including a memory and a processor. The memory stores a computer program, and the processor executes the computer program to implement the steps in the above-described method embodiment:

[0109] Obtain the original image containing the user's face, and obtain the mask image of the upper eyelid region based on the facial key points obtained from face detection;

[0110] The average brightness of the skin color in the eye area of ​​the mask image is obtained by statistical analysis, and the weight of the skin color light and shadow smoothness in the eye area is calculated.

[0111] Perform eye contour segmentation on the eyelid portion of the image to obtain the segmentation result image;

[0112] The obtained segmentation result image is converted into a binary image, connected component detection is performed, and the eyelid contour is selected by combining the facial feature point set to obtain the initial contour line.

[0113] The initial contour line is parameterized, a discrete point set is extracted and a quadratic curve is fitted to obtain a new contour line. The points on the new contour line are used as the coordinates of the points on the initial contour line after deformation. The image is then deformed to obtain the result image of the eye shape contour adjustment.

[0114] In another embodiment of the present invention, a storage medium is provided on which a computer program is stored, which, when executed by a processor, implements the steps in the above-described method embodiments:

[0115] Obtain the original image containing the user's face, and obtain the mask image of the upper eyelid region based on the facial key points obtained from face detection;

[0116] The average brightness of the skin color in the eye area of ​​the mask image is obtained by statistical analysis, and the weight of the skin color light and shadow smoothness in the eye area is calculated.

[0117] Perform eye contour segmentation on the eyelid portion of the image to obtain the segmentation result image;

[0118] The obtained segmentation result image is converted into a binary image, connected component detection is performed, and the eyelid contour is selected by combining the facial feature point set to obtain the initial contour line.

[0119] The initial contour line is parameterized, a discrete point set is extracted and a quadratic curve is fitted to obtain a new contour line. The points on the new contour line are used as the coordinates of the points on the initial contour line after deformation. The image is then deformed to obtain the result image of the eye shape contour adjustment.

[0120] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium. When executed, the computer program can include the processes of the embodiments of the above methods. Furthermore, any references to memory, storage, databases, or other media used in the embodiments provided in this application can include at least one of non-volatile and volatile memory.

[0121] In summary, the method, system, device, and storage medium for removing eye fat from facial images of this invention utilize an innovative human portrait eye lighting and shadow smoothing algorithm combined with an eye shape contour adjustment algorithm. This allows even those without professional photo retouching knowledge to quickly and accurately retouch eye images, far exceeding the speed of manual retouching. Furthermore, it retains the advantages of manual retouching, such as natural lighting and shadows, reduction of swelling in the eye fat and orbicularis oculi muscle, and no impact on the eyebrows and eyes. It also exhibits seamlessness in both single-person and multi-person environments and lighting conditions, significantly improving image processing efficiency and the quality of eye retouching. Simultaneously, given the high risks and irreversible nature of cosmetic surgery, simulating eye fat removal surgery using image processing technology can assist consumers in their decision-making.

[0122] This invention discloses a method, system, device, and storage medium for removing eye fat from facial images. By using algorithms for smoothing light and shadow in the eyes and adjusting eye shape contours, it enables convenient and precise intelligent retouching of facial images with eye fat removal. Firstly, this invention significantly improves the efficiency of users' eye fat removal retouching, meeting their needs for intelligence and simplicity in eye retouching. Secondly, this invention can improve issues such as eye puffiness and drooping eyelids, satisfying clients' eye retouching requirements. Simultaneously, this invention meets the medical aesthetic requirements for eye fat removal in comprehensive medical eye care, allowing users to preview medical aesthetic eye fat removal solutions.

[0123] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A method of fat removal from the eye portion of a human face image, characterized by, The method comprises the following steps: An original image with a user's face is acquired, a mask image of an upper eyelid region is obtained according to face key points obtained through face detection, and the skin color of an eye region where the mask image is located is counted to obtain the average brightness of the skin color and calculate a weight of the skin color of the eye region for light and shadow flattening; An eye contour segmentation is performed on the upper eyelid part image to obtain a segmentation result image; The segmentation result image is converted into a binary image, connected domain detection is performed, and eyelid contours are screened out in combination with a face feature point set to obtain an initial contour line; The initial contour line is parameterized, a discrete point set is extracted for quadratic curve fitting to obtain a new contour line, and points on the new contour line are taken as coordinates of the points on the initial contour line after morphing, and the image is morphed to obtain an eye contour adjustment result image; The method for counting the skin color of the eye region where the mask image is located to obtain the average brightness of the skin color and calculate the weight of the skin color of the eye region for light and shadow flattening comprises the following steps: A skin color adjustment formula is used to obtain the skin color after light and shadow flattening; On the eye image Upper mask image Skin color statistics result Average brightness of the region ; The skin color of the region is adjusted by using a weighted formula of the average brightness of the skin color. the region. The weighting formula of the average brightness of the skin color is: Save the result image of the upper eyelid region light and shadow flat as ; The skin color adjustment formula is: wherein, is the luminance of the current pixel, is a skin tone adjustment weight; The method for converting the segmentation result image into a binary image, performing connected domain detection, screening out the eyelid contours in combination with the face feature point set, and obtaining the initial contour line is: wherein, is the current adjusted color, is a color within the neighborhood, is a skin tone adjustment weight for the color within the neighborhood; The method for obtaining the mask image of the upper eyelid region comprises the following steps: Traverse each pixel point in the binary image, and designate adjacent pixels of the same color as the same connected domain, denoted as connected domain , to obtain n connected domains; Based on the feature point set of the eyelid Feature points in Calculate the contour points of each connected component in turn. distance To obtain the minimum value The minimum value The outline of the corresponding connected region is the eyelid outline; each feature point , find the two nearest points on the connected domain contour, and take the two points as the points on the upper eyelid contour line, and all the points on the upper eyelid contour line form a contour line , the contour line of the connected domain containing the feature points of the human face of the human face is designated as the contour of the upper eyelid; wherein the contour line is parameterized, a discrete point set is extracted according to a step of 0.1, a quadratic curve fitting is performed on the point set, a new contour line is obtained, and points on the contour line are taken as the coordinates of the points on the contour line after deformation, and the image is deformed to obtain a result image of eye shape contour adjustment. 2.The method of claim 1, wherein: The method for counting the skin color of the eye region where the mask image is located to obtain the average brightness of the skin color comprises: Obtain a feature point set of a face in the original image through face detection Wherein, the original image with the face of the user is marked as ; Finding a feature point set of an eyelid part through indexing Finding a feature point set of an eyelid part through indexing Finding a feature point set of an eyelid part through indexing Finding a feature point set of an eyelid part through indexing According to the acquired eyelid partial image , judge whether it is a skin part, the skin part is an area needing to be processed, the non-skin part directly sets all channels to 0, and an eye image is obtained after processing ; Eye image The image is converted to a single-channel image and binarized. The skin portion of the processing area is set to 255 channels, while the non-skin portion, which does not need processing, is set to 0 channels, resulting in a mask image. . 3.The method of claim 2, wherein: The RGB values of each pixel point in the eye region of the acquired original image are traversed, and the average values of the colors are obtained by separately accumulating and averaging, and the average brightness is obtained by using an RGB and HSV mutual conversion algorithm; the original image is composed of data of an RGB channel. The method for performing eye contour segmentation on the upper eyelid part image to obtain the segmentation result image comprises the following steps:

4. The method for removing eye fat from a facial image as described in claim 3, characterized in that: The face image eye fat removal system adopts the face image eye fat removal method in any one of claims 1-4 to remove fat from a portrait eye; Based on Canny edge detection algorithm, the eyelid part image Fuzzy processing, fuzzy processing and Prewitt filter convolution processing, get horizontal and vertical derivative image, recorded as H, V; A gradient direction and magnitude are calculated for the current pixel of the eyelid portion, and when the magnitude exceeds a threshold value, an edge is assigned, and the image inside the edge is the segmented image to be retained .

5. A system for fat removal of eyes of a human face image, characterized by: The face image eye fat removal system comprises: An upper eyelid region light and shadow flattening module for processing an acquired original image with a user's face to obtain a result image of the upper eyelid region with flattened light and shadow; and An eye contour adjustment module for processing the upper eyelid part image to obtain an eye contour adjustment result image; The upper eyelid region light and shadow flattening module comprises a mask image generation unit and a weight calculation unit, the mask image generation unit is used to obtain a mask image of an upper eyelid region according to face key points of the original image obtained through face detection, and the weight calculation unit is used to count the skin color of an eye region where the mask image is located to obtain the average brightness of the skin color and calculate a weight of the skin color of the eye region for light and shadow flattening. ​ The eye contour adjustment module comprises an image segmentation unit, a contour line screening unit and a graphic contour morphing unit, the image segmentation unit is used for performing eye contour segmentation on an eyelid part image to obtain a segmentation result image; the contour line screening unit is used for converting the obtained segmentation result image into a binary image, performing connected domain detection, and screening out an eyelid contour in combination with a face feature point set to obtain an initial contour line; the graphic contour morphing unit is used for parameterizing the obtained initial contour line, extracting a discrete point set for quadratic curve fitting to obtain a new contour line, taking the points on the new contour line as the coordinates of the points on the initial contour line after morphing, morphing the image to obtain an eye type contour adjustment result image.

6. An eye fat removal apparatus comprising a memory and a processor, the memory storing a computer program, characterized in that, The computer program, when executed by the processor, implements the steps of the method of any one of claims 1 to 4.

7. A storage medium storing a computer program, characterized by The computer program, when executed by the processor, implements the steps of the method of any one of claims 1 to 4.

Images