Eye makeup processing method, device, electronic equipment and medium

Abstract

Embodiments of the present invention disclose an eye makeup processing method, apparatus, electronic device, and medium. The method comprises: constructing a current eye grid based on a currently input facial image; determining a current eye pixel movement information map based on the current eye grid, a reference eye grid, and a reference eye pixel movement information map; and determining a current eye makeup effect map for the facial image based on the current eye pixel movement information map and a reference makeup template map. By implementing this technical solution, it is possible to achieve the technical effect of precisely controlling eye shape while ensuring a tightly fitting eye makeup look.

Description

Technical Field

[0001] Embodiments of the present invention relate to the field of image processing, and in particular to an eye makeup processing method, device, electronic device, and medium. Background Art

[0002] On major live broadcast platforms, real-time makeup is one of the basic functions in live broadcast scenes. Makeup is an important factor that affects personal image, and changes in makeup will give people different feelings. Eye makeup is an important part of beauty makeup. Its main function is to adjust the light and shadow, correct protrusions and depressions, and shape the eye shape. In the existing technology, the eye contour is generally adjusted by moving the key points of the face, and the movement of the pixels around the key points is interpolated. Such eye shape adjustment cannot achieve precise control. The grid constructed by the key points around the eyes and the auxiliary key points derived based on the key points around the eyes is not accurate enough, resulting in the eye makeup not fitting well in certain conditions. Therefore, there is a need for an eye makeup processing method that can accurately control the eye shape while ensuring that the eye makeup fits tightly. Summary of the Invention

[0003] The embodiments of the present invention provide an eye makeup processing method, device, electronic device and medium to achieve the technical effect of accurately controlling the eye shape while ensuring that the eye makeup fits tightly.

[0004] In a first aspect, an embodiment of the present invention provides an eye makeup processing method, comprising:

[0005] Construct the current eye grid based on the current input face image;

[0006] Determining a current eye pixel movement information map based on the current eye grid, the reference eye grid, and the reference eye pixel movement information map;

[0007] A current eye makeup effect image of the face image is determined according to the current eye pixel movement information image and the reference makeup template image.

[0008] In a second aspect, an embodiment of the present invention further provides an eye makeup processing device, comprising:

[0009] A mesh construction module is used to construct the current eye mesh based on the current input face image;

[0010] an information map determining module, configured to determine a current eye pixel movement information map based on the current eye grid, the reference eye grid, and the reference eye pixel movement information map;

[0011] The effect diagram determination module is used to determine the current eye makeup effect diagram of the face image based on the current eye pixel movement information diagram and the reference makeup template diagram.

[0012] In a third aspect, an embodiment of the present invention further provides an electronic device, including:

[0013] one or more processors;

[0014] a storage device for storing one or more programs;

[0015] The one or more programs are executed by the one or more processors, so that the one or more processors implement the eye makeup processing method provided in any embodiment of the present invention.

[0016] In a fourth aspect, an embodiment of the present invention further provides a computer-readable storage medium on which a computer program is stored. When the program is executed by a processor, the eye makeup processing method provided in any embodiment of the present invention is implemented.

[0017] An embodiment of the present invention provides an eye makeup processing method, which constructs a current eye grid in a currently input facial image, determines a current eye pixel movement information map based on the current eye grid, a reference eye grid, and a reference eye pixel movement information map, and finally determines a current eye makeup effect map of the facial image based on the current eye pixel movement information map and a reference makeup template.

[0018] By adopting the technical solution of the present application, a current eye grid is constructed from the currently input facial image, which can ensure that it follows the changes in eye movements in real time. The current eye pixel movement information map is determined based on the current eye grid, the reference eye grid and the reference eye pixel movement information map. The current eye movement information map includes the element information of the eye shape change. Through the pixel movement map, it is possible to achieve precise control of the eye shape, and the constructed current eye grid can achieve the technical effect of close fit between the human eye and the eye makeup.

[0019] The above content of the invention is only an overview of the technical solution of the present invention. In order to more clearly understand the technical means of the present invention, it can be implemented in accordance with the contents of the specification. In order to make the above and other purposes, features and advantages of the present invention more obvious and easy to understand, the specific implementation methods of the present invention are listed below. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] Other features, objects, and advantages of the present invention will become more apparent upon reading the detailed description of non-limiting embodiments made with reference to the following drawings. The drawings are for the purpose of illustrating preferred embodiments only and are not to be considered as limiting the present invention. Like reference characters are used throughout the drawings to denote like parts. In the drawings:

[0021] Figure 1 This is a flow chart of an eye makeup processing method provided in Example 1 of the present application;

[0022] Figure 2This is a schematic diagram of the magnified results of a reference model's eyes provided in Example 1 of the present application;

[0023] Figure 3 This is a reference eye pixel movement information graph after the reference model's eyes are enlarged, provided in Example 1 of the present application;

[0024] Figure 4 This is a reference makeup template diagram provided in an embodiment of the present application;

[0025] Figure 5 This is an eye makeup effect diagram applied on a model diagram provided by an embodiment of the present application;

[0026] Figure 6 This is a schematic diagram of an eye mark provided in an embodiment of the present application;

[0027] Figure 7 This is a flow chart of another eye makeup processing method provided in Example 2 of the present application;

[0028] Figure 8 This is a schematic diagram of the first and second eye periocular key points provided in Example 2 of the present application;

[0029] Figure 9 This is a schematic diagram of the third and fourth key points around the eyes provided in Example 2 of the present application;

[0030] Figure 10 This is a schematic diagram of the first circle of auxiliary key points provided in Example 2 of the present application;

[0031] Figure 11 This is a schematic diagram of a second circle of auxiliary key points provided in Example 2 of the present application;

[0032] Figure 12 This is a schematic diagram of the third circle of auxiliary key points provided in Example 2 of the present application;

[0033] Figure 13 This is a schematic diagram of an eye grid provided in an embodiment of the present application;

[0034] Figure 14 This is a schematic diagram of an eye grid on a standard model provided by an embodiment of the present application;

[0035] Figure 15 This is a schematic diagram of the structure of the eye makeup processing device provided in Example 3 of the present application;

[0036] Figure 16 This is a structural diagram of an electronic device provided in Example 4 of the present application. DETAILED DESCRIPTION

[0037] The present invention will be further described in detail below with reference to the accompanying drawings and examples. It will be understood that the specific embodiments described herein are intended only to illustrate the present invention and are not intended to limit the present invention. It should also be noted that, for ease of description, the accompanying drawings only illustrate portions relevant to the present invention, not all structures.

[0038] It should be noted that before discussing exemplary embodiments in more detail, some exemplary embodiments are described as processes or methods depicted as flow charts. Although the flow charts describe the various operations (or steps) as sequential processes, many of the operations therein can be implemented in parallel, concurrently, or simultaneously. In addition, the order of the various operations can be rearranged. The process can be terminated when its operation is completed, but can also have additional steps not included in the accompanying drawings. The process can correspond to a method, function, procedure, subroutine, subprogram, etc.

[0039] Example 1

[0040] Figure 1 This is a flow chart of an eye makeup processing method provided in the first embodiment of the present invention. This method is applicable to the case of creating eye makeup effects on an input face image. This method can be executed by an eye makeup processing device, which can be implemented by software and / or hardware and can be integrated into an electronic device. Figure 1 As shown, the eye makeup processing method in this embodiment includes the following steps:

[0041] At present, in the beauty function of many live broadcast platforms, the eye makeup cannot be closely fitted to the eye contour in real time, and after the eyes are enlarged and deformed, it is more likely that the eye makeup cannot be closely fitted to the eyes in real time.

[0042] S110: Construct a current eye grid based on the currently input face image.

[0043] The eye makeup processing system performs image processing on the face image based on the current input face image, determines the eye contour, and constructs the current eye grid. The current eye grid can be a grid composed of multiple points obtained by recognizing and processing the eye contour according to the face image.

[0044] In an optional solution of this embodiment, which can be combined with one or more optional solutions of this embodiment, before constructing the current eye grid based on the currently input facial image, the method may include: magnifying the eyes of the reference model image, and determining a reference eye grid and a reference eye pixel movement information map based on the magnification result; and determining a makeup template map.

[0045] Before constructing the current eye mesh for the current input face image, the reference model image needs to be processed first to obtain the reference eye mesh, the reference eye pixel movement information map, and the eye makeup template map. The result after enlarging the eye of the reference model image is as follows: Figure 2 As shown, after the model's eyes are enlarged, the enlarged results are further Figure 2 Determine a reference eye grid and reference eye pixel information map for the enlarged model image. The reference eye grid can be a grid composed of multiple points obtained by processing the eye contours of the reference model. Determine a makeup template image that can be a professionally designed eye makeup image, including eyeshadow, eyeliner, and other effects.

[0046] Figure 3 This is a magnified version of a model's eye. The Eye Pixel Movement Information Chart is a three-channel, 8-bit color image. The R channel records left-right movement in the x-axis: values ​​greater than 127 indicate leftward movement, values ​​less than 127 indicate rightward movement, and values ​​equal to 127 indicate no movement. The magnitude of the movement is determined by the absolute value of the difference between the two values. The G channel records up-and-down movement in the y-axis: values ​​greater than 127 indicate upward movement, values ​​less than 127 indicate downward movement, and values ​​equal to 127 indicate no movement. The magnitude of the movement is determined by the absolute value of the difference between the two values. The B channel, with all values ​​set to 127, does not record any information. Therefore, the Eye Pixel Movement Information Chart records changes in eye contour by recording changes in pixel values. The R and G channels of the Pixel Movement Information Chart record changes in eye shape along the x- and y-axes, respectively. This allows for precise control of eye shape through the Eye Pixel Movement Information Chart.

[0047] By adopting the above technical solution, the technical effect of controlling the eye shape can be achieved by first magnifying the eyes of the reference model image before processing the facial image and determining the reference eye grid and the reference eye pixel movement information map based on the magnification result.

[0048] S120: Determine a current eye pixel movement information map according to the current eye grid, the reference eye grid, and the reference eye pixel movement information map.

[0049] Among them, the eye makeup processing system constructs a reference eye grid from a reference model image based on the current eye grid constructed from the face image, performs eye enlargement design in the reference model image, and obtains a reference eye pixel movement information map from the enlarged reference model image.

[0050] S130: Determine a current eye makeup effect image of the face image according to the current eye pixel movement information image and the reference makeup template image.

[0051] Among them, the eye makeup processing system determines the actual effect of the current eye makeup on the face image based on the current eye pixel movement information map and the reference makeup template map designed by professionals. Figure 4 This is a reference makeup template for an example.

[0052] In an optional solution of this embodiment, one or more optional solutions of this embodiment can be combined. Determining the current eye makeup effect image of the face image based on the current eye pixel movement information map and the reference makeup template map may include steps A1-A2:

[0053] Step A1: Determine a current eye makeup map based on the current eye grid, the reference eye grid, the face image, and the reference makeup template image.

[0054] Step A2: determining a current eye makeup effect map based on the current eye pixel movement information map and the current eye makeup map.

[0055] The eye makeup processing system determines the current eye makeup map based on the current eye grid, the eye grid of the reference model image, the face image, and the reference makeup template. The current eye makeup map can be the eye makeup map after adjusting the size, position, and deformation parameters of the eye makeup template image based on the face image. The current eye makeup effect map can be obtained based on the current eye pixel movement information map containing the eye shape adjustment information and the current eye makeup map. Figure 5 This is a schematic diagram of the final eye makeup effect applied on the model image.

[0056] In an optional solution of this embodiment, which can be combined with one or more optional solutions of this embodiment, before determining the current eye makeup effect image of the face image based on the current eye pixel movement information map and the reference makeup template map, steps B1-B2 may be included:

[0057] Step B1: controlling the degree of deformation of the two eyes based on the different degrees of opening and closing of the two eyes as adjustment parameters.

[0058] Step B2: updating and adjusting the current eye pixel movement information map according to the degree of deformation of the two eyes.

[0059] Among them, considering that the deformation information of eyes at different degrees of closure is different from the deformation information of eyes open in the standard model image, the deformation degree of closed eyes is smaller than that of open eyes. At the same time, considering that the degree of closure of the two eyes is likely to be different, the left and right eyes are calculated separately. Figure 6As shown in the figure, the four points are marked with U, D, L, and R. The ratio of the vectors UD to LR reflects the degree of eye opening and closing. Therefore, the ratio of the vectors UD to LR for the left and right eyes is used as an adjustment parameter to control the degree of deformation of the left and right eyes, resulting in an adjusted eye pixel displacement information map. Finally, this adjusted real-time information displacement map is combined with the real-time eye makeup image through a rendering algorithm to produce the real-time eye makeup effect.

[0060] By adopting the above technical solution, four points are marked on the upper, lower, left and right sides of the eye diagram respectively. The ratio of the vector formed by the upper and lower marked points to the vector formed by the left and right marked points is used to reflect the degree of opening and closing of the eyes. The ratio of the two eyes is used as an adjustment parameter to control the degree of deformation of the left and right eyes. The adjusted eye pixel movement information map is obtained to fit the eye makeup, so that the eye makeup fits better when the eyes are deformed.

[0061] The technical solution of this embodiment is to first magnify the eyes of a reference model image and obtain the eye grid and pixel movement information map of the reference model image, then construct a current eye grid based on the currently input facial image, determine the current eye pixel movement information map based on the current eye grid, the reference eye grid and the reference eye pixel movement information map, and determine the current eye makeup effect map of the facial image based on the current eye pixel movement information map and the reference makeup template. This can achieve the technical effect of ensuring that the eye makeup fits tightly while precisely controlling the eye shape.

[0062] Example 2

[0063] Figure 7 This is a flow chart of another eye makeup processing method provided by Example 2 of this application. This embodiment of the present invention further optimizes the above embodiment on the basis of the above embodiment. This embodiment of the present invention can be combined with various optional solutions in one or more of the above embodiments. Figure 7 As shown, the eye makeup processing method provided in the embodiment of the present invention may include the following steps:

[0064] S710: Determine the coordinates of key points around the eyes of the face image.

[0065] Among them, the eye makeup processing system first determines the key points around the eyes, which can be the key points on the eye sockets of the face image.

[0066] In an optional solution of this embodiment, one or more optional solutions of this embodiment can be combined. Determining the coordinates of key points around the eyes of a face image may include steps C1-C2:

[0067] Step C1: obtaining the first peri-eye key point by performing face recognition on the real-time input face image.

[0068] Step C2: interpolate the first periocular key point using a third-order Bezier curve to obtain a second periocular key point, a third periocular key point, and a fourth periocular key point.

[0069] Among them, Figure 8 As shown, A0-A7 can be the first eye key point obtained by face recognition. By using the third-order Bezier curve for interpolation, the interpolation parameter is 0.5 between the first eye key points A0-A7 to obtain Figure 8 The second eye key point B0-B7 in the image is then interpolated with the interpolation parameters of 0.25 and 0.75 respectively. Figure 9 The third periocular key points C0-C7 and the fourth periocular key points D0-D7.

[0070] S720: Determine the coordinates of the auxiliary key points based on the coordinates of the eye periorbital key points.

[0071] Among them, the eye makeup processing system determines the coordinates of the auxiliary key points based on the first, second, third and fourth eye periorbital key points.

[0072] In an optional solution of this embodiment, one or more optional solutions of this embodiment can be combined. Determining the coordinates of the auxiliary key points based on the coordinates of the eye periphery key points may include steps D1-D4:

[0073] Step D1: Move the third and fourth periorbital key points by a first preset distance in the direction of a first midpoint vector to obtain a first circle of auxiliary key points, where the first midpoint vector is a vector from the midpoint of a reference vector to the third and fourth periorbital key points, and the reference vector is a vector from the left corner of the eye to the right corner of the eye.

[0074] Step D2: Move the first and second periorbital key points by a second preset distance in the direction of a second midpoint vector to obtain a second circle of auxiliary key points, where the second midpoint vector is a vector from the midpoint of the reference vector to the first and second periorbital key points.

[0075] Step D3: Move the lower eyelid key points of the first and second eye periorbital key points by a third preset distance along the direction of the third midpoint vector to obtain the middle auxiliary point of the lower eyelid, where the third midpoint vector is the vector from the midpoint of the reference vector to the lower eyelid key points of the first and second eye periorbital key points; move the upper eyelid key points of the third and fourth eye periorbital key points by a third preset distance along the direction of the fourth midpoint vector to obtain the middle auxiliary point of the upper eyelid, where the fourth midpoint vector is the vector from the midpoint of the reference vector to the upper eyelid key points of the third and fourth eye periorbital key points.

[0076] Step D4: determining the third circle of auxiliary key points and the fourth circle of auxiliary key points based on the middle auxiliary point of the upper eyelid, the middle auxiliary point of the lower eyelid, and the eyelid following degree coefficient.

[0077] The reference vector can be Figure 8 The first midpoint vector can be the vector from the midpoint of the reference vector A0A4 to the third periorbital key point C0-C7 and the fourth periorbital key point D0-D7. The first preset distance moved from the third periorbital key point C0-C7 and the fourth periorbital key point D0-D7 can be the modulus of the reference vector multiplied by the super coefficient. The super coefficient is set according to actual needs. Thus, the first circle of auxiliary key points is obtained by moving the first preset distance from the third periorbital key point and the fourth periorbital key point, as shown in FIG. Figure 10 As shown, the points on L1 and L2 are the auxiliary key points of the first circle.

[0078] The second midpoint vector can be a vector from the midpoint of the reference vector A0A4 to the first periorbital key point A0-A7 and the second periorbital key point B0-B7. The second preset distance can be a super coefficient multiplied by the modulus of the reference vector. The super coefficient is set according to actual needs. Generally, the super coefficient of the second preset distance is greater than the super coefficient of the first preset distance. Thus, the second circle of auxiliary key points is obtained by moving the first preset distance from the first periorbital key point and the second periorbital key point respectively. Figure 11 As shown, the points on L3 and L4 are the auxiliary key points of the second circle.

[0079] The first eyelid key point and the second eyelid key point are at Figure 8 The third midpoint vector can be represented as A4-A7 and B4-B7. The third midpoint vector can be the vector from the midpoint of the reference vector A0A4 to the lower eyelid key point A4-A7 of the first periocular key point and the lower eyelid key point B4-B7 of the second periocular key point. The third preset distance can be the modulus of the reference vector multiplied by the super coefficient. In practical applications, the super coefficient of the third preset distance is generally greater than the super coefficient of the second preset distance. The lower eyelid key point A4-A7 of the first periocular key point and the lower eyelid key point B4-B7 of the second periocular key point are moved along the direction of the third midpoint vector by the third preset distance to obtain the lower eyelid middle auxiliary point, which can be represented as U1-U8.

[0080] The third eyelid key point and the fourth eyelid key point are at the upper eyelid key point Figure 9The fourth midpoint vector can be represented as C0-C3 and D0-D3. The fourth midpoint vector can be the vector from the midpoint of the reference vector A0A4 to the upper eyelid keypoint C0-C3 of the third eye periorbital keypoint and the upper eyelid keypoint D0-D3 of the fourth eye periorbital keypoint. The upper eyelid keypoint C0-C3 of the third eye periorbital keypoint and the upper eyelid keypoint D0-D3 of the fourth eye periorbital keypoint are moved a third preset distance in the direction of the fourth midpoint vector to obtain the upper eyelid middle auxiliary point, which can be represented as N1-N8.

[0081] Considering that when the eyes are closed, the effect of eye shadow near the eye frame area following the upper eyelid should be weakened, so M is set to R*U+(1-R)*N. In practical applications, when R=0.625, M can be used as the third circle of auxiliary key points. When R=0, M can be used as the fourth circle of auxiliary key points, such as Figure 12 As shown, L5 is the third circle auxiliary key point, such as Figure 13 As shown in the schematic diagram of the eye grid, in addition to the auxiliary key points on L1, L2, L3, L4, and L5, the outermost auxiliary key points of the eye contour are the fourth circle of auxiliary key points.

[0082] By adopting the above technical solution, by determining the first eye periorbital key point, and then determining the second, third and fourth eye periorbital key points based on the third-order Bezier curve, and then determining the first, second, third and fourth circles of auxiliary key points through the eye periorbital key points, the technical effect of constructing an eye mesh that can make the eye makeup effect and the eye contour fit closely is achieved.

[0083] S730: Construct a current eye mesh based on the eye periphery key point coordinates and the auxiliary key point coordinates.

[0084] Among them, Figure 13 As shown in the eye mesh diagram, the first, second, third and fourth eye periphery key points and the first, second, third and fourth circle auxiliary key points are combined to construct the current eye mesh. The diagram of the eye mesh applied to a standard model is shown in Figure 14 shown.

[0085] S740: Determine a current eye pixel movement information map according to the current eye grid, the reference eye grid, and the reference eye pixel movement information map.

[0086] S750: Determine a current eye makeup effect image of the face image according to the current eye pixel movement information image and the reference makeup template image.

[0087] The technical solution of this embodiment determines the coordinates of the key points around the eyes of the facial image, then determines the coordinates of the auxiliary key points based on the coordinates of the key points around the eyes, constructs a complete current eye grid based on the coordinates of the key points around the eyes and the auxiliary key points, determines the current eye pixel movement information map based on the constructed current eye grid, and finally determines the current eye makeup effect map of the facial image based on the current eye pixel movement information map and the reference makeup template, thereby achieving the technical effect of ensuring that the eye makeup fits tightly while precisely controlling the eye shape.

[0088] Example 3

[0089] Figure 15 This is a schematic diagram of the structure of an eye makeup processing device provided in Example 3 of the present invention. This device is applicable to creating eye makeup effects on input facial images. The device can be implemented using software and / or hardware and integrated into an electronic device. This device is used to implement the eye makeup processing method provided in the above embodiment.

[0090] like Figure 15 As shown, the eye makeup processing device provided in this embodiment includes:

[0091] A grid construction module 151 is used to construct a current eye grid based on the currently input face image;

[0092] an information map determining module 152 for determining a current eye pixel movement information map based on the current eye grid, the reference eye grid, and the reference eye pixel movement information map;

[0093] The effect image determining module 153 is used to determine the current eye makeup effect image of the face image based on the current eye pixel movement information image and the reference makeup template image.

[0094] Based on the above embodiment, optionally, the grid construction module 151 is configured to:

[0095] The eyes of the reference model image are enlarged, and a reference eye grid and a reference eye pixel movement information map are determined based on the enlargement result; and a makeup template map is determined.

[0096] Based on the above embodiment, optionally, the grid construction module 151 is further configured to:

[0097] Determine the coordinates of key points around the eyes of the face image;

[0098] Determining the coordinates of the auxiliary key points based on the coordinates of the eye periorbital key points;

[0099] The current eye mesh is constructed according to the eye periphery key point coordinates and the auxiliary key point coordinates.

[0100] Based on the above embodiment, optionally, the grid construction module 151 is further configured to:

[0101] The first peri-ocular key points are obtained by performing face recognition on the real-time input face image;

[0102] The first periocular key point is interpolated using a third-order Bezier curve to obtain a second periocular key point, a third periocular key point, and a fourth periocular key point.

[0103] Based on the above embodiment, optionally, the grid construction module 151 is further configured to:

[0104] Move the third and fourth eye periorbital key points by a first preset distance in the direction of a first midpoint vector to obtain a first circle of auxiliary key points, where the first midpoint vector is a vector from the midpoint of a reference vector to the third and fourth eye periorbital key points, and the reference vector is a vector from the left corner of the eye to the right corner of the eye;

[0105] Move the first and second periorbital key points by a second preset distance in the direction of a second midpoint vector to obtain a second circle of auxiliary key points, where the second midpoint vector is a vector from the midpoint of the reference vector to the first and second periorbital key points;

[0106] The lower eyelid key points of the first and second eye periorbital key points are moved by a third preset distance in the direction of a third midpoint vector to obtain a lower eyelid middle auxiliary point, where the third midpoint vector is a vector from the midpoint of a reference vector to the lower eyelid key points of the first and second eye periorbital key points; the upper eyelid key points of the third and fourth eye periorbital key points are moved by a third preset distance in the direction of a fourth midpoint vector to obtain an upper eyelid middle auxiliary point, where the fourth midpoint vector is a vector from the midpoint of a reference vector to the upper eyelid key points of the third and fourth eye periorbital key points;

[0107] The third circle of auxiliary key points and the fourth circle of auxiliary key points are determined based on the middle auxiliary point of the upper eyelid, the middle auxiliary point of the lower eyelid and the eyelid following degree coefficient.

[0108] Based on the above embodiment, optionally, the effect diagram determination module 153 is configured to:

[0109] Determining a current eye makeup map according to the current eye grid, the reference eye grid, the face image, and the reference makeup template image;

[0110] A current eye makeup effect map is determined according to the current eye pixel movement information map and the current eye makeup map.

[0111] Based on the above embodiment, optionally, the effect diagram determining module 153 is further configured to:

[0112] The degree of deformation of the two eyes is controlled according to the different opening and closing degrees of the two eyes as adjustment parameters;

[0113] The current eye pixel movement information map is updated and adjusted according to the degree of deformation of the two eyes.

[0114] The eye makeup processing device provided in the embodiment of the present invention can execute the eye makeup processing method provided in any embodiment of the present invention mentioned above, and has the corresponding functions and beneficial effects of executing the eye makeup processing method. For detailed process, please refer to the relevant operations of the eye makeup processing method in the aforementioned embodiment.

[0115] Example 4

[0116] Figure 16 This is a structural diagram of an electronic device provided in the fourth embodiment of the present application. This embodiment of the present application provides an electronic device, in which the eye makeup processing device provided in the embodiment of the present application can be integrated. Figure 16 As shown, this embodiment provides an electronic device 160, which includes: one or more processors 162; a storage device 161 for storing one or more programs. When the one or more programs are executed by the one or more processors 162, the one or more processors 162 implement the eye makeup processing method provided in the embodiment of the present application. The method includes:

[0117] Construct the current eye grid based on the current input face image;

[0118] Determining a current eye pixel movement information map based on the current eye grid, the reference eye grid, and the reference eye pixel movement information map;

[0119] A current eye makeup effect image of the face image is determined according to the current eye pixel movement information image and the reference makeup template image.

[0120] Of course, those skilled in the art will appreciate that the processor 162 also implements the technical solution of the eye makeup processing method provided in any embodiment of the present application.

[0121] Figure 16 The electronic device 160 shown is merely an example and should not limit the functions and scope of use of the embodiments of the present application.

[0122] like Figure 16 As shown, the electronic device 160 includes a processor 162, a storage device 161, an input device 163 and an output device 164; the number of processors 162 in the electronic device can be one or more. Figure 16In the figure, a processor 162 is used as an example; the processor 162, the storage device 161, the input device 163 and the output device 164 in the electronic device can be connected by a bus or other means. Figure 16 The connection via bus 165 is taken as an example.

[0123] The storage device 161 is a computer-readable storage medium that can be used to store software programs, computer executable programs, and module units, such as program instructions corresponding to the eye makeup processing method in the embodiment of the present application.

[0124] The storage device 161 may primarily include a program storage area and a data storage area. The program storage area may store an operating system and at least one application required for a function; the data storage area may store data generated based on the use of the terminal. Furthermore, the storage device 161 may include high-speed random access memory and non-volatile memory, such as at least one disk storage device, flash memory device, or other non-volatile solid-state memory device. In some instances, the storage device 161 may further include memory remotely located relative to the processor 162, and such remote memory may be connected via a network. Examples of such networks include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0125] The input device 163 may be used to receive input numbers, character information or voice information, and generate key signal input related to user settings and function control of the electronic device. The output device 164 may include electronic devices such as a display screen and a speaker.

[0126] The electronic device provided in the embodiment of the present application can achieve the technical effect of accurately controlling the eye shape while ensuring that the eye makeup fits tightly.

[0127] Example 5

[0128] A fifth embodiment of the present invention provides a computer-readable medium having a computer program stored thereon. When the program is executed by a processor, the computer-readable medium is used to perform an eye makeup processing method. The method includes:

[0129] Construct the current eye grid based on the current input face image;

[0130] Determining a current eye pixel movement information map based on the current eye grid, the reference eye grid, and the reference eye pixel movement information map;

[0131] A current eye makeup effect image of the face image is determined according to the current eye pixel movement information image and the reference makeup template image.

[0132] Optionally, when the program is executed by a processor, it can also be used to execute the eye makeup processing method provided in any embodiment of the present invention.

[0133] The computer storage medium of the embodiment of the present invention may adopt any combination of one or more computer-readable media. The computer-readable medium may be a computer-readable signal medium or a computer-readable storage medium. The computer-readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or component, or any combination thereof. More specific examples (a non-exhaustive list) of computer-readable storage media include: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM), a flash memory, an optical fiber, a portable CD-ROM, an optical storage device, a magnetic storage device, or any suitable combination thereof. The computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, device or device.

[0134] A computer-readable signal medium may include a data signal propagated in baseband or as part of a carrier wave, carrying computer-readable program code. Such a propagated data signal may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. A computer-readable signal medium may also be any computer-readable medium other than a computer-readable storage medium that can transmit, propagate, or transport a program for use by or in conjunction with an instruction execution system, apparatus, or device.

[0135] The program code contained on the computer-readable medium may be transmitted using any appropriate medium, including but not limited to wireless, wire, optical cable, radio frequency (RF), etc., or any suitable combination of the foregoing.

[0136] Computer program code for performing the operations of the present invention may be written in one or more programming languages, or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, C++, and conventional procedural programming languages ​​such as "C" or similar programming languages. The program code may be executed entirely on the user's computer, partially on the user's computer, as a stand-alone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving a remote computer, the remote computer may be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., through the Internet using an Internet service provider).

[0137] Throughout this specification, reference to terms such as "one embodiment," "some embodiments," "examples," "specific examples," or "some examples" means that a specific feature, structure, material, or characteristic described in conjunction with that embodiment or example is included in at least one embodiment or example of the present invention. In this specification, schematic representations of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in any one or more embodiments or examples.

[0138] Note that the above are only preferred embodiments of the present invention and the technical principles employed. Those skilled in the art will understand that the present invention is not limited to the specific embodiments described herein, and that various obvious changes, readjustments, and substitutions can be made by those skilled in the art without departing from the scope of protection of the present invention. Therefore, although the present invention has been described in detail through the above embodiments, the present invention is not limited to the above embodiments and may include many other equivalent embodiments without departing from the concept of the present invention. The scope of the present invention is determined by the scope of the appended claims.

Claims

1. An eye makeup processing method, characterized in that: include: Construct the current eye grid based on the current input face image; Determining a current eye pixel movement information map based on the current eye grid, the reference eye grid, and the reference eye pixel movement information map; Determining a current eye makeup effect image of the face image based on the current eye pixel movement information image and the reference makeup template image; The current eye mesh is constructed based on the current input face image, including: The first peri-ocular key points are obtained by performing face recognition on the real-time input face image; Interpolating the first periorbital key point using a third-order Bezier curve to obtain a second periorbital key point, a third periorbital key point, and a fourth periorbital key point; Move the third and fourth eye periorbital key points by a first preset distance in the direction of a first midpoint vector to obtain a first circle of auxiliary key points, where the first midpoint vector is a vector from the midpoint of a reference vector to the third and fourth eye periorbital key points, and the reference vector is a vector from the left corner of the eye to the right corner of the eye; Move the first and second periorbital key points by a second preset distance in the direction of a second midpoint vector to obtain a second circle of auxiliary key points, where the second midpoint vector is a vector from the midpoint of the reference vector to the first and second periorbital key points; The lower eyelid key points of the first and second eye periorbital key points are moved by a third preset distance in the direction of a third midpoint vector to obtain a lower eyelid middle auxiliary point, where the third midpoint vector is a vector from the midpoint of a reference vector to the lower eyelid key points of the first and second eye periorbital key points; the upper eyelid key points of the third and fourth eye periorbital key points are moved by a third preset distance in the direction of a fourth midpoint vector to obtain an upper eyelid middle auxiliary point, where the fourth midpoint vector is a vector from the midpoint of a reference vector to the upper eyelid key points of the third and fourth eye periorbital key points; Determining a third circle of auxiliary key points and a fourth circle of auxiliary key points based on the upper eyelid middle auxiliary point, the lower eyelid middle auxiliary point, and the eyelid following degree coefficient; Constructing a current eye mesh according to the eye periphery key point coordinates and the auxiliary key point coordinates; The third circle auxiliary key points and the fourth circle auxiliary key points are determined according to the following formula: M=R*U+(1-R)*N; Among them, if R=0.625, M is the third circle auxiliary key point; when R=0, M is the fourth circle auxiliary key point; U is the middle auxiliary point of the lower eyelid, and N is the middle auxiliary point of the upper eyelid.

2. The method according to claim 1, characterized in that Before constructing the current eye mesh based on the current input face image, it includes: The eyes of the reference model image are enlarged, and a reference eye grid and a reference eye pixel movement information map are determined based on the enlargement result; and a makeup template map is determined.

3. The method according to claim 1, characterized in that Determining a current eye makeup effect image of the face image based on the current eye pixel movement information image and the reference makeup template image includes: Determining a current eye makeup map according to the current eye grid, the reference eye grid, the face image, and the reference makeup template image; A current eye makeup effect map is determined according to the current eye pixel movement information map and the current eye makeup map.

4. The method according to claim 1 or 3, characterized in that Before determining the current eye makeup effect image of the face image based on the current eye pixel movement information image and the reference makeup template image, the method includes: The degree of deformation of the two eyes is controlled according to the different opening and closing degrees of the two eyes as adjustment parameters; The current eye pixel movement information map is updated and adjusted according to the degree of deformation of the two eyes.

5. An eye makeup processing device, characterized in that: The device comprises: A mesh construction module is used to construct the current eye mesh based on the current input face image; an information map determining module, configured to determine a current eye pixel movement information map based on the current eye grid, the reference eye grid, and the reference eye pixel movement information map; An effect image determining module, configured to determine a current eye makeup effect image of the face image based on the current eye pixel movement information image and the reference makeup template image; Wherein, the grid construction module is further used for: Determine the coordinates of key points around the eyes of the face image; Determining the coordinates of the auxiliary key points based on the coordinates of the eye periorbital key points; Constructing a current eye mesh according to the eye periphery key point coordinates and the auxiliary key point coordinates; The grid construction module is further used to: The first peri-ocular key points are obtained by performing face recognition on the real-time input face image; Interpolating the first periorbital key point using a third-order Bezier curve to obtain a second periorbital key point, a third periorbital key point, and a fourth periorbital key point; The grid construction module is further used to: Move the third and fourth eye periorbital key points by a first preset distance in the direction of a first midpoint vector to obtain a first circle of auxiliary key points, where the first midpoint vector is a vector from the midpoint of a reference vector to the third and fourth eye periorbital key points, and the reference vector is a vector from the left corner of the eye to the right corner of the eye; Move the first and second periorbital key points by a second preset distance in the direction of a second midpoint vector to obtain a second circle of auxiliary key points, where the second midpoint vector is a vector from the midpoint of the reference vector to the first and second periorbital key points; The lower eyelid key points of the first and second eye periorbital key points are moved by a third preset distance in the direction of a third midpoint vector to obtain a lower eyelid middle auxiliary point, where the third midpoint vector is a vector from the midpoint of a reference vector to the lower eyelid key points of the first and second eye periorbital key points; the upper eyelid key points of the third and fourth eye periorbital key points are moved by a third preset distance in the direction of a fourth midpoint vector to obtain an upper eyelid middle auxiliary point, where the fourth midpoint vector is a vector from the midpoint of a reference vector to the upper eyelid key points of the third and fourth eye periorbital key points; Determining a third circle of auxiliary key points and a fourth circle of auxiliary key points based on the upper eyelid middle auxiliary point, the lower eyelid middle auxiliary point, and the eyelid following degree coefficient; The third circle auxiliary key points and the fourth circle auxiliary key points are determined according to the following formula: M=R*U+(1-R)*N; Among them, if R=0.625, M is the third circle auxiliary key point; when R=0, M is the fourth circle auxiliary key point; U is the middle auxiliary point of the lower eyelid, and N is the middle auxiliary point of the upper eyelid.

6. An electronic device, characterized in that: include: one or more processors; a storage device for storing one or more programs; When the one or more programs are executed by the one or more processors, the one or more processors implement the eye makeup processing method according to any one of claims 1 to 4.

7. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the eye makeup processing method described in any one of claims 1 to 4 is implemented.

Images