Image Processing Method, Apparatus and Device

Abstract

An embodiment of the present application provides an image processing method, apparatus, and device. The method includes: obtaining an image to be processed, where the image to be processed includes an eye image, and the eye image includes a pupil image; determining the identifiers and positions of a plurality of key points corresponding to the eye image according to the image to be processed; determining the size of the pupil image according to the identifiers and positions of the plurality of key points; obtaining a colored contact lens style image; and determining a colored contact lens try-on image according to the identifiers and positions of the plurality of key points, the size, the colored contact lens style image, and the image to be processed. The image processing method, apparatus, and device provided by the embodiments of the present application can improve the try-on effect of colored contact lenses.

Description

Technical Field

[0001] This application relates to image processing technology, and in particular, to an image processing method, apparatus, and device. Background Art

[0002] Colored contact lenses are a type of glasses used to modify the iris texture of the eye pupil and enhance the beauty of the eye pupil. Usually, when a user selects colored contact lenses, it is impossible to try them on.

[0003] In this regard, in the related art, a preset colored contact lens image is output in a first area so that the preset colored contact lens image is mapped onto the user's eyes to achieve trying on of colored contact lenses, and a captured portrait image is output in a second area. The second area is superimposed on the first area, and the preset colored contact lens image is mapped onto the user's eyes to achieve trying on of colored contact lenses.

[0004] In the above-mentioned related art, a preset colored contact lens image is output in the first area, and a captured portrait image is output in the second area. The second area is superimposed on the first area to achieve trying on of colored contact lenses, resulting in poor fitting between the colored contact lenses and the eye pupil and a poor trying-on effect of the colored contact lenses. Summary of the Invention

[0005] Embodiments of this application provide an image processing method, apparatus, and device to solve the problem of poor trying-on effect of colored contact lenses.

[0006] In a first aspect, embodiments of this application provide an image processing method, including:

[0007] Obtain an image to be processed, where the image to be processed includes an eye image, and the eye image includes an eye pupil image;

[0008] Determine the identifiers and positions of multiple key points corresponding to the eye image according to the image to be processed;

[0009] Determine the size of the eye pupil image according to the identifiers and positions of the multiple key points;

[0010] Obtain a colored contact lens style image;

[0011] Determine a colored contact lens trying-on image according to the identifiers and positions of the multiple key points, the size, the colored contact lens style image, and the image to be processed.

[0012] In a possible design, determining a colored contact lens trying-on image according to the identifiers and positions of the multiple key points, the size, the colored contact lens style image, and the image to be processed includes:

[0013] Determine a colored contact lens fusion mask according to the identifiers and positions of the multiple key points, the size, and the colored contact lens style image;

[0014] Obtain the makeup intensity;

[0015] Perform adaptive weight fusion processing on the image to be processed and the colored contact lens style image according to the colored contact lens fusion mask and the makeup trial strength to obtain a colored contact lens trial-wearing image.

[0016] In a possible design, determine the colored contact lens fusion mask according to the identification, position, size of multiple key points, and the colored contact lens style image, including:

[0017] Determine the iris binary mask according to the identification, position, size of multiple key points, and the colored contact lens style image;

[0018] Perform filtering processing on the iris binary mask according to the size to obtain the colored contact lens fusion mask.

[0019] In a possible design, determine the iris binary mask according to the identification, position, size of multiple key points, and the colored contact lens style image, including:

[0020] Determine the eye region binary mask according to the identification and position of multiple key points;

[0021] Perform scaling processing on the colored contact lens style image according to the size to obtain a scaled style image;

[0022] Perform transparency analysis processing on the scaled style image to obtain the iris region binary mask;

[0023] Determine the iris binary mask according to the eye region binary mask and the iris region binary mask.

[0024] In a possible design, determine the size of the eye pupil image according to the identification and position of multiple key points, including:

[0025] Respectively determine the eye pupil region binary mask and the center position of the eye pupil image according to the identification and position of multiple key points;

[0026] Determine the size according to the eye pupil region binary mask, the center position, the identification and position of multiple key points.

[0027] In a possible design, determine the size according to the eye pupil region binary mask, the center position, the identification and position of multiple key points, including:

[0028] Search for the pixels passed by the first direction in the eye pupil region binary mask along the first direction pointing from the center position to the second position to obtain the first edge position of the eye pupil in the first direction; the second position is the position corresponding to the key point with the second preset identification among multiple key points;

[0029] Search for the pixels passed by the second direction in the binary mask of the eye pupil region along the second direction pointing from the central position to the third position, and obtain the second edge position of the eye pupil in the second direction; the third position is the position corresponding to the key point with the third preset identifier among the multiple key points.

[0030] Determine the first distance between the first edge position and the central position, and the second distance between the second edge position and the central position respectively;

[0031] Determine the size according to the first distance and the second distance.

[0032] In a possible design, determining the identifiers and positions of multiple key points corresponding to the eye image according to the image to be processed includes:

[0033] Perform face recognition on the image to be processed through a preset face recognition model to obtain the face image included in the image to be processed;

[0034] Perform key point detection on the face image through a preset key point detection model to obtain the identifiers and positions of multiple key points corresponding to the eye image.

[0035] In a possible design, obtaining the colored contact lens style image includes:

[0036] Display at least one colored contact lens type;

[0037] In response to the user's first selection operation on the target type among the at least one colored contact lens type, obtain at least one style image corresponding to the target type from the preset database, and display at least one style image;

[0038] In response to the user's second selection operation on the target style image among the at least one style image, determine the target style image as the colored contact lens style image.

[0039] In a second aspect, an embodiment of the present application provides an image processing device, including:

[0040] A first acquisition module, configured to acquire an image to be processed, where the image to be processed includes an eye image, and the eye image includes an eye pupil image;

[0041] A first determination module, configured to determine the identifiers and positions of multiple key points corresponding to the eye image according to the image to be processed;

[0042] A second determination module, configured to determine the size of the eye pupil image according to the identifiers and positions of the multiple key points;

[0043] A second acquisition module, configured to acquire a colored contact lens style image;

[0044] A third determination module, configured to determine a colored contact lens trial image according to the identifiers, positions, sizes, colored contact lens style images, and images to be processed of multiple key points.

[0045] In a possible design, the third determination module is specifically configured to:

[0046] Determine a colored contact lens fusion mask according to the identifiers, positions, sizes, and colored contact lens style images of multiple key points;

[0047] Obtain the trial makeup intensity;

[0048] Perform adaptive weighted fusion processing on the image to be processed and the colored contact lens style image according to the colored contact lens fusion mask and the trial makeup intensity to obtain a colored contact lens trial image.

[0049] In a possible design, the third determination module is specifically configured to:

[0050] Determine an iris binary mask according to the identifiers, positions, sizes, and colored contact lens style images of multiple key points;

[0051] Perform filtering processing on the iris binary mask according to the size to obtain a colored contact lens fusion mask.

[0052] In a possible design, the third determination module is specifically configured to:

[0053] Determine an eye region binary mask according to the identifiers and positions of multiple key points;

[0054] Perform scaling processing on the colored contact lens style image according to the size to obtain a scaled style image;

[0055] Perform transparency analysis processing on the scaled style image to obtain an iris region binary mask;

[0056] Determine an iris binary mask according to the eye region binary mask and the iris region binary mask.

[0057] In a possible design, the second determination module is specifically configured to:

[0058] Respectively determine a pupil region binary mask and the central position of the pupil image according to the identifiers and positions of multiple key points;

[0059] Determine the size according to the pupil region binary mask, the central position, the identifiers and positions of multiple key points.

[0060] In a possible design, the second determination module is specifically configured to:

[0061] Search for the pixels passed by the first direction in the binary mask of the pupil area along the first direction pointing from the central position to the second position, and obtain the first edge position of the pupil in the first direction; the second position is the position corresponding to the key point with the second preset identifier among the multiple key points;

[0062] Search for the pixels passed by the second direction in the binary mask of the pupil area along the second direction pointing from the central position to the third position, and obtain the second edge position of the pupil in the second direction; the third position is the position corresponding to the key point with the third preset identifier among the multiple key points;

[0063] Respectively determine the first distance between the first edge position and the central position, and the second distance between the second edge position and the central position;

[0064] Determine the size according to the first distance and the second distance.

[0065] In a possible design, the first determination module is specifically configured to:

[0066] Perform face recognition on the image to be processed through a preset face recognition model, and obtain the face image included in the image to be processed;

[0067] Perform key point detection on the face image through a preset key point detection model, and obtain the identifiers and positions of multiple key points corresponding to the eye image.

[0068] In a possible design, the first acquisition module is specifically configured to:

[0069] Display at least one colored contact lens type;

[0070] In response to the user's first selection operation on the target type among at least one colored contact lens type, obtain at least one style image corresponding to the target type from the preset database, and display at least one style image;

[0071] In response to the user's second selection operation on the target style image among at least one style image, determine the target style image as the colored contact lens style image.

[0072] In a third aspect, an embodiment of the present application provides an electronic device, including: a processor, and a memory communicatively connected to the processor;

[0073] The memory stores computer execution instructions;

[0074] The processor executes the computer execution instructions stored in the memory to implement the image processing method of any one of the first aspects.

[0075] Fourthly, an embodiment of the present application provides a computer-readable storage medium storing computer-executable instructions, which are used to implement the image processing method according to any one of the first aspect when executed by a processor.

[0076] Fifthly, an embodiment of the present application provides a computer program product including a computer program, which implements the image processing method according to any one of the first aspect when executed by a processor.

[0077] An embodiment of the present application provides an image processing method, device and equipment. The method includes: obtaining an image to be processed, where the image to be processed includes an eye image, and the eye image includes a pupil image; determining the identifiers and positions of multiple key points corresponding to the eye image according to the image to be processed; determining the size of the pupil image according to the identifiers and positions of the multiple key points; obtaining a colored contact lens style image; and determining a colored contact lens try-on image according to the identifiers and positions of the multiple key points, the size, the colored contact lens style image and the image to be processed. In the above method, determining the size of the pupil image according to the identifiers and positions of the multiple key points, and determining the colored contact lens try-on image according to the identifiers and positions of the multiple key points, the size and the colored contact lens style image can improve the fitting degree between the colored contact lens style image and the pupil, and further improve the try-on effect of the colored contact lens. Description of the Drawings

[0078] The drawings here are incorporated into the description and form a part of this description, showing embodiments consistent with the present application, and are used together with the description to explain the principles of the present application.

[0079] Figure 1 It is a flowchart of the image processing method provided by the embodiment of the present application;

[0080] Figure 2 It is a schematic diagram of facial key points provided by the embodiment of the present application;

[0081] Figure 3 It is a flowchart of determining the size of the pupil image provided by the embodiment of the present application;

[0082] Figure 4 It is a filled image corresponding to the left eye image provided by the embodiment of the present application;

[0083] Figure 5 It is a flowchart of determining the colored contact lens try-on image provided by the embodiment of the present application;

[0084] Figure 6 It is a try-on effect diagram provided by the embodiment of the present application;

[0085] Figure 7 It is a structural diagram of the image processing device provided by the embodiment of the present application;

[0086] Figure 8 This is a schematic diagram of the hardware of the electronic device provided by the embodiment of the present application.

[0087] Through the above-mentioned drawings, specific embodiments of the present application have been shown, and more detailed descriptions will be given later. These drawings and textual descriptions are not intended to limit the scope of the concept of the present application in any way, but to illustrate the concept of the present application to those skilled in the art by referring to specific embodiments. Detailed implementation manners

[0088] Here, the exemplary embodiments will be described in detail, and the examples are shown in the drawings. When the following description refers to the drawings, unless otherwise indicated, the same numbers in different drawings represent the same or similar elements. The implementation manners described in the following exemplary embodiments do not represent all implementation manners consistent with the present application. On the contrary, they are merely examples of devices and methods consistent with some aspects of the present application as detailed in the appended claims.

[0089] Currently, there are a variety of colored contact lens products on the market. Since it is not suitable to try on makeup with colored contact lens products, with the development of the Internet and artificial intelligence (AI), virtual makeup enables users to try on colored contact lenses.

[0090] In the related art, a preset colored contact lens image is output in a first area, and a captured portrait image is output in a second area. The second area is superimposed on the first area to achieve colored contact lens try-on, resulting in a poor fit between the colored contact lens and the pupil, and thus a poor try-on effect of the colored contact lens.

[0091] In the present application, in order to improve the try-on effect of colored contact lenses, the inventor thought that: obtaining the central position and size from the image to be processed, and determining the colored contact lens try-on image according to the central position, size, and colored contact lens style image can improve the fit between the colored contact lens style image and the pupil in the colored contact lens try-on image, thereby improving the try-on effect of the colored contact lens.

[0092] The following uses specific embodiments to detail the technical solution of the present application and how the technical solution of the present application solves the above technical problems. These several specific embodiments below can be combined with each other, and the same or similar concepts or processes may not be repeated in some embodiments. The embodiments of the present application will be described below with reference to the drawings.

[0093] Figure 1 This is a flowchart of the image processing method provided by the embodiment of the present application. As Figure 1 shown, the method includes:

[0094] S101: Obtain an image to be processed, where the image to be processed includes an eye image, and the eye image includes a pupil image.

[0095] Optionally, the execution subject of the embodiments of the present application may be an electronic device, or an image processing device provided in the electronic device, and the image processing device may be implemented by a combination of software and / or hardware. For example, when the image processing device is implemented by software, the image processing device may be a piece of code stored in the storage area of the chip, and when the processor of the chip executes this piece of code, the image processing method to be executed by the device to be charged in the present application is implemented.

[0096] For example, when the image processing device is implemented by hardware, the image processing device may be a chip or a chip module, etc., and each module in the image processing device may include hardware modules such as the above-mentioned chip and chip module. When the image processing device works, the image processing method to be executed by the device to be charged in the present application can be implemented.

[0097] For example, when the image processing device is implemented by a combination of software and hardware, each module of the image processing device includes hardware such as a chip and a chip module, and a piece of code stored in the chip. When the above-mentioned chip and chip module work, the above code is executed to implement the image processing method to be executed by the device to be charged in the present application.

[0098] Among them, the eye image may be an image area including an eye image.

[0099] Optionally, the image to be processed may be an image collected by a camera on the electronic device, or an image stored in the electronic device.

[0100] S102: Determine the identifiers and positions of multiple key points corresponding to the eye image according to the image to be processed.

[0101] Optionally, the following two methods may be adopted to determine the identifiers and positions of multiple key points of the eye image according to the image to be processed.

[0102] Method 1: Perform face recognition on the image to be processed through a preset face recognition model to obtain the face image included in the image to be processed; perform key point detection on the face image through a preset key point detection model to obtain the identifiers and positions of multiple key points corresponding to the eye image.

[0103] Method 2: Perform face recognition on the image to be processed through a preset face recognition model to obtain the face image included in the image to be processed; perform key point detection on the face image through a preset key point detection model to obtain the identifiers and positions of N key points; obtain the identifiers and positions of multiple key points corresponding to the eye image from the identifiers and positions of the N key points.

[0104] The identifiers of the N key points include the identifiers of multiple key points corresponding to the face contour image, the mouth image, the nose image, the eyebrow image, and the eye image respectively.

[0105] The positions of N key points include the positions of multiple key points corresponding to the facial contour image, mouth image, nose image, eyebrow image, and eye image respectively.

[0106] Among them, from the identifiers and positions of the N key points, obtaining the identifiers and positions of the multiple key points corresponding to the eye image includes: determining multiple preset identifiers among the identifiers of the N key points as the identifiers of the multiple key points corresponding to the eye image; determining the positions of the key points corresponding to the multiple preset identifiers among the positions of the N key points as the positions of the multiple key points corresponding to the eye image. Optionally, N is an integer greater than or equal to 1.

[0107] For example, the identifiers of the multiple key points corresponding to the eye image are 67 - 75, 105, 76 - 84, 106. Specifically, please refer to Figure 2 the embodiments.

[0108] Figure 2 A schematic diagram of the facial key points provided by the embodiments of this application. As Figure 2 shown, for example, the identifiers and positions of 106 (i.e., N = 106) key points are obtained by the above method 2.

[0109] The facial key points include: the identifiers (1 - 33) and positions corresponding to the multiple key points of the facial contour image.

[0110] The identifiers (85 - 104) and positions corresponding to the multiple key points of the mouth image.

[0111] The identifiers (52 - 66) and positions corresponding to the multiple key points of the nose image.

[0112] The identifiers (34 - 42, 43 - 51) and positions corresponding to the multiple key points of the eyebrow image.

[0113] The identifiers (67 - 75, 105, 76 - 84, 106) and positions of the multiple key points of the eye image.

[0114] Among them, the positions of the key points corresponding to the identifier 75 and the identifier 105 coincide, and the positions of the key points corresponding to the identifier 84 and the identifier 106 coincide.

[0115] Optionally, the preset facial recognition model and the preset key point detection model are different lightweight neural network models.

[0116] In this application, the preset face recognition model and the preset key point detection model are lightweight different neural network models. While ensuring the timeliness of the algorithm, they can obtain high detection efficiency and alignment accuracy, and enable mobile terminals with limited performance to perform real-time video colored contact lens try-on and apply colored contact lenses to high-resolution images, etc.

[0117] S103: Determine the size of the pupil image according to the identifiers and positions of multiple key points.

[0118] Specifically, for the explanation of S103, reference can be made to Figure 3 the embodiments, which will not be elaborated here.

[0119] S104: Obtain the colored contact lens style image.

[0120] Optionally, the colored contact lens style image can be obtained in the following two ways.

[0121] Way 1: Display at least one style image, and in response to the user's first selection operation on the target style image among the at least one style image; determine the target style image as the colored contact lens style image.

[0122] Way 2: Display at least one colored contact lens type; in response to the user's first selection operation on the target type among the at least one colored contact lens type, obtain at least one style image corresponding to the target type from the preset database, and display at least one style image; in response to the user's second selection operation on the target style image among the at least one style image, determine the target style image as the colored contact lens style image.

[0123] Among them, the preset database includes multiple style images corresponding to each of multiple colored contact lens types.

[0124] Optionally, the preset database can be created and stored in the following two ways.

[0125] Way 1: Developers draw the images of each colored contact lens product according to at least one colored contact lens product corresponding to each of multiple colored contact lens types, using a drawing tool (image processing application); and adjust the size of the image of each colored contact lens product to a preset size to obtain the style image corresponding to each colored contact lens product; then, according to the colored contact lens type, store at least one style image corresponding to the colored contact lens type in the preset database.

[0126] Way 2: Developers spontaneously create multiple style images according to the iris texture and color rules; then adjust the size of each style image to a preset size; and then store the multiple style images in the preset database according to the custom colored contact lens type.

[0127] For example, the preset database can have the form of Table 1 below.

[0128] Table 1

[0129]

[0130] In the present application, the preset database includes multiple style images, and different colored contact lens fitting experiences can be provided to users.

[0131] S105: Determine the colored contact lens fitting image according to the identification, position, size of multiple key points, the colored contact lens style image, and the image to be processed.

[0132] Specifically, for the explanation of S105, reference can be made to Figure 5 the embodiments, which will not be elaborated here.

[0133] In Figure 1 the image processing method provided in the embodiment, according to the identification and position of multiple key points, determine the size of the pupil image, and according to the identification, position, size of multiple key points, and the colored contact lens style image, determine the colored contact lens fitting image, which can improve the fitting degree between the colored contact lens style image and the pupil, and further improve the fitting effect of the colored contact lens.

[0134] On the basis of the above embodiments, the execution process of the above S103 will be described in detail below in combination with Figure 3 the embodiments.

[0135] Figure 3 It is a flowchart for determining the size of the pupil image provided by the embodiment of the present application. As Figure 3 shown, the method includes:

[0136] S301: According to the identification and position of multiple key points, respectively determine the binary mask of the pupil area and the central position of the pupil image.

[0137] Optionally, the central position of the pupil image can be determined in the following 2 ways.

[0138] Way 1, according to the identification and position of multiple key points, determine the central position of the pupil image, including: determining the position of the key point with the first preset identification among multiple key points as the central position of the pupil image.

[0139] Optionally, for the left eye image, the preset identification can be 75 or 105; for the right eye image, the preset identification can be 84 or 106.

[0140] Way 2, determine the position to be calibrated of the pupil image as the position of the key point with the preset identification among multiple key points, and use the position of the key point with the second preset identification and the position of the key point with the third preset identification among multiple key points to perform calibration processing on the position to be calibrated to obtain the central position of the pupil image.

[0141] Optionally, the following method can be used to determine the binary mask of the pupil area:

[0142] S3010, according to the identifiers and positions of multiple key points, perform binary filling on the first preset image (second preset image) corresponding to the eye image (image to be processed) to obtain an eye image mask.

[0143] The eye image and the first preset image have the same size.

[0144] The image to be processed and the second preset image have the same size.

[0145] Optionally, performing binary filling on the first preset image corresponding to the eye image according to the identifiers and positions of multiple key points includes: determining the positions of the key points with multiple preset identifiers according to the identifiers and positions of multiple key points; performing triangulation on the eye image according to the positions of the key points with multiple preset identifiers to obtain multiple triangular regions; determining at least one pixel included in each of the multiple triangular regions; filling the pixels in the first preset image corresponding to at least one pixel included in each of the multiple triangular regions with a first value, and filling the other pixels in the first preset image except those filled with the first value with a second value.

[0146] The following combines Figure 4 , taking the left eye image in the eye image as an example, to illustrate the filled image corresponding to the left eye image. Figure 4 This is the filled image corresponding to the left eye image provided by the embodiment of the present application. As Figure 4 shown, first, perform triangulation on the left eye image. Optionally, the triangulation includes: connecting 67 to 74 in sequence, and connecting 75 (or 105) with 67 to 74 respectively to obtain 8 triangular regions, and the identifier of the common key point of the 8 triangular regions is 75 (or 105). Further, fill the 8 triangular regions with a first value, and fill the regions outside the 8 triangular regions with a second value to obtain the filled image corresponding to the left eye image. Perform operations similar to those on the left eye image on the right eye image to obtain the filled image corresponding to the right eye image, thereby obtaining the eye image mask.

[0147] Optionally, the process of performing binary filling on the second preset image corresponding to the image to be processed according to the identifiers and positions of multiple key points is similar to the process of performing binary filling on the first preset image corresponding to the eye image according to the identifiers and positions of multiple key points, and will not be elaborated here.

[0148] Optionally, the first value is 255 and the second value is 0. Or the first value is 0 and the second value is 255.

[0149] S3011. Determine multiple pixels to be counted in the eye image (image to be processed) according to the pixels filled with the first value in the eye image mask.

[0150] The multiple pixels to be counted correspond to the pixels filled with the first value in the eye image mask.

[0151] S3012. Determine the average value of the brightness of the multiple pixels to be counted.

[0152] Optionally, when the format of the image to be processed is the YVU format, count the brightness of each pixel among the multiple pixels to be counted, determine the sum of the brightness of the multiple pixels to be counted, and determine the ratio of the sum to the total number of the multiple pixels to be counted as the average value.

[0153] S3013. For each pixel to be counted among the multiple pixels to be counted, if the position of the pixel to be counted is within the preset range, determine the pixel to be counted as the pixel corresponding to the pupil; if the position of the pixel to be counted is not within the preset range, when the brightness of the pixel to be counted is less than or equal to the above average value, determine the pixel to be counted as the pixel corresponding to the pupil, and when the brightness of the pixel to be counted is greater than the above average value, determine the pixel to be counted as the pixel corresponding to the sclera.

[0154] S3014. Adjust the eye image mask according to the pixels corresponding to the sclera and the pixels corresponding to the pupil to obtain the binary mask of the pupil area.

[0155] Optionally, the binary mask of the pupil area can be obtained in the following two ways.

[0156] Method 1. Optionally, fill the pixels corresponding to the pupil in the eye image mask with the first value, fill the other pixels in the eye image mask except the pixels corresponding to the pupil with the second value, and the binary mask of the pupil area is obtained.

[0157] Method 2. S1: For the i-th pixel among the pixels corresponding to the pupil, determine whether the filling value of the corresponding pixel of the i-th pixel in the eye image mask is the first value; if so, execute S2, if not, execute S3. S2: Increment i by 1 and repeat S1; S3: Fill the corresponding pixel of the i-th pixel in the eye image mask with the first value and increment i by 1, repeat S1; S4: Fill the other pixels in the adjusted image mask except the pixels corresponding to the pupil with the second value, and the binary mask of the pupil area is obtained. Wherein, the adjusted image mask is obtained after repeating S1 - S3 until i is equal to the total number of the pixels corresponding to the pupil.

[0158] S302: Determine the size of the pupil image according to the binary mask of the pupil area, the center position, the identifiers and positions of multiple key points.

[0159] Specifically, the following method can be used to determine the size of the pupil image:

[0160] Search for the pixels passed by the first direction along the first direction pointing from the central position to the second position in the binary mask of the pupil area to obtain the first edge position of the pupil in the first direction; the second position is the position corresponding to the key point with the second preset identifier among the multiple key points;

[0161] Search for the pixels passed by the second direction along the second direction pointing from the central position to the third position in the binary mask of the pupil area to obtain the second edge position of the pupil in the second direction; the third position is the position corresponding to the key point with the third preset identifier among the multiple key points;

[0162] Respectively determine the first distance between the first edge position and the central position, and the second distance between the second edge position and the central position;

[0163] Determine the size according to the first distance and the second distance.

[0164] Optionally, for the left pupil image of the binary mask of the pupil area, the second preset identifier can be 67, or the third preset identifier is 71; for the right pupil image of the binary mask of the pupil area, the second preset identifier can be 76, and the third preset identifier is 80.

[0165] It should be noted that the filling values of the pixels on the left and right sides of the first edge position are different, and the filling values of the pixels on the left and right sides of the second edge position are different.

[0166] Among them, the size is the diameter of the pupil (i.e., 2r + b). Optionally, r can be the minimum value, maximum value, or average value of the first distance and the second distance. Optionally, b can be 0 or 1

[0167] It should be noted that the sizes of the left pupil image and the right pupil image can be determined respectively according to the method for determining the size of the pupil image above, or the size of only any one of the left pupil image and the right pupil image can be determined (by default, the sizes of the left pupil image and the right pupil image are the same).

[0168] In Figure 3 In the embodiment, according to the identifiers and positions of multiple key points, the central positions of the binary mask of the pupil area and the pupil image are determined respectively. According to the binary mask of the pupil area, the central position, the identifiers and positions of multiple key points, the size of the pupil image can be determined, and the size of the pupil image can be accurately determined, thereby ensuring the fitting degree between the colored contact lens style image and the pupil.

[0169] On the basis of the above embodiment, the execution process of S105 will be described in detail below in combination with Figure 5 the following

[0170] Figure 5 This is a flowchart for determining the colored contact lens fitting image provided by the embodiments of the present application. As Figure 5 shown, the method includes:

[0171] S501. Determine a colored contact lens fusion mask according to the identifiers, positions, sizes of multiple key points, and the colored contact lens style image.

[0172] Optionally, the following method can be used to determine the colored contact lens fusion mask:

[0173] Determine an iris binary mask according to the identifiers, positions, sizes of multiple key points, and the colored contact lens style image; perform filtering processing on the iris binary mask according to the size to obtain the colored contact lens fusion mask.

[0174] Optionally, the following method can be used to determine the iris binary mask:

[0175] Determine an eye region binary mask according to the identifiers and positions of multiple key points;

[0176] Perform scaling processing on the colored contact lens style image according to the size to obtain a scaled style image;

[0177] Perform transparency analysis processing on the scaled style image to obtain an iris region binary mask;

[0178] Determine the iris binary mask according to the eye region binary mask and the iris region binary mask.

[0179] Among them, the method for determining the eye region binary mask is the same as the method for determining the eye image mask in S3010, and will not be elaborated here.

[0180] Optionally, perform scaling processing on the colored contact lens style image respectively according to the size of the left pupil image and / or the size of the right pupil image to obtain a scaled style image. Among them, the size of the scaled style image is the same as the size of the left pupil image and / or the size of the right pupil image.

[0181] Optionally, the following method can be used to determine the iris region binary mask:

[0182] Judge whether the format of the scaled style image is RGB format;

[0183] If so, determine the iris region binary mask through the following formula;

[0184] If not, perform format conversion on the scaled style image to obtain a scaled style image with RGB format, and for the scaled style image with RGB format, determine the iris region binary mask through the following formula 1;

[0185] IRIS_i = CLIP(768 - (Contect_i_R + Contect_i_G + Contect_i_B) * a, 0, 768) Formula 1;

[0186] Wherein, IRIS_i is the value of the i-th pixel in the binary mask of the iris region, CLIP limits the value of the i-th pixel between [0, 768], Contect_i_R is the value of the i-th pixel in the R channel of the scaled style image, Contect_i_G is the value of the i-th pixel in the G channel of the scaled style image, Contect_i_B is the value of the i-th pixel in the B channel of the scaled style image, and a is the weight coefficient for transparency analysis (with a value between [1, 20]), where * represents multiplication.

[0187] Optionally, the following method can be used to determine the iris binary mask:

[0188] Normalize the binary mask of the iris region to obtain an intermediate binary mask;

[0189] For each pixel in the binary mask of the eye region and the intermediate binary mask, multiply the value of the i-th pixel in the binary mask of the eye region by the value of the i-th pixel in the intermediate binary mask to obtain the iris binary mask.

[0190] In this application, by determining the iris binary mask based on the binary mask of the eye region and the binary mask of the iris region, an iris binary mask that does not overflow the orbital boundary can be obtained, thereby improving the guarantee of the fitting degree between the colored contact lens style image and the pupil.

[0191] Optionally, the following method can be used to determine the colored contact lens fusion mask, including:

[0192] According to the size, perform adaptive mean filtering or Gaussian filtering on the iris binary mask to obtain the colored contact lens fusion mask.

[0193] In this application, by performing adaptive mean filtering or Gaussian filtering on the iris binary mask, a colored contact lens fusion mask with smooth and gradual boundaries can be obtained.

[0194] S502. Obtain the trial makeup intensity.

[0195] Optionally, the following 2 methods can be used to obtain the trial makeup intensity.

[0196] Method 1: Display multiple trial makeup intensities. In response to the user's selection operation on the target trial makeup intensity among the multiple trial makeup intensities, determine the target trial makeup intensity as the trial makeup intensity required in S502.

[0197] Method 2, displaying a trial makeup intensity scale and a sliding control on the trial makeup intensity scale, and determining the trial makeup intensity required in S502 according to the intensity corresponding to the position of the sliding control on the trial makeup intensity scale in response to the user's sliding operation on the sliding control.

[0198] S503 , performing adaptive weight fusion processing on the image to be processed and the contact lens style image according to the contact lens fusion mask and the trial makeup intensity, to obtain a contact lens trial wear image.

[0199] Alternatively, according to the contact lens fusion mask and the trial makeup intensity, the eye image and the contact lens style image are adaptively weighted fused to obtain a contact lens trial wear image.

[0200] When the eye image and the contact lens style image are adaptively weighted fused according to the contact lens fusion mask and the trial makeup intensity to obtain the contact lens trial image, a preset area including the eye image in the image to be processed can be determined as the eye image.

[0201] Specifically, the cosmetic contact lens trial image may be determined by the following formula 2 and formula 3.

[0202] Alpha=Contact_mask·Level Formula 2;

[0203]

[0204] Among them, Alpha is the strength of the contact lens fusion effect (used to control and prevent the contact lens effect from overflowing the eye socket area), Contact_mask is the contact lens fusion mask, Level is the makeup trial strength, Img_src is the image to be processed, Contact_model is the contact lens style image, and · also represents multiplication.

[0205] exist Figure 5 In the embodiment, according to the identification and position, size and color contact lens style image of multiple key points, the color contact lens fusion mask is determined, which can assist in preserving the iris, pupil color and texture of the pupil. The trial makeup strength is obtained, and according to the color contact lens fusion mask and the trial makeup strength, the image to be processed and the color contact lens style image are adaptively weighted fused to obtain the color contact lens trial wear image, which can be combined with user needs to obtain the best color contact lens trial wear image, thereby improving the user's color contact lens trial wear experience.

[0206] Combine the following Figure 6 , taking the eye image as an example, the trial fitting effect diagram is shown exemplarily.

[0207] Figure 6 This is a trial installation effect diagram provided by the embodiment of this application. Figure 6As shown, it includes: an eye image 61, a binary mask of the eye region 62, a binary mask of the pupil region 63, a colored contact lens style image 64, a colored contact lens fusion mask 65, and a colored contact lens try-on image 66.

[0208] The binary mask of the eye region 62 is obtained from the eye image 61.

[0209] The binary mask of the pupil region 63 is obtained from the binary mask of the eye region 62.

[0210] The size of the pupil image is obtained from the binary mask of the pupil region 63.

[0211] The colored contact lens style image 64 is processed according to the size to obtain the colored contact lens fusion mask 65. Specifically, for the process of obtaining the colored contact lens fusion mask 65, please refer to the above method embodiments.

[0212] Finally, the colored contact lens try-on image 66 is obtained from the colored contact lens fusion mask 65, the makeup try-on intensity, the eye image 61, and the colored contact lens style image 64.

[0213] Figure 7 It is a structural diagram of the image processing device provided by the embodiment of the present application. As Figure 7 shown, the image processing device 10 includes:

[0214] A first acquisition module 101 for acquiring an image to be processed, where the image to be processed includes an eye image, and the eye image includes a pupil image;

[0215] A first determination module 102 for determining the identifiers and positions of multiple key points corresponding to the eye image according to the image to be processed;

[0216] A second determination module 103 for determining the size of the pupil image according to the identifiers and positions of the multiple key points;

[0217] A second acquisition module 104 for acquiring a colored contact lens style image;

[0218] A third determination module 105 for determining a colored contact lens try-on image according to the identifiers and positions of the multiple key points, the size, the colored contact lens style image, and the image to be processed.

[0219] The image processing device provided by the embodiment of the present application can execute the above image processing method, and its implementation principle and beneficial effects are similar, so details are not described here again.

[0220] In a possible design, the third determination module 105 is specifically used for:

[0221] Determining a colored contact lens fusion mask according to the identifiers and positions of the multiple key points, the size, and the colored contact lens style image;

[0222] Acquiring the makeup try-on intensity;

[0223] According to the colored contact lens fusion mask and the makeup trial intensity, perform adaptive weight fusion processing on the image to be processed and the colored contact lens style image to obtain a colored contact lens trial image.

[0224] In a possible design, the third determination module 105 is specifically configured to:

[0225] Determine an iris binary mask according to the identifiers, positions, sizes of multiple key points, and the colored contact lens style image;

[0226] Perform filtering processing on the iris binary mask according to the size to obtain a colored contact lens fusion mask.

[0227] In a possible design, the third determination module 105 is specifically configured to:

[0228] Determine an eye region binary mask according to the identifiers and positions of multiple key points;

[0229] Perform scaling processing on the colored contact lens style image according to the size to obtain a scaled style image;

[0230] Perform transparency analysis processing on the scaled style image to obtain an iris region binary mask;

[0231] Determine an iris binary mask according to the eye region binary mask and the iris region binary mask.

[0232] In a possible design, the second determination module 103 is specifically configured to:

[0233] Respectively determine a pupil region binary mask and the central position of the pupil image according to the identifiers and positions of multiple key points;

[0234] Determine the size according to the pupil region binary mask, the central position, the identifiers and positions of multiple key points.

[0235] In a possible design, the second determination module 103 is specifically configured to:

[0236] Search for the pixels passed by the first direction in the pupil region binary mask along the first direction pointing from the central position to the second position to obtain the first edge position of the pupil in the first direction; the second position is the position corresponding to the key point with the second preset identifier among multiple key points;

[0237] Search for the pixels passed by the second direction in the pupil region binary mask along the second direction pointing from the central position to the third position to obtain the second edge position of the pupil in the second direction; the third position is the position corresponding to the key point with the third preset identifier among multiple key points;

[0238] Determine a first distance between the first edge position and the center position, and a second distance between the second edge position and the center position respectively;

[0239] Determine the size according to the first distance and the second distance.

[0240] In a possible design, the first determination module 102 is specifically configured to:

[0241] Perform face recognition on the image to be processed through a preset face recognition model to obtain the face image included in the image to be processed;

[0242] Perform key point detection on the face image through a preset key point detection model to obtain the identifiers and positions of multiple key points corresponding to the eye image.

[0243] In a possible design, the first acquisition module 101 is specifically configured to:

[0244] Display at least one colored contact lens type;

[0245] In response to a user's first selection operation on a target type among the at least one colored contact lens types, obtain at least one style image corresponding to the target type from a preset database, and display the at least one style image;

[0246] In response to a user's second selection operation on a target style image among the at least one style images, determine the target style image as the colored contact lens style image.

[0247] The image processing device provided by the embodiments of the present application can execute the above image processing method, and its implementation principle and beneficial effects are similar, which will not be elaborated here.

[0248] Figure 8 This is a schematic diagram of the hardware of the electronic device provided by the embodiments of the present application. As Figure 8 shown, the electronic device 20 may include: a transceiver 201, a memory 202, and a processor 203. The transceiver 201 may include: a transmitter and / or a receiver. The transmitter may also be referred to as a sender, a transmitter, a transmission port, or a transmission interface, etc. Similar descriptions, and the receiver may also be referred to as a receiver, a receiver, a receiving port, or a receiving interface, etc. Exemplarily, the transceiver 201, the memory 202, and the processor 203 are interconnected with each other through a bus 204.

[0249] The memory 202 is used to store computer execution instructions;

[0250] The processor 203 is used to execute the computer execution instructions stored in the memory 202, so that the processor 203 executes the above image processing method.

[0251] An embodiment of the present application provides a computer-readable storage medium, in which computer-executable instructions are stored, and when the computer-executable instructions are executed by a processor, they are used to implement any one of the above image processing methods.

[0252] An embodiment of the present application further provides a computer program product, including a computer program, and when the computer program is executed by a processor, it implements any one of the above image processing methods.

[0253] All or part of the steps of implementing the above method embodiments can be completed by hardware related to program instructions. The foregoing program can be stored in a readable memory. When the program is executed, it executes the steps including the above method embodiments; and the foregoing memory (storage medium) includes: read-only memory (ROM), RAM, flash memory, hard disk, solid state drive, magnetic tape, floppy disk, optical disc, and any combination thereof.

[0254] Embodiments of the present application are described with reference to the flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to embodiments of the present application. It should be understood that each process and / or block in the flowchart and / or block diagram, and the combination of processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to the processing unit of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing devices to generate a machine, so that the instructions executed by the processing unit of the computer or other programmable data processing devices generate for implementing in the process Figure 1 one process or multiple processes and / or blocks Figure 1 a device for implementing the functions specified in one block or multiple blocks.

[0255] These computer program instructions can also be stored in a computer-readable memory that can direct a computer or other programmable data processing device to work in a specific manner, so that the instructions stored in the computer-readable memory generate a manufactured product including an instruction device, and the instruction device implements in the process Figure 1 one process or multiple processes and / or blocks Figure 1 a device for implementing the functions specified in one block or multiple blocks.

[0256] These computer program instructions can also be loaded onto a computer or other programmable data processing device, so that a series of operation steps are executed on the computer or other programmable device to generate a computer-implemented process, and thus the instructions executed on the computer or other programmable device provide for implementing in the process Figure 1 one process or multiple processes and / or blocks Figure 1Steps of the functions specified in one or more boxes.

[0257] Obviously, those skilled in the art can make various changes and modifications to the embodiments of the present application without departing from the spirit and scope of the present application. Thus, if these modifications and variations of the embodiments of the present application fall within the scope of the claims of the present application and their equivalent technologies, the present application is also intended to include these changes and modifications.

[0258] In the present application, the term "including" and its variations may refer to non-limiting inclusion; the term "or" and its variations may refer to "and / or". In the present application, terms such as "first", "second", etc. are used to distinguish similar objects and do not necessarily have to be used to describe a specific order or sequence. In the present application, "a plurality of" means two or more. "And / or" describes the relationship between associated objects and indicates that there can be three relationships. For example, A and / or B can represent: A exists alone, A and B exist simultaneously, and B exists alone. The character " / " generally indicates that the associated objects before and after are in an "or" relationship.

[0259] After considering the specification and practicing the invention disclosed herein, those skilled in the art will readily conceive of other embodiments of the present application. The present application is intended to cover any variations, uses, or adaptations of the present application, which follow the general principles of the present application and include common general knowledge or conventional technical means in the technical field not disclosed in the present application. The specification and examples are only considered exemplary, and the true scope and spirit of the present application are pointed out by the following claims.

[0260] It should be understood that the present application is not limited to the exact structures already described and shown in the drawings, and various modifications and changes can be made without departing from its scope. The scope of the present application is only limited by the appended claims.

Claims

1. An image processing method, characterized in that, comprising: obtaining an image to be processed, where the image to be processed includes an eye image, and the eye image includes a pupil image; determining the identifiers and positions of multiple key points corresponding to the eye image according to the image to be processed; determining the size of the pupil image according to the identifiers and positions of the multiple key points; obtaining a colored contact lens style image; determining a binary mask of the eye region according to the identifiers and positions of the multiple key points; scaling the colored contact lens style image according to the size to obtain a scaled style image; performing transparency analysis processing on the scaled style image to obtain a binary mask of the iris region; determining the binary mask of the iris according to the binary mask of the eye region and the binary mask of the iris region; performing filtering processing on the binary mask of the iris according to the size to determine a colored contact lens fusion mask; performing adaptive weight fusion processing on the image to be processed and the colored contact lens style image according to the colored contact lens fusion mask and the trial makeup intensity to obtain the colored contact lens trial wearing image.

2. The method according to claim 1, characterized in that, the method further comprises: obtaining the trial makeup intensity.

3. The method according to claim 1 or 2, characterized in that, the determining the size of the pupil image according to the identifiers and positions of the multiple key points includes: respectively determining a binary mask of the pupil region and the central position of the pupil image according to the identifiers and positions of the multiple key points; determining the size according to the binary mask of the pupil region, the central position, the identifiers and positions of the multiple key points.

4. The method according to claim 3, characterized in that, the determining the size according to the binary mask of the pupil region, the central position, the identifiers and positions of the multiple key points includes: searching for pixels passed by the first direction in the binary mask of the pupil region along the first direction pointing from the central position to the second position to obtain the first edge position of the pupil in the first direction; the second position is the position corresponding to the key point with the second preset identifier among the multiple key points; searching for pixels passed by the second direction in the binary mask of the pupil region along the second direction pointing from the central position to the third position to obtain the second edge position of the pupil in the second direction; the third position is the position corresponding to the key point with the third preset identifier among the multiple key points; respectively determining a first distance between the first edge position and the central position, and a second distance between the second edge position and the central position; determining the size according to the first distance and the second distance.

5. The method according to claim 1 or 2, characterized in that, the determining the identifiers and positions of the multiple key points corresponding to the eye image according to the image to be processed includes: performing face recognition on the image to be processed through a preset face recognition model to obtain the face image included in the image to be processed; By using a pre-set key point detection model, perform key point detection on the facial image to obtain the identifiers and positions of multiple key points corresponding to the eye image.

6. The method according to claim 1 or 2, wherein, the obtaining of the colored contact lens style image includes: displaying at least one colored contact lens type; in response to a first selection operation of a target type from the at least one colored contact lens type by the user, obtaining at least one style image corresponding to the target type from a pre-set database and displaying the at least one style image; in response to a second selection operation of a target style image from the at least one style image by the user, determining the target style image as the colored contact lens style image.

7. An image processing apparatus, wherein, it includes: a first obtaining module, configured to obtain an image to be processed, where the image to be processed includes an eye image, and the eye image includes a pupil image; a first determining module, configured to determine the identifiers and positions of multiple key points corresponding to the eye image according to the image to be processed; a second determining module, configured to determine the size of the pupil image according to the identifiers and positions of the multiple key points; a second obtaining module, configured to obtain a colored contact lens style image; a third determining module, configured to determine a colored contact lens try-on image according to the identifiers and positions of the multiple key points, the size, the colored contact lens style image, and the image to be processed; the third determining module is specifically configured to determine a binary mask of the eye region according to the identifiers and positions of the multiple key points; perform a scaling process on the colored contact lens style image according to the size to obtain a scaled style image; perform a transparency analysis process on the scaled style image to obtain a binary mask of the iris region; determine the binary mask of the iris according to the binary mask of the eye region and the binary mask of the iris region; perform a filtering process on the binary mask of the iris according to the size to determine a colored contact lens fusion mask; perform an adaptive weight fusion process on the image to be processed and the colored contact lens style image according to the colored contact lens fusion mask and the makeup intensity to obtain the colored contact lens try-on image.

8. An electronic device, wherein, it includes: a processor and a memory communicatively connected to the processor; the memory stores computer execution instructions; the processor executes the computer execution instructions stored in the memory to implement the image processing method according to any one of claims 1-6.

9. A computer-readable storage medium, wherein, computer execution instructions are stored in the computer-readable storage medium, and when the computer execution instructions are executed by a processor, they are used to implement the image processing method according to any one of claims 1-6.

10. A computer program product, wherein, it includes a computer program, and when the computer program is executed by a processor, it implements the image processing method according to any one of claims 1-6.

Images