Method and device for removing wrinkles and redrawing portrait in picture, and computer equipment

Abstract

The invention provides a method and a device for removing wrinkles and redrawing a portrait in a picture, and computer equipment, and the method comprises the steps: obtaining an original picture to be subjected to wrinkle removal and a wrinkle mask picture of the original picture, carrying out the wrinkle removal bounding box re-cutting and resolution compression processing operation on the original picture and the wrinkle mask picture, and obtaining the image to be subjected to wrinkle removal; obtaining a high-resolution original image, a low-resolution original image to be redrawn and a redrawn area mask image; processing the low-resolution original image to be redrawn and the redrawn area mask image by using a preset target wrinkle-removing redrawing model to obtain a low-resolution wrinkle-removing result image; processing the low-resolution wrinkle removal result image into a high-resolution wrinkle removal result image with the resolution consistent with that of the high-resolution original image by using an up-sampling reversible hybrid algorithm; and replacing a corresponding region in the high-resolution original image with the high-resolution wrinkle removal result image to obtain a target wrinkle removal image. According to the invention, a more natural and fine wrinkle removing effect is realized, and the user experience and the portrait beautifying quality are improved.

Description

Technical Field

[0001] The present invention relates to the technical field of image processing, and in particular to a method and device for removing wrinkles and redrawing a portrait in a picture, and a computer device. Background Art

[0002] In the field of modern digital image processing, wrinkle removal and redrawing technology is increasingly widely used in portrait beautification and has great significance. However, existing wrinkle removal algorithms have significant deficiencies in performance and effect. Existing technical solutions are generally divided into the following two categories:

[0003] The first is to remove wrinkles by filtering and restoring the image based on traditional digital image processing algorithms, etc. The disadvantage of this method is that it lacks the necessary semantic information of wrinkles and cannot distinguish between wrinkles and normal skin texture. After filtering the high-frequency information, the skin details will be lost, and the processed area will become flat and blurred, losing the natural skin texture. At the same time, the wrinkle removal effect is poor, and the connection with the surrounding skin edges is unnatural, resulting in a mottled skin condition on the face.

[0004] The second is to use a machine learning algorithm based on a convolutional neural network to repair and redraw the wrinkle area to obtain a new wrinkle-free skin result. The disadvantage of this method is that it has extremely high performance requirements when processing high-resolution images, and the computational complexity is high. It cannot run smoothly on low-end devices with poor performance, and it cannot support users to freely and smoothly manually paint. When the resolution of the model processing images is reduced, although the processing efficiency is improved, the skin details of the high-definition original image will be lost. At the same time, the current model wrinkle removal effect is not robust. When processing special skin colors and large areas of wrinkles, it is easy to have processing traces that do not resemble skin texture, which also causes unnatural texture and mottled skin conditions on the face.

[0005] In summary, many existing algorithms cannot run smoothly on low-end devices with poor performance due to their high computational complexity, which limits their scope of application; or they only use simple skin smoothing technology, resulting in insufficient wrinkle removal and easy loss of texture details, making the entire portrait image have obvious processing traces or blemishes. Summary of the invention

[0006] To this end, the present invention provides a method and device for removing wrinkles and redrawing a portrait in a picture, and a computer device, aiming to solve the technical problems in the prior art that the portrait wrinkle removal algorithm causes unnatural texture and poor wrinkle removal effect.

[0007] To achieve the above objectives, the present invention adopts the following technical solutions:

[0008] According to a first aspect of the present invention, the present invention provides a method for removing wrinkles and redrawing a portrait in a picture, the method comprising:

[0009] Obtaining an original image to be wrinkle-removed and a wrinkle mask image of the original image, performing wrinkle-removal boundary box re-cropping and resolution compression processing operations on the original image and the wrinkle mask image to obtain a high-resolution original image, a low-resolution original image to be redrawn, and a redrawing area mask image;

[0010] The low-resolution original image to be redrawn and the redrawing area mask image are processed using a preset target wrinkle removal redrawing model to obtain a low-resolution wrinkle removal result image;

[0011] Using an upsampling reversible hybrid algorithm, the low-resolution wrinkle removal result image is processed into a high-resolution wrinkle removal result image having a resolution consistent with that of the high-resolution original image;

[0012] The high-resolution wrinkle removal result image is used to replace the corresponding area in the high-resolution original image to obtain a target wrinkle removal image.

[0013] Optionally, the step of obtaining an original image to be wrinkle-removed and a wrinkle mask image of the original image includes:

[0014] Obtain the original image to be wrinkle-removed;

[0015] Processing the original image using a preset wrinkle segmentation model to obtain the wrinkle mask image; or,

[0016] The painting data generated by manual painting by the user based on the original picture is received to generate the wrinkle mask image.

[0017] Optionally, performing wrinkle removal boundary box re-cropping and resolution compression processing operations on the original image and the wrinkle mask image to obtain a high-resolution original image, a low-resolution original image to be redrawn, and a redrawing area mask image includes:

[0018] Selecting a wrinkle bounding box of the wrinkle mask image;

[0019] For the original image and the wrinkle mask image, respectively crop the corresponding areas of the wrinkle bounding box to obtain a high-resolution original image and a wrinkle mask bounding box image;

[0020] Compressing the high-resolution original image and the wrinkle mask boundary box image according to a preset resolution to obtain a low-resolution original image to be redrawn and a redrawing area mask image;

[0021] The redrawing area mask image is a single-channel grayscale image, and its resolution is consistent with the resolution of the low-resolution original image to be redrawn.

[0022] Optionally, before processing the low-resolution original image to be redrawn and the redrawing area mask image using a preset target wrinkle removal redrawing model, the method further includes:

[0023] Constructing a neural network model based on GAN technology as a wrinkle removal and repainting model; the wrinkle removal model includes a generator sub-model for removing wrinkles and a discriminator sub-model for identifying whether wrinkles have been removed and whether the repair result is true;

[0024] The wrinkle removal and redrawing model is trained to obtain a target wrinkle removal and redrawing model for repairing and redrawing skin with realistic texture and without wrinkles on the wrinkle mask area of ​​the original image.

[0025] Optionally, the training of the wrinkle removal and redrawing model includes:

[0026] Acquire two groups of skin image data; the two groups of skin image data include a skin image group and a random skin mask image group; each skin image included in the skin image group has a unique corresponding matching relationship with each skin mask image included in the random skin mask image group;

[0027] The wrinkle removal and redrawing model is trained using the two sets of skin image data to obtain a universal skin image restoration model; the universal skin image restoration model is used to redraw the skin with realistic texture in the random skin mask area of ​​the original image;

[0028] Acquire three groups of wrinkle image data; the three groups of wrinkle image data include a wrinkle skin image group, a wrinkle mask image group and a wrinkle removal result image group; each wrinkle skin image included in the wrinkle skin image group, each wrinkle mask image included in the wrinkle mask image group and each wrinkle removal result image included in the wrinkle removal result image group have a unique corresponding matching relationship;

[0029] The three groups of wrinkle image data are used to train the universal skin image restoration model to obtain the target wrinkle removal and redrawing model.

[0030] Optionally, before using the upsampling reversible hybrid algorithm to process the low-resolution wrinkle removal result image into a high-resolution wrinkle removal result image having a resolution consistent with that of the high-resolution original image, the method further comprises:

[0031] Construct a hybrid algorithm with unknown parameters, the formula is as follows:

[0032] Result=f(Base,a0,a1,…,an)+f(Base,b0,b1,…,bn)×Overlay

[0033] Among them, Base is the underlying image, Overlay is the mixed image, Result is the result image, f is a function composed of the non-negative integer powers of the underlying image and its corresponding undetermined coefficients; a0, a1, ..., an and b0, b1, ..., bn are all undetermined coefficients;

[0034] The three groups of wrinkle image data are used to optimize and train the undetermined parameters included in the hybrid algorithm to obtain an upsampling reversible hybrid algorithm.

[0035] Optionally, the processing of the low-resolution wrinkle removal result image into a high-resolution wrinkle removal result image having a resolution consistent with that of the high-resolution original image by using an upsampling reversible hybrid algorithm comprises:

[0036] The upsampling reversible hybrid algorithm is used to perform inverse operation, the low-resolution original image to be redrawn is used as the bottom image, the low-resolution wrinkle removal result image is used as the result image, and a hybrid image is calculated, that is, a wrinkle removal hybrid image;

[0037] The upsampling reversible hybrid algorithm is used to perform a forward operation, the high-resolution original image is used as the underlying image, the wrinkle-removed hybrid image is used as the hybrid image, and the result image is calculated, which is the high-resolution wrinkle-removed result image.

[0038] Optionally, the using the high-resolution wrinkle removal result image to replace a corresponding area in the high-resolution original image to obtain a target wrinkle removal image includes:

[0039] The high-resolution result image is pasted back to the high-resolution original image in the corresponding area of ​​the wrinkle boundary box to obtain a complete and high-resolution target wrinkle-removed image.

[0040] According to a second aspect of the present invention, the present invention provides a device for removing wrinkles and redrawing a portrait in a picture, the device comprising:

[0041] An image acquisition module is used to acquire an original image to be wrinkle-removed and a wrinkle mask image of the original image, and perform wrinkle-removal boundary box re-cropping and resolution compression processing operations on the original image and the wrinkle mask image to obtain a high-resolution original image, a low-resolution original image to be redrawn, and a redrawing area mask image;

[0042] An image wrinkle removal module is used to process the low-resolution original image to be redrawn and the redrawing area mask image using a preset target wrinkle removal redrawing model to obtain a low-resolution wrinkle removal result image;

[0043] An image processing module, used for processing the low-resolution wrinkle removal result image into a high-resolution wrinkle removal result image having a resolution consistent with that of the high-resolution original image by using an upsampling reversible hybrid algorithm;

[0044] The image replacement module is used to replace the corresponding area in the high-resolution original image with the high-resolution wrinkle removal result image to obtain a target wrinkle removal image.

[0045] According to a third aspect of the present invention, the present invention provides a computer device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements a method for removing wrinkles and redrawing a portrait in a picture as described in any one of the first aspects of the present invention when executing the computer program.

[0046] The present invention adopts the above technical solution and has at least the following beneficial effects:

[0047] Through the scheme of the present invention, the original image to be wrinkled and the wrinkle mask map of the original image are obtained, and the wrinkle boundary box is re-cropped and the resolution compression processing operation is performed on the original image and the wrinkle mask map to obtain a high-resolution original image, a low-resolution original image to be redrawn, and a redrawing area mask map; the low-resolution original image to be redrawn and the redrawing area mask map are processed using a preset target wrinkle removal redrawing model to obtain a low-resolution wrinkle removal result map; the low-resolution wrinkle removal result map is processed into a high-resolution wrinkle removal result map with the same resolution as the high-resolution original image using an upsampling reversible hybrid algorithm; the high-resolution wrinkle removal result map is used to replace the corresponding area in the high-resolution original image to obtain a target wrinkle removal image. In this way, the facial details such as skin color, texture details, etc. are learned, and different races and facial states can be intelligently processed, thereby achieving a more natural and delicate wrinkle removal effect, improving the user experience and the quality of portrait beautification.

[0048] It is to be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0049] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0050] Figure 1 A schematic diagram showing a flow chart of a method for removing wrinkles and redrawing a portrait in a picture provided by an embodiment of the present invention;

[0051] Figure 2 A schematic diagram showing a flow chart of a method for removing wrinkles and redrawing a portrait in a picture provided by another embodiment of the present invention;

[0052] Figure 3 An example of an original picture provided by an embodiment of the present invention is shown;

[0053] Figure 4 An example of a wrinkle mask image provided by an embodiment of the present invention is shown;

[0054] Figure 5 An example of a high-resolution original image provided by an embodiment of the present invention is shown;

[0055] Figure 6 An example of a low-resolution original image to be redrawn provided by an embodiment of the present invention is shown;

[0056] Figure 7 An example of a redrawing area mask map provided by an embodiment of the present invention is shown;

[0057] Figure 8 An example of a low-resolution wrinkle removal result image provided by an embodiment of the present invention is shown;

[0058] Fig. 9 An example of a wrinkle-removed mixed image provided by an embodiment of the present invention is shown;

[0059] Fig.10 An example of a high-resolution wrinkle removal result image provided by an embodiment of the present invention is shown;

[0060] Fig.11 An example of a target wrinkle removal image provided by an embodiment of the present invention is shown;

[0061] Fig.12 A schematic diagram of the structure of a device for removing wrinkles and redrawing a portrait in a picture provided by an embodiment of the present invention is shown;

[0062] Fig.13 A schematic diagram showing the structure of a device for removing wrinkles and redrawing a portrait in a picture provided by another embodiment of the present invention is shown;

[0063] Fig.14 A schematic diagram of the physical structure of a computer device provided by an embodiment of the present invention is shown. DETAILED DESCRIPTION

[0064] The exemplary embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. Although the exemplary embodiments of the present disclosure are shown in the accompanying drawings, it should be understood that the present disclosure can be implemented in various forms and should not be limited by the embodiments set forth herein. On the contrary, these embodiments are provided to enable a more thorough understanding of the present disclosure and to fully convey the scope of the present disclosure to those skilled in the art.

[0065] It should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the statement "include..." do not exclude the presence of other identical elements in the process, method, article or device including the elements.

[0066] The embodiment of the present invention provides a method for removing wrinkles and redrawing a portrait in a picture, such as Figure 1 As shown, at least the following steps S101 to S104 may be included:

[0067] Step S101, obtaining the original image to be de-wrinkled and the wrinkle mask image of the original image, performing de-wrinkle boundary box re-cropping and resolution compression processing operations on the original image and the wrinkle mask image, and obtaining a high-resolution original image, a low-resolution original image to be redrawn, and a redrawing area mask image.

[0068] like Figure 3 As shown in FIG. 1 , an example of an original image to be wrinkle-removed is shown. The original image refers to a portrait image input by the user without any processing. Figure 4 As shown, Figure 3 The wrinkle mask image corresponding to the original image shown is the wrinkle mask image to be redrawn of the original image. Figure 2 As shown, the present invention can obtain the wrinkle mask image of the original image in two ways. Specifically, in the automatic wrinkle removal mode, the original image is processed using a preset wrinkle segmentation model to obtain a wrinkle mask image. The wrinkle segmentation model in the embodiment of the present invention can be an existing model in the prior art, as long as it can achieve wrinkle recognition and image segmentation effects. In the manual wrinkle removal mode, it can be obtained by manual painting input by the user, that is, receiving the painting data generated by the user's manual painting based on the original image to generate a wrinkle mask image.

[0069] Furthermore, in order to facilitate the subsequent processing of the image, the original image to be wrinkled and the wrinkle mask image need to be processed into low-resolution images to minimize the image input size in the subsequent processing. Specifically, the wrinkle bounding box of the wrinkle mask image can be selected; for the original image and the wrinkle mask image, the corresponding areas of the wrinkle bounding box are cropped out respectively to obtain the high-resolution original image and the wrinkle mask bounding box image; the high-resolution original image and the wrinkle mask bounding box image are compressed according to the preset resolution to obtain the low-resolution original image to be redrawn and the redrawing area mask image.

[0070] That is, the embodiment of the present invention calculates the wrinkle bounding box according to the wrinkle mask image, reduces the image processing range, and cuts out the bounding box area from the original image and the wrinkle mask image to obtain the following: Figure 5 The cropped high-resolution original image and the wrinkle mask bounding box image are shown in FIG. , and the high-resolution original image and the wrinkle mask bounding box image are compressed to the input resolution required for subsequent data processing, that is, the following is obtained: Figure 6 The compressed low-resolution original image to be redrawn is shown, and Figure 7 The mask image of the redrawing area is shown. The low-resolution original image to be redrawn only contains the wrinkle area that needs to be redrawn, and has a low resolution; the redrawing area mask image is a single-channel grayscale image, and the resolution is consistent with the resolution of the low-resolution original image to be redrawn, and the pixel value is 0 or 255, 0 means that the pixels at the same pixel position in the original image to be redrawn do not need to be redrawn, and 255 means that the pixels at the same pixel position in the original image to be redrawn need to be redrawn.

[0071] Step S102, using a preset target wrinkle removal redrawing model to process the low-resolution original image to be redrawn and the redrawing area mask image to obtain a low-resolution wrinkle removal result image.

[0072] The low-resolution wrinkle removal result in the embodiment of the present invention is as follows: Figure 8 As shown in the figure, the wrinkle-free skin result is obtained by redrawing the original image with low resolution. That is to say, except for the mask part corresponding to the mask image of the redrawn area, the content of other parts is the same as the original image with low resolution. Figure 1It should be noted that before processing the low-resolution original image to be redrawn and the mask image of the redrawn area, it is necessary to first construct and train a target wrinkle removal and redrawing model. Specifically, a neural network model based on the GAN (Generative Adversarial Network) technology can be constructed as a wrinkle removal and redrawing model; the wrinkle removal model includes a generator sub-model for removing wrinkles and a discriminator sub-model for identifying whether wrinkles are removed and whether the repair result is true; the wrinkle removal and redrawing model is trained to obtain a target wrinkle removal and redrawing model for repairing and redrawing a skin with a realistic texture and no wrinkles on the wrinkle mask area of ​​the original image.

[0073] The training of the wrinkle removal and redrawing model according to the embodiment of the present invention may specifically include the following steps: obtaining two sets of skin image data; the two sets of skin image data include a skin image group and a random skin mask image group; each skin image contained in the skin image group has a unique corresponding matching relationship with each skin mask image contained in the random skin mask image group; the wrinkle removal and redrawing model is trained using the two sets of skin image data to obtain a universal skin image repair model; the universal skin image repair model is used to redraw the skin with a realistic texture in the random skin mask area of ​​the original image; obtaining three sets of wrinkle image data; the three sets of wrinkle image data include a wrinkle skin image group, a wrinkle mask image group and a wrinkle removal result image group; each wrinkle skin image contained in the wrinkle skin image group, each wrinkle mask image contained in the wrinkle mask image group, and each wrinkle removal result image contained in the wrinkle removal result image group have a unique corresponding matching relationship; the universal skin image repair model is trained using the three sets of wrinkle image data to obtain a target wrinkle removal and redrawing model.

[0074] That is to say, it is necessary to first obtain two sets of skin image data with matching correspondence, including a skin image group and a random skin mask image group, and three sets of wrinkle image data with matching correspondence, including a wrinkle skin image group, a wrinkle mask image group, and a wrinkle removal result image group, as the training input of the model. Among them, the number of two sets of materials in the two sets of skin image data is consistent, and a configuration file can be set to specify the correspondence between the two sets of images. For example, if image A1 in the skin image group and image A2 in the random skin mask image group are matched, it means that image A2 points out a local skin area of ​​image A1. The number of materials in each set of the three sets of wrinkle image data is consistent, and a configuration file can be set to specify the correspondence between the three sets of images. For example, if image A1 in the wrinkle skin image group, image A2 in the wrinkle mask image group, and image A3 in the wrinkle removal result image group are matched, it means that the wrinkle area of ​​image A1 is pointed out pixel by pixel by image A2, and image A3 is the result of redrawing A1 by removing wrinkles on the wrinkle area of ​​A2.

[0075] Furthermore, the two sets of skin image data are used as training data to train the wrinkle removal and redrawing model until the indicators converge, and the converged model is used as a general skin image repair model. The effect of the general skin image repair model is to repair and redraw the skin with real texture on the random skin mask area of ​​the original image. Then, the three sets of wrinkle images are used as training data for the general skin image repair model, and the model is trained and fine-tuned until the indicators converge, and a target wrinkle removal and redrawing model that can achieve wrinkle removal effect is obtained. The effect of the target wrinkle removal and redrawing model is to repair and redraw the skin with real texture and no wrinkles on the wrinkle mask area of ​​the original image.

[0076] Step S103: using an upsampling reversible hybrid algorithm, the low-resolution wrinkle removal result image is processed into a high-resolution wrinkle removal result image having a resolution consistent with that of the high-resolution original image.

[0077] It can be understood that, in the low-resolution wrinkle removal result image processed in step S102, except for the mask portion corresponding to the mask image of the redrawn area, the contents of other portions are consistent with the low-resolution original image to be redrawn. Figure 1 Therefore, the purpose of step S103 is to restore the low-resolution part of the low-resolution wrinkle removal result image to a high-resolution image. Specifically, the embodiment of the present invention uses the trained upsampling reversible hybrid algorithm to process the low-resolution wrinkle removal result image. First, a hybrid algorithm containing undetermined parameters can be constructed, and the formula is as follows:

[0078] Result=f(Base,a0,a1,…,an)+f(Base,b0,b1,…,bn)×Overlay

[0079] Among them, Base is the underlying image, Overlay is the mixed image, Result is the result image, f is a function composed of the non-negative integer power of the underlying image and its corresponding undetermined coefficients; a0, a1, ..., an and b0, b1, ..., bn are all undetermined coefficients to be trained. By determining the undetermined coefficients, the upsampling reversible mixing algorithm can be obtained. The upsampling reversible mixing algorithm can use the underlying image Base and the mixed image Overlay to calculate the result image Result, and can also infer the mixed image Overlay from the underlying image Base and the result image Result.

[0080] It should be noted that, in the embodiment of the present invention, for the training of the hybrid algorithm, the three groups of wrinkle image data in step S102 may be used to optimize the training of the undetermined parameters included in the hybrid algorithm to obtain an up-sampling reversible hybrid algorithm.

[0081] In an embodiment of the present invention, the input data of the upsampling reversible hybrid algorithm are a low-resolution original image to be redrawn, a low-resolution wrinkle removal result image and a high-resolution original image. The low-resolution wrinkle removal result image can be repaired to the high resolution before compression, and the high-definition skin details and texture of the original image can be restored, so that the target wrinkle removal and redrawing model can use the low-resolution input image for fast inference, reducing time consumption while not losing high-definition image details.

[0082] Specifically, the upsampling reversible hybrid algorithm is used to process the low-resolution wrinkle removal result image into a high-resolution wrinkle removal result image with the same resolution as the high-resolution original image. The upsampling reversible hybrid algorithm can be used for reverse operation, and the low-resolution original image to be redrawn is used as the bottom image, and the low-resolution wrinkle removal result image is used as the result image to calculate the mixed image, which is as follows: Fig. 9 The wrinkle-removed mixed image shown in the figure; the upsampling reversible mixing algorithm is used for forward operation, the high-resolution original image is used as the bottom image, the wrinkle-removed mixed image is used as the mixed image, and the result image is calculated, which is as shown in the figure. Fig.10 High-resolution wrinkle removal results are shown.

[0083] The characteristic of the high-resolution wrinkle removal result image is that the resolution is improved to the same level as the high-resolution original image. Figure 1 It not only removes wrinkles but also retains the skin details and texture of the original image.

[0084] Step S104: using the high-resolution wrinkle removal result image to replace the corresponding area in the high-resolution original image to obtain a target wrinkle removal image.

[0085] Specifically, the high-resolution result image can be pasted back to the corresponding area of ​​the wrinkle bounding box in the high-resolution original image to obtain Fig.11 The complete and high-resolution de-wrinkled image of the target is shown.

[0086] The embodiment of the present invention provides a method and device for wrinkle removal and redrawing of a portrait in an image, and a computer device, including: obtaining an original image to be wrinkled and a wrinkle mask map of the original image, performing wrinkle removal boundary box re-cropping and resolution compression processing operations on the original image and the wrinkle mask map to obtain a high-resolution original image, a low-resolution original image to be redrawn, and a redrawing area mask map; using a preset target wrinkle removal and redrawing model to process the low-resolution original image to be redrawn and the redrawing area mask map to obtain a low-resolution wrinkle removal result map; using an upsampling reversible hybrid algorithm to process the low-resolution wrinkle removal result map into a high-resolution wrinkle removal result map with the same resolution as the high-resolution original image; using the high-resolution wrinkle removal result map to replace the corresponding area in the high-resolution original image to obtain a target wrinkle removal image. The present invention has at least the following advantages:

[0087] 1. By using a large amount of skin image data to train the basic skin repair model and combining it with fine-tuning training for wrinkle removal tasks, the effect stability of the wrinkle removal and redrawing model has been significantly improved, and the wrinkle removal effect in complex skin color and wrinkle scenes is realistic and coherent;

[0088] 2. Accurate and controllable automatic wrinkle removal is achieved by accurately locating wrinkle areas and manually editing through skin segmentation models;

[0089] 3. Use model training to implement an upsampling reversible hybrid algorithm, which can restore the wrinkle removal result image from low resolution to high resolution, making the wrinkle-removed skin high-definition and realistic. On the premise of ensuring the high-definition and realistic skin after wrinkle removal, the model is allowed to use low-resolution images for inference, which reduces time consumption and memory usage, and realizes fast automatic wrinkle removal processing and smooth manual editing on mobile terminals, taking into account both performance and effects.

[0090] Further, as Figure 1 In a specific implementation, an embodiment of the present invention provides a device for removing wrinkles and redrawing a portrait in a picture, such as Fig.12 As shown, the apparatus may include: an image acquisition module 1210 , an image wrinkle removal module 1220 , an image processing module 1230 , and an image replacement module 1240 .

[0091] The image acquisition module 1210 can be used to acquire the original image to be wrinkle-removed and the wrinkle mask image of the original image, perform wrinkle-removal boundary box re-cropping and resolution compression processing operations on the original image and the wrinkle mask image to obtain a high-resolution original image, a low-resolution original image to be redrawn, and a redrawing area mask image;

[0092] The image wrinkle removal module 1220 may be used to process the low-resolution original image to be redrawn and the redrawing area mask image using a preset target wrinkle removal redrawing model to obtain a low-resolution wrinkle removal result image;

[0093] The image processing module 1230 may be used to process the low-resolution wrinkle removal result image into a high-resolution wrinkle removal result image having a resolution consistent with that of the high-resolution original image by using an upsampling reversible hybrid algorithm;

[0094] The image replacement module 1240 may be used to replace a corresponding region in the high-resolution original image with the high-resolution wrinkle removal result image to obtain a target wrinkle removal image.

[0095] Alternatively, if Fig.13 As shown, another embodiment of the present invention provides a wrinkle removal and redrawing device for a portrait in a picture, which also includes: a model construction module 1250 and an algorithm construction module 1260.

[0096] The model building module 1250 can be used to build a neural network model based on GAN technology as a wrinkle removal and repainting model; the wrinkle removal model includes a generator sub-model for removing wrinkles and a discriminator sub-model for identifying whether wrinkles are removed and whether the repair result is true;

[0097] The wrinkle removal and redrawing model is trained to obtain a target wrinkle removal and redrawing model for repairing and redrawing skin with realistic texture and without wrinkles on the wrinkle mask area of ​​the original image.

[0098] Optionally, the model building module 1250 can also be used to obtain two groups of skin image data; the two groups of skin image data include a skin image group and a random skin mask image group; each skin image included in the skin image group has a unique corresponding matching relationship with each skin mask image included in the random skin mask image group;

[0099] The wrinkle removal and redrawing model is trained using two sets of skin image data to obtain a universal skin image restoration model; the universal skin image restoration model is used to redraw the skin with realistic texture in the random skin mask area of ​​the original image;

[0100] Acquire three groups of wrinkle image data; the three groups of wrinkle image data include a wrinkle skin image group, a wrinkle mask image group and a wrinkle removal result image group; each wrinkle skin image included in the wrinkle skin image group, each wrinkle mask image included in the wrinkle mask image group and each wrinkle removal result image included in the wrinkle removal result image group have a unique corresponding matching relationship;

[0101] The three sets of wrinkle image data are used to train the universal skin image restoration model to obtain the target wrinkle removal and repainting model.

[0102] The algorithm construction module 1260 can be used to construct a hybrid algorithm including undetermined parameters, and the formula is as follows:

[0103] Result=f(Base,a0,a1,…,an)+f(Base,b0,b1,…,bn)×Overlay

[0104] Among them, Base is the underlying image, Overlay is the mixed image, Result is the result image, f is a function composed of the non-negative integer powers of the underlying image and its corresponding undetermined coefficients; a0, a1, ..., an and b0, b1, ..., bn are all undetermined coefficients;

[0105] Three sets of wrinkle image data are used to optimize and train the unknown parameters contained in the hybrid algorithm, and an upsampling reversible hybrid algorithm is obtained.

[0106] Optionally, the image acquisition module 1210 may also be used to acquire an original image to be wrinkle-removed;

[0107] Use the preset wrinkle segmentation model to process the original image to obtain a wrinkle mask image; or,

[0108] Receive the painting data generated by the user's manual painting based on the original picture, and generate a wrinkle mask image.

[0109] Optionally, the image wrinkle removal module 1220 may also be used to select a wrinkle boundary box of a wrinkle mask image;

[0110] For the original image and the wrinkle mask image, the corresponding areas of the wrinkle bounding box are cropped respectively to obtain the high-resolution original image and the wrinkle mask bounding box image;

[0111] Compress the high-resolution original image and the wrinkle mask boundary box image according to the preset resolution to obtain the low-resolution original image to be redrawn and the redrawing area mask image;

[0112] The redrawn area mask image is a single-channel grayscale image, and its resolution is consistent with the resolution of the low-resolution original image to be redrawn.

[0113] Optionally, the image processing module 1230 may also be used to perform a reverse operation using an upsampling reversible hybrid algorithm, taking the low-resolution original image to be redrawn as the underlying image and the low-resolution wrinkle removal result image as the result image, and calculating a hybrid image, namely, a wrinkle removal hybrid image;

[0114] The upsampling reversible hybrid algorithm is used for forward operation, with the high-resolution original image as the underlying image and the wrinkle removal hybrid image as the hybrid image, and the result image is calculated, which is the high-resolution wrinkle removal result image.

[0115] Optionally, the image replacement module 1240 may also be used to paste the high-resolution result image back to the corresponding area of ​​the wrinkle boundary box in the high-resolution original image to obtain a complete and high-resolution target wrinkle-removed image.

[0116] It should be noted that for other corresponding descriptions of the functional modules involved in the device for removing wrinkles and redrawing a portrait in a picture provided by an embodiment of the present invention, reference can be made to Figure 1 The corresponding description of the method shown will not be repeated here.

[0117] Based on the above Figure 1 The method shown and Fig.13 The embodiment of the device shown in the figure, the embodiment of the present invention also provides a physical structure diagram of a computer device, such as Fig.14 As shown, the computer device may include a communication bus, a processor, a memory and a communication interface, and may also include an input / output interface and a display device, wherein each functional unit may communicate with each other through the bus. The memory stores a computer program, and the processor is used to execute the program stored in the memory and execute the steps of the method for removing wrinkles and redrawing a portrait in a picture in the above embodiment.

[0118] Those skilled in the art can clearly understand that the specific working processes of the systems, devices, modules and units described above can refer to the corresponding processes in the aforementioned method embodiments, and for the sake of brevity, they are not further described here.

[0119] In addition, the functional units in various embodiments of the present invention may be physically independent of each other, or two or more functional units may be integrated together, or all functional units may be integrated into one processing unit. The above integrated functional units may be implemented in the form of hardware, or in the form of software or firmware.

[0120] Those skilled in the art can understand that if the integrated functional unit is implemented in the form of software and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention can essentially or all or part of the technical solution can be embodied in the form of a software product. The computer software product is stored in a storage medium, which includes a number of instructions to enable a computing device (such as a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the method described in each embodiment of the present invention when running the instructions. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM), random access memory (RAM), magnetic disk or optical disk and other media that can store program codes.

[0121] Alternatively, all or part of the steps of implementing the aforementioned method embodiments may be accomplished by hardware associated with program instructions (such as a computing device such as a personal computer, a server, or a network device), and the program instructions may be stored in a computer-readable storage medium. When the program instructions are executed by a processor of a computing device, the computing device executes all or part of the steps of the methods described in the embodiments of the present invention.

[0122] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that within the spirit and principles of the present invention, the technical solutions described in the aforementioned embodiments can still be modified, or some or all of the technical features therein can be replaced by equivalents. However, these modifications or replacements do not deviate from the protection scope of the present invention.

Claims

1. A method for removing wrinkles and redrawing a portrait in a picture, characterized in that: The method comprises: Obtaining an original image to be wrinkle-removed and a wrinkle mask image of the original image, performing wrinkle-removal boundary box re-cropping and resolution compression processing operations on the original image and the wrinkle mask image to obtain a high-resolution original image, a low-resolution original image to be redrawn, and a redrawing area mask image; The low-resolution original image to be redrawn and the redrawing area mask image are processed using a preset target wrinkle removal redrawing model to obtain a low-resolution wrinkle removal result image; Using an upsampling reversible hybrid algorithm, the low-resolution wrinkle removal result image is processed into a high-resolution wrinkle removal result image having a resolution consistent with that of the high-resolution original image; The high-resolution wrinkle removal result image is used to replace the corresponding area in the high-resolution original image to obtain a target wrinkle removal image.

2. The method according to claim 1, characterized in that The step of obtaining the original image to be wrinkle-removed and a wrinkle mask image of the original image includes: Obtain the original image to be wrinkle-removed; Processing the original image using a preset wrinkle segmentation model to obtain the wrinkle mask image; or, The painting data generated by manual painting by the user based on the original picture is received to generate the wrinkle mask image.

3. The method according to claim 1, characterized in that The wrinkle removal boundary box re-cropping and resolution compression processing operations are performed on the original image and the wrinkle mask image to obtain a high-resolution original image, a low-resolution original image to be redrawn, and a redrawing area mask image, including: Selecting a wrinkle bounding box of the wrinkle mask image; For the original image and the wrinkle mask image, respectively crop the corresponding areas of the wrinkle bounding box to obtain a high-resolution original image and a wrinkle mask bounding box image; Compressing the high-resolution original image and the wrinkle mask boundary box image according to a preset resolution to obtain a low-resolution original image to be redrawn and a redrawing area mask image; The redrawing area mask image is a single-channel grayscale image, and its resolution is consistent with the resolution of the low-resolution original image to be redrawn.

4. The method according to claim 1, characterized in that Before processing the low-resolution original image to be redrawn and the redrawing area mask image using the preset target wrinkle removal redrawing model, the method further includes: Constructing a neural network model based on GAN technology as a wrinkle removal and repainting model; the wrinkle removal model includes a generator sub-model for removing wrinkles and a discriminator sub-model for identifying whether wrinkles have been removed and whether the repair result is true; The wrinkle removal and redrawing model is trained to obtain a target wrinkle removal and redrawing model for repairing and redrawing skin with realistic texture and without wrinkles on the wrinkle mask area of ​​the original image.

5. The method according to claim 4, characterized in that The step of training the wrinkle removal and redrawing model comprises: Acquire two groups of skin image data; the two groups of skin image data include a skin image group and a random skin mask image group; each skin image included in the skin image group has a unique corresponding matching relationship with each skin mask image included in the random skin mask image group; The wrinkle removal and redrawing model is trained using the two sets of skin image data to obtain a universal skin image restoration model; the universal skin image restoration model is used to redraw the skin with realistic texture in the random skin mask area of ​​the original image; Acquire three groups of wrinkle image data; the three groups of wrinkle image data include a wrinkle skin image group, a wrinkle mask image group and a wrinkle removal result image group; each wrinkle skin image included in the wrinkle skin image group, each wrinkle mask image included in the wrinkle mask image group and each wrinkle removal result image included in the wrinkle removal result image group have a unique corresponding matching relationship; The three groups of wrinkle image data are used to train the universal skin image restoration model to obtain the target wrinkle removal and redrawing model.

6. The method according to claim 5, characterized in that Before processing the low-resolution wrinkle removal result image into a high-resolution wrinkle removal result image having the same resolution as the high-resolution original image by using an upsampling reversible hybrid algorithm, the method further comprises: Construct a hybrid algorithm with unknown parameters, the formula is as follows: Result=f(Base,a0,a1,…,an)+f(Base,b0,b1,…,bn)×Overlay Among them, Base is the underlying image, Overlay is the mixed image, Result is the result image, f is a function composed of the non-negative integer powers of the underlying image and its corresponding undetermined coefficients; a0, a1, ..., an and b0, b1, ..., bn are all undetermined coefficients; The three groups of wrinkle image data are used to optimize and train the undetermined parameters included in the hybrid algorithm to obtain an upsampling reversible hybrid algorithm.

7. The method according to claim 6, characterized in that The step of processing the low-resolution wrinkle removal result image into a high-resolution wrinkle removal result image having the same resolution as the high-resolution original image by using an upsampling reversible hybrid algorithm comprises: The upsampling reversible hybrid algorithm is used to perform inverse operation, the low-resolution original image to be redrawn is used as the bottom image, the low-resolution wrinkle removal result image is used as the result image, and a hybrid image is calculated, that is, a wrinkle removal hybrid image; The upsampling reversible hybrid algorithm is used to perform a forward operation, the high-resolution original image is used as the underlying image, the wrinkle-removed hybrid image is used as the hybrid image, and the result image is calculated, which is the high-resolution wrinkle-removed result image.

8. The method according to claim 3, characterized in that The step of replacing a corresponding region in the high-resolution original image with the high-resolution wrinkle removal result image to obtain a target wrinkle removal image includes: The high-resolution result image is pasted back to the high-resolution original image in the corresponding area of ​​the wrinkle boundary box to obtain a complete and high-resolution target wrinkle-removed image.

9. A device for removing wrinkles and redrawing a portrait in a picture, characterized in that: The device comprises: An image acquisition module is used to acquire an original image to be wrinkle-removed and a wrinkle mask image of the original image, and perform wrinkle-removal boundary box re-cropping and resolution compression processing operations on the original image and the wrinkle mask image to obtain a high-resolution original image, a low-resolution original image to be redrawn, and a redrawing area mask image; An image wrinkle removal module is used to process the low-resolution original image to be redrawn and the redrawing area mask image using a preset target wrinkle removal redrawing model to obtain a low-resolution wrinkle removal result image; An image processing module, used for processing the low-resolution wrinkle removal result image into a high-resolution wrinkle removal result image having a resolution consistent with that of the high-resolution original image by using an upsampling reversible hybrid algorithm; The image replacement module is used to replace the corresponding area in the high-resolution original image with the high-resolution wrinkle removal result image to obtain a target wrinkle removal image.

10. A computer device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the computer program, the method for removing wrinkles and redrawing a portrait in a picture according to any one of claims 1 to 8 is implemented.

Images