Image processing method and device, electronic equipment and storage medium

Abstract

Embodiments of the present disclosure provide an image processing method and device, electronic equipment and storage medium. The method comprises: obtaining a to-be-processed image comprising first hairstyle information, and determining a target mask image corresponding to a hair region of the to-be-processed image; splicing the to-be-processed image and the target mask image to obtain a target processing image; inputting the target processing image into a pre-trained hairstyle changing model to obtain a target display image corresponding to the to-be-processed image and comprising second hairstyle information; wherein the hairstyle changing model is trained based on a sample original image comprising a hair region, a sample mask image corresponding to the hair region of the sample original image, an expected image corresponding to the sample original image, and an expected mask image corresponding to the hair region of the expected image. Through the technical scheme disclosed in the embodiments of the present disclosure, the hairstyle effect after the user changes the hairstyle is truly displayed, thereby improving the effect experience of the user for hairstyle preview.

Description

TECHNICAL FIELD

[0001] The embodiment of the present disclosure relates to the image processing technology, and in particular to an image processing method and device, an electronic device and a storage medium. BACKGROUND

[0002] With the development of multimedia technology, people urgently need a method to automatically identify and change hairstyles. Computer vision technology makes this possible. Through a computer vision model, hairstyle factors in a person's picture can be automatically identified, analyzed and modified. Users can try different hairstyles through a computer model according to their preferences, compare the results, and then find a barber to achieve the most satisfactory hairstyle effect.

[0003] The prior art method of changing hairstyles based on a computer model has poor fit with the face, and cannot well meet the user's demand for real effect preview of the changed hairstyle. SUMMARY

[0004] The present disclosure provides an invention name to realize real display of the hairstyle effect after the user changes the hairstyle, thereby improving the user's effect experience of hairstyle preview.

[0005] In a first aspect, the embodiment of the present disclosure provides an image processing method, which comprises:

[0006] obtaining a to-be-processed image comprising first hairstyle information, and determining a target mask image corresponding to a hair region of the to-be-processed image;

[0007] splicing the to-be-processed image and the target mask image to obtain a target processing image;

[0008] inputting the target processing image into a pre-trained hairstyle changing model to obtain a target display image corresponding to the to-be-processed image and comprising second hairstyle information;

[0009] The hairstyle changing model is trained based on a sample original image comprising a hair region, a sample mask image corresponding to the hair region of the sample original image, an expected image corresponding to the sample original image, and an expected mask image corresponding to the hair region of the expected image.

[0010] In a second aspect, the embodiment of the present disclosure further provides an image processing device, which comprises:

[0011] an image acquisition module configured to obtain a to-be-processed image comprising first hairstyle information, and determine a target mask image corresponding to a hair region of the to-be-processed image;

[0012] an image splicing module configured to splice the to-be-processed image and the target mask image to obtain a target processed image;

[0013] a hairstyle changing module configured to input the target processed image into a pre-trained hairstyle changing model to obtain a target display image containing second hairstyle information corresponding to the to-be-processed image;

[0014] The hairstyle changing model is trained based on a sample original image including a hair region, a sample mask image corresponding to the hair region of the sample original image, an expected image corresponding to the sample original image, and an expected mask image corresponding to the hair region of the expected image.

[0015] In a third aspect, the present disclosure provides an electronic device, which includes:

[0016] one or more processors;

[0017] a storage device configured to store one or more programs,

[0018] When the one or more programs are executed by the one or more processors, the one or more processors implement the image processing method provided in any of the embodiments of the present disclosure.

[0019] In a fourth aspect, the present disclosure provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the image processing method provided in any of the embodiments of the present disclosure.

[0020] The technical solution of the present disclosure includes obtaining a to-be-processed image containing first hairstyle information and determining a target mask image corresponding to a hair region of the to-be-processed image; splicing the to-be-processed image and the target mask image to obtain a target processed image; inputting the target processed image into a pre-trained hairstyle changing model to obtain a target display image containing second hairstyle information corresponding to the to-be-processed image; and training the hairstyle changing model based on a sample original image including a hair region, a sample mask image corresponding to the hair region of the sample original image, an expected image corresponding to the sample original image, and an expected mask image corresponding to the hair region of the expected image. The above technical solution obtains a to-be-processed image before changing a hairstyle and a mask image of a hair region in the image, performs hairstyle changing based on the to-be-processed image and the mask image, and obtains an image containing a changed hairstyle. The hairstyle changing is assisted based on the mask image in the above operation, which improves the recognition of the hair region and the hairstyle processing, so that the hairstyle effect in the image after changing the hairstyle is more realistic, thereby improving the effect experience of the user for hairstyle preview. BRIEF DESCRIPTION OF DRAWINGS

[0021] The above described and other features, advantages, and aspects of the embodiments of the present disclosure will become more apparent with regard to the following detailed description, when taken in conjunction with the accompanying drawings. In the drawings, like reference numerals refer to like elements throughout. It should be understood that the drawings are schematic and elements in the drawings have not necessarily been drawn to scale.

[0022] Figure 1 is a flowchart of an image processing method provided by an embodiment of the present disclosure;

[0023] Figure 2 is a flowchart of another image processing method provided by an embodiment of the present disclosure;

[0024] Figure 3 is a flowchart of another image processing method provided by an embodiment of the present disclosure;

[0025] Figure 4 is a flowchart of another image processing method provided by an embodiment of the present disclosure;

[0026] Figure 5 is a structural schematic diagram of an image processing apparatus provided by an embodiment of the present disclosure;

[0027] Figure 6 is a structural schematic diagram of an electronic device provided by an embodiment of the present disclosure. DETAILED DESCRIPTION

[0028] Embodiments of the present disclosure will be described more fully hereinafter with reference to the accompanying drawings. While several embodiments of the present disclosure are shown in the drawings, it is understood that the present disclosure can be embodied in various forms and should not be construed as being limited to the embodiments set forth herein; rather, these embodiments are provided so that this disclosure will be thorough and complete, and fully convey the scope of the present disclosure to those skilled in the art. It should be understood that the drawings and embodiments of the present disclosure are for illustrative purposes only and are not intended to limit the scope of the present disclosure.

[0029] It should be understood that the various steps of the method embodiments of the present disclosure can be performed in different orders and / or in parallel. In addition, the method embodiments can include additional steps and / or omit performing the steps shown. The scope of the present disclosure is not limited in this regard.

[0030] The term "comprising" and variations thereof as used herein are open-ended, and mean "including but not limited to". The term "based on" means "based, at least in part, on". The term "one embodiment" means "at least one embodiment"; the term "another embodiment" means "at least one additional embodiment"; the term "some embodiments" means "at least some embodiments". Related terms are defined as follows.

[0031] It should be noted that the terms "first", "second", and the like in the present disclosure are merely used to distinguish different devices, modules or units, and do not imply the sequence or interdependence of the functions performed by these devices, modules or units.

[0032] It should be noted that the terms "one", "multiple" in the present disclosure are illustrative rather than restrictive, and those skilled in the art should understand that "one" or "multiple" should be understood as "one or more" unless otherwise explicitly indicated in the context.

[0033] The names of the messages or information exchanged between the plurality of devices in the embodiments of the present disclosure are only for illustrative purposes, and are not intended to limit the scope of the messages or information.

[0034] It can be understood that before using the technical solutions disclosed in the embodiments of the present disclosure, the type, scope of use, use scenario, etc. of the personal information involved in the present disclosure should be informed to the user and the authorization of the user should be obtained in a proper manner according to relevant laws and regulations.

[0035] For example, in response to receiving the active request of the user, the user is sent prompt information to explicitly prompt the user that the operation requested to be performed will require obtaining and using the personal information of the user. Thus, the user can voluntarily choose whether to provide personal information to the electronic device, application program, server or storage medium, etc. software or hardware performing the operation of the technical solutions of the present disclosure according to the prompt information.

[0036] As an optional but non-limiting implementation, in response to receiving the active request of the user, the user can be sent prompt information in the form of a pop-up window, for example, in which the prompt information can be presented in the form of text. In addition, the pop-up window can also carry selection controls for the user to select "agree" or "disagree" to provide personal information to the electronic device.

[0037] It can be understood that the above notification and user authorization process is only illustrative and does not limit the implementation of the present disclosure. Other ways that meet the relevant laws and regulations can also be applied to the implementation of the present disclosure.

[0038] It can be understood that the data involved in the present technical solution (including but not limited to the data itself, the acquisition or use of the data) should comply with the requirements of the relevant laws and regulations and relevant provisions.

[0039] Figure 1A flowchart of an image processing method provided by an embodiment of the present disclosure is shown. The embodiment of the present disclosure is applicable to the case of changing the hairstyle of a target object in an image. The method can be executed by an image processing device, which can be implemented in the form of software and / or hardware, and can be implemented by an electronic device, such as a mobile terminal, a PC terminal, or a server.

[0040] As shown in Figure 1 , the method includes:

[0041] In S110, a to-be-processed image including first hairstyle information is acquired, and a target mask image corresponding to a hair region of the to-be-processed image is determined.

[0042] In some other embodiments, in order to know the effect of changing the hairstyle, a user can fuse the hairstyle to be changed with the face of the user in advance according to the hairstyle changing technology, to show the effect of changing the hairstyle of the user. However, the hairstyle changing effect of the existing hairstyle changing technology is not ideal, so that the user cannot obtain a relatively real hairstyle effect preview. Therefore, in view of the above technical problems, the technical solution of the embodiment of the present disclosure acquires an original image before changing the hairstyle, and acquires a mask image of a hair region in the original image, performs hairstyle changing based on the to-be-processed image and the mask image, and obtains a display image including the changed hairstyle. In the above operation, the hairstyle changing is assisted based on the mask image, which improves the recognition and processing of the hair region, so that the hairstyle effect in the image after changing the hairstyle is more real.

[0043] In the embodiment of the present disclosure, the to-be-processed image can be an image including at least one object and hairstyle information of each object in the image; the object can be any person or animal in the image; the hairstyle information can be understood as description information of the hair or fur, and specifically, the hairstyle information can include display information of at least part of pixel points in the hair region or fur region in the to-be-converted image, such as display information of pixel points representing the length, color, and style of the hair; correspondingly, the first hairstyle information can be understood as display information of at least part of pixel points in the hair region or fur region of the object in the image before changing the hairstyle. Optionally, the acquired to-be-processed image can be original image data of the to-be-processed image, or can be a picture directly, which is not limited. Further, after the to-be-processed image is acquired, it is segmented and processed to determine the hair region in the to-be-processed image, and a target mask image corresponding to the hair region is obtained. Optionally, the segmentation method can be a traditional segmentation algorithm or a segmentation network model based on pre-training, and the segmentation method is not limited in the embodiment.

[0044] On the basis of the above-mentioned embodiments, the technical scheme of the present embodiment can further perform image preprocessing on the to-be-processed image when the to-be-processed image is obtained. Specifically, the preprocessing includes but is not limited to aligning the to-be-processed image based on a preset key point, cropping the aligned to-be-processed image into a preset size, and performing sharpening processing on the cropped to-be-processed image. It should be noted that the processing mode and processing order of the above-mentioned preprocessing are only optional embodiments, and it cannot be concluded that the technical preprocessing of the present embodiment can only be processed according to the above-mentioned embodiments. The preprocessing mode of the to-be-processed image of the present embodiment is not limited.

[0045] Optionally, the effect of aligning the to-be-processed image in the above-mentioned embodiments is to improve the recognition efficiency of the hairstyle information in the image in the subsequent process of changing the hairstyle of the to-be-processed image by the hairstyle changing model. The effect of cropping the image is that the appropriate image size can improve the processing efficiency in the subsequent processing process while ensuring the image quality. The effect of sharpening the image is to enhance the image features in the image to improve the efficiency of the subsequent model processing.

[0046] S120, splicing the to-be-processed image and the target mask image to obtain a target processing image.

[0047] In the embodiments of the present disclosure, the to-be-processed image is obtained based on the above-mentioned embodiments, and the target mask image corresponding to the hair region in the to-be-processed image. On this basis, the to-be-processed image and the target mask image are spliced to obtain a spliced target processing image, and a hairstyle changing operation is performed based on the target processing image.

[0048] Optionally, the image splicing method in the embodiment can be that pixel data of each channel in the to-be-processed image and the target mask image is respectively acquired, and each channel data of the to-be-processed image and each channel data of the target mask image are spliced in a preset order, so as to obtain the spliced target processing image. For example, each channel data of the to-be-processed image can include a processing image red channel, a processing image green channel and a processing image blue channel, each channel data of the target mask image can include a mask image red channel, a mask image green channel and a mask image blue channel, and the method of splicing each channel data can include splicing in the order of the processing image red channel, the processing image green channel, the processing image blue channel, the mask image red channel, the mask image green channel and the mask image blue channel, so as to obtain a six-channel target processing image. Optionally, the splicing can also be performed in the order of the processing image red channel, the mask image red channel, the processing image green channel, the mask image green channel, the processing image blue channel and the mask image blue channel, so as to obtain a six-channel target processing image. It should be noted that the above embodiment is only an optional splicing method, and other splicing methods can also be used in the embodiment, for example, the to-be-processed image and the target mask image are respectively input into a pre-trained splicing model, and a target processing image output by the splicing model is obtained. The splicing method in the above embodiment is not limited in the embodiment, that is, the to-be-processed image and the target mask image can be spliced.

[0049] In S130, the target processing image is input into a pre-trained hairstyle changing model to obtain a target display image containing second hairstyle information corresponding to the to-be-processed image.

[0050] In the embodiment of the present disclosure, the second hairstyle information can be understood as display information of at least part of the pixel points in the hair region or the hair region of the object after the hairstyle change. In the embodiment, the objects corresponding to the second hairstyle information and the first hairstyle information are the same object, but the information content of the two types of hairstyle information can be partially different, for example, different in length and color, and of course can be completely different. The embodiment is not limited in this regard.

[0051] In the embodiment, the hairstyle changing model is trained based on a sample original image including a hair region, a sample mask image corresponding to the hair region of the sample original image, an expected image corresponding to the sample original image, and an expected mask image corresponding to the hair region of the expected image. It should be noted that the sample original image can be understood as a hairstyle image of the object before the hairstyle change, and the expected image corresponding to the sample original image can be understood as a hairstyle image of the same object after the hairstyle change.

[0052] Optionally, based on the above operation mode, the target processing image corresponding to the to-be-processed image is obtained, and on this basis, the target processing image is taken as input information and input into the hairstyle changing model that has been pre-trained to obtain a target display image output by the model, which is a display image corresponding to the to-be-processed image and containing second hairstyle information.

[0053] It should be noted that if the image pre-processing is performed on the image input into the model before the hairstyle changing model, that is, the image pre-processing such as image alignment and image cropping is performed on the to-be-processed image, then on the basis of the target display image output by the model, the same degree of reverse alignment and image padding and other image post-processing need to be performed on the target display image to make the processed target display image consistent with the initially obtained to-be-processed image, thereby improving the preview effect of the image.

[0054] The technical scheme of the embodiment of the present disclosure specifically includes obtaining a to-be-processed image containing first hairstyle information, and determining a target mask image corresponding to a hair region of the to-be-processed image; splicing the to-be-processed image and the target mask image to obtain a target processing image; inputting the target processing image into a pre-trained hairstyle changing model to obtain a target display image corresponding to the to-be-processed image and containing second hairstyle information; wherein the hairstyle changing model is trained based on a sample original image containing a hair region, a sample mask image corresponding to the hair region of the sample original image, an expected image corresponding to the sample original image, and an expected mask image corresponding to the hair region of the expected image. The above technical scheme obtains the to-be-processed image containing the hairstyle before changing the hairstyle, and the mask image of the hair region in the image, performs hairstyle changing based on the to-be-processed image and the mask image, and obtains an image containing the changed hairstyle. In the above operation, the hairstyle changing is assisted based on the mask image, the recognition of the hair region and the hairstyle processing are improved, so that the hairstyle effect in the image after changing the hairstyle is more realistic, thereby improving the effect experience of the user for hairstyle preview.

[0055] Figure 2 The flowchart of another image processing method provided by the embodiment of the present disclosure, on the basis of any optional technical scheme in the embodiment of the present disclosure, the image processing method provided by the embodiment of the present disclosure further includes:

[0056] constructing a first sample image according to the sample original image containing the hair region and the sample mask image corresponding to the hair region of the sample original image;

[0057] constructing a second sample image according to the sample mask image, the expected image corresponding to the sample original image, and the expected mask image corresponding to the hair region of the expected image;

[0058] The first sample image and the second sample image are used to train a generative model of a pre-established generative adversarial network, to obtain the hairstyle changing model.

[0059] The explanations of the terms same as or corresponding to those in the above embodiments are not repeated here. See Figure 2 The image processing method provided in this embodiment includes the following steps.

[0060] In S210, a first sample image is constructed according to a sample original image containing a hair region and a sample mask image corresponding to the hair region of the sample original image.

[0061] In the embodiments of this disclosure, in order to realize the changing of the hairstyle information of the object in the image based on the hairstyle changing model, the hairstyle changing model needs to be trained in advance, so that the image containing the hairstyle information to be changed can be input into the trained hairstyle changing model, and an image including the hairstyle information after changing the hairstyle is obtained as the model output. In this embodiment, during the training of the hairstyle changing model, training samples need to be constructed in advance. The training samples in this embodiment include a first sample image containing image information of a to-be-processed image and a second sample image containing image information of the to-be-processed image and image information of an expected image corresponding thereto, and the hairstyle changing model is trained based on the first sample image and the second sample image to improve the processing performance of the hairstyle changing model.

[0062] It should be explained that the first sample image containing image information of the to-be-processed image specifically includes a sample original image containing a hair region and a sample mask image corresponding to the hair region of the sample original image. The sample mask region corresponding to the hair region in the to-be-processed image is taken as a component of the first sample image, and the purpose is to improve the recognition ability of the hair region of the object in the image during the training of the hairstyle changing model, so that the hairstyle changing model can pay more attention to learning the processing of the hair region, improve the processing performance of the hairstyle changing model for the hairstyle changing, and thus make the changed hairstyle more realistic.

[0063] Optionally, the method for constructing the first sample image based on the to-be-processed sample image and the sample mask image of the hair region in the to-be-processed sample image in this embodiment can include: splicing the to-be-processed sample image and the sample mask image to obtain the first sample image. Specifically, the red channel, the green channel, and the blue channel of the to-be-processed sample image can be acquired, and the red channel, the green channel, and the blue channel of the sample mask image can be acquired. The red channel of the to-be-processed image and the red channel of the sample mask image are spliced to obtain a spliced red channel, the green channel of the to-be-processed image and the green channel of the sample mask image are spliced to obtain a spliced green channel, and the blue channel of the to-be-processed image and the blue channel of the sample mask image are spliced to obtain a spliced blue channel. Further, the spliced red channel, the spliced green channel, and the spliced blue channel are spliced to obtain a spliced first sample image. Of course, the above embodiment is only an optional sample construction method, and the first sample image can also be constructed by using other methods in this embodiment, and the sample construction method in the above embodiment is not limited, that is, the first sample image can be constructed based on the to-be-processed image and the sample mask image.

[0064] In S220, a second sample image is constructed according to the sample mask image, an expected image corresponding to the sample original image, and an expected mask image corresponding to the hair region of the expected image.

[0065] In the embodiments of the present disclosure, the expected image corresponding to the sample original image is acquired, and the expected image and the expected mask image of the expected image are used as the components of the second sample image to train the hairstyle changing model, so that the hairstyle changing model can pay more attention to the identification and processing of the hair region in the expected image (i.e., the image after changing the hairstyle) in the training process, thereby making the details of the hair region of the changed hairstyle more realistic, and improving the realistic experience of the user for the hairstyle preview.

[0066] Optionally, the method for constructing the second sample image can include: determining a difference region between the sample mask image and the expected mask image corresponding to the hair region of the expected image corresponding to the sample original image; and processing the expected image according to the difference region to obtain the second sample image. Optionally, the method for processing the expected image according to the difference region can include: performing random occlusion processing on part of the difference region in the expected image to obtain the second sample image.

[0067] Specifically, the difference between the two mask images is determined by performing difference processing on the hair region of the expected image corresponding to the sample mask image and the sample original image. The difference region between the two mask images can be understood as a region with different pixel values in the two mask images. For example, the sample hair region corresponding to the sample mask image is determined, and the expected hair region corresponding to the expected mask image is determined. The pixel value coincidence degree of the corresponding pixel points of the two hair regions is calculated, and the region composed of the non-coincidence pixel points is the difference region between the sample mask image and the expected mask image. Of course, the difference region can also be determined based on other methods, which are not limited in the present embodiment.

[0068] On this basis, the difference region is determined to be located in the image region in the expected image, and the image region in the expected image is processed. The processed expected image is used as the second sample image.

[0069] Specifically, the method of processing the image region in the expected image can include: determining the image region in the expected image, and performing random occlusion processing on part of the image region, thereby obtaining the processed expected image, which is used as the second sample image. It should be noted that the selection of the part of the image region can be based on the position of the image region. For example, if the image region is located in the upper garment region, the part of the image region that needs to be processed can be determined based on the style and pattern of the upper garment. Of course, the part of the image region can also be randomly selected, which is not limited in the present embodiment. Optionally, in the case of determining the part of the image region, the random occlusion processing can be performed on the part of the image region. The part of the image region can also be filled based on the pixel values within the preset range of the region, and the part of the image region can also be processed in other ways, which are not limited in the present embodiment.

[0070] It is worth noting that the difference region is determined based on the sample mask image and the expected mask image, and the expected image is processed based on the difference region, thereby obtaining the processed expected image, which is used as the second sample image to train the hairstyle changing model. The above operation has the beneficial effect of improving the recognition of the difference before and after the hairstyle change in the training process of the hairstyle changing model, thereby improving the learning ability of the model in the changing process, to strengthen the authenticity of the details of the hair surrounding region in the image after the hairstyle change, thereby improving the authenticity of the hairstyle preview experience of the user.

[0071] S230, training the generation model of the pre-established generative adversarial network according to the first sample image and the second sample image, to obtain a hairstyle changing model.

[0072] In the embodiments of the present disclosure, the first loss function and the second loss function in the model training process of the first sample image and the second sample image can be determined respectively, and the generation model of the pre-established generative adversarial network is trained based on the first loss function and the second loss function respectively, so as to obtain the hairstyle changing model after training. In the above embodiments, different loss functions are used to train different sample images, which can improve the adaptability of model training, so that the hairstyle changing model after training can obtain a more realistic image after changing the hairstyle during the hairstyle changing process.

[0073] S240, obtaining a to-be-processed image including first hairstyle information, and determining a target mask image corresponding to a hair region of the to-be-processed image.

[0074] S250, splicing the to-be-processed image and the target mask image to obtain a target processing image.

[0075] S260, inputting the target processing image into the hairstyle changing model pre-trained to obtain a target display image corresponding to the to-be-processed image and containing second hairstyle information.

[0076] In the embodiments of the present disclosure, by obtaining the first sample image and the second sample image, training the generation model of the pre-established generative adversarial network based on the first sample image and the second sample image to obtain the hairstyle changing model, and using the trained hairstyle changing model to change the hairstyle based on the to-be-processed image and the mask image, an image including the changed hairstyle is obtained. In the above operation, the trained hairstyle changing model based on the sample mask image, the expected image and the expected mask image corresponding to the expected image is used to change the hairstyle, which improves the recognition of the hair region and the hairstyle processing, so that the hairstyle effect in the changed hairstyle image is more realistic, thereby improving the effect experience of the user for the hairstyle preview.

[0077] Figure 3 The flowchart of another image processing method provided in the embodiments of the present disclosure, in the embodiments of the present disclosure, any optional technical scheme can be used, and the pre-established generative adversarial network is trained according to the first sample image and the second sample image, including:

[0078] The first sample image and the second sample image are input into the generation model of the pre-established generative adversarial network for training to obtain a first output image and a second output image;

[0079] The first model loss of the generation model is calculated according to the first sample image, the first output image and the expected image, and the model parameters of the generation model are adjusted according to the first model loss;

[0080] A second model loss of the generation model is calculated according to the second output image and the expected image, and a model parameter of the generation model is adjusted according to the second model loss.

[0081] Wherein the explanations of the terms same or corresponding to those in the above embodiments are not repeated here. See Figure 3 The image processing provided in this embodiment comprises:

[0082] S310, constructing a first sample image according to a sample original image containing a hair region and a sample mask image corresponding to the hair region of the sample original image.

[0083] S320, constructing a second sample image according to the sample mask image, an expected image corresponding to the sample original image, and an expected mask image corresponding to a hair region of the expected image.

[0084] S330, inputting the first sample image and the second sample image into a generation model of a pre-established generative adversarial network respectively for training to obtain a first output image and a second output image.

[0085] In the embodiments of the present disclosure, the generation model of the pre-established generative adversarial network is acquired, the first sample image is input into the generation model to obtain a first output image output by the model, and the second sample image is input into the generation model to obtain a second output image output by the model. Further, the generation model is trained based on the first sample image, the second sample image, the first output image and the second output image to obtain a trained hairstyle changing model.

[0086] S340, calculating a first model loss of the generation model according to the first sample image, the first output image and the expected image, and adjusting a model parameter of the generation model according to the first model loss.

[0087] In the embodiments of the present disclosure, the first model loss comprises a first generation loss and an adversarial loss. Optionally, the method of calculating the first model loss of the generation model according to the first sample image, the first output image and the expected image can comprise: calculating a first generation loss of the generation model based on the first output image and the expected image, calculating an adversarial loss of the generation model based on the first sample image, the first output image and the expected image, and generating the first model loss based on the first generation loss and the adversarial loss.

[0088] In this embodiment, the first generation loss includes a first regression loss and / or a first perception loss; optionally, the method for calculating the first generation loss of the generation model based on the first output image and the expected image can include: calculating a first perception loss between the first output image and the expected image based on a preset perception loss function; and / or, calculating a first regression loss between the first output image and the expected image based on a Gram matrix and / or a mean absolute error function.

[0089] Specifically, the method for constructing the first perception loss can include: inputting the first output image and the expected image into a pre-trained convolutional neural network for feature extraction to obtain first image features and second image features, and calculating the first perception loss based on the first image features and the second image features based on a preset perception loss function. Optionally, the pre-trained convolutional neural network can include but is not limited to a VGG16 network. Optionally, the method for calculating the first perception loss based on the first image features and the second image features based on the preset perception loss function can include: using an L1 loss function to calculate the loss function of the first image features and the second image features to obtain the first perception loss. Exemplarily, the expression of the L1 loss function can be as follows:

[0090]

[0091] wherein y represents the first image features, represents the second image features; and m represents the number of pixel points.

[0092] It should be noted that the above preset perception loss function is only an exemplary introduction of this embodiment, and other perception loss functions can also be used to calculate the first perception loss based on the first image features and the second image features in this embodiment, which is not limited in this embodiment.

[0093] In this embodiment, the first regression loss includes a first regression sub-loss and a second regression sub-loss; optionally, the method for calculating the first regression loss between the first output image and the expected image based on the Gram matrix and / or the mean absolute error function can include: calculating a first regression sub-loss between the first output image and the expected image based on the Gram matrix, and calculating a second regression sub-loss between the first output image and the expected image based on the mean absolute error function; based on the above embodiment, the first regression loss is determined based on the first regression sub-loss and / or the second regression sub-loss.

[0094] Specifically, the method for calculating the first regression sub-loss between the first output image and the expected image based on the Gram matrix can include: inputting the first output image and the expected image into a pre-trained convolutional neural network for feature extraction to obtain first image features and second image features, and calculating the first regression sub-loss based on the Gram matrix for the first image features and the second image features. Optionally, the pre-trained convolutional neural network can include but is not limited to a VGG16 network.

[0095] Specifically, the method for calculating the second regression sub-loss between the first output image and the expected image based on the mean absolute error function can include: substituting image data of the first output image and image data of the expected image into the mean absolute error function, and determining the second regression sub-loss between the output image and the expected image based on the mean absolute error function. Optionally, the mean absolute error function includes but is not limited to an L1 loss function; for example, the L1 loss function is used to calculate the loss function between the first output image and the expected image to obtain the second regression sub-loss. For example, the expression of the L1 loss function can be as follows:

[0096]

[0097] wherein y represents the first output image, represents the expected image; and m represents the number of pixel points.

[0098] It is worth noting that the above-mentioned embodiments of determining the first regression sub-loss based on the Gram matrix and determining the second regression sub-loss based on the mean absolute error function are only optional embodiments. The technical solution of the present embodiment can also determine the first regression sub-loss based on the mean absolute error function, and determine the second regression sub-loss based on the Gram matrix. It can also determine the first regression sub-loss and the second regression sub-loss based on the Gram matrix at the same time, or determine the first regression sub-loss and the second regression sub-loss based on the mean absolute error function at the same time. The present embodiment does not limit the way of determining the first regression loss.

[0099] Optionally, the method for calculating the adversarial loss of the generation model based on the first sample image, the first output image and the expected image can include: splicing the first sample image and the first output image to obtain a fake sample image; splicing the first sample image and the expected image to obtain a true sample image; discriminating the fake sample image and the true sample image based on a pre-trained discriminator model to obtain a model discrimination result, and calculating the adversarial loss of the generative adversarial network according to the model discrimination result.

[0100] Specifically, each channel image of the first sample image, such as a red channel, a green channel and a blue channel, can be obtained, and each channel image of the first output image, such as a red channel, a green channel and a blue channel, can be obtained; the red channel, the green channel and the blue channel of the first sample image and the first output image are spliced respectively to obtain a spliced false sample image. The false sample image can be understood as a generated image obtained after the processing sample is processed based on the generation model, and the sample image generated based on the generated image and the processing sample, in other words, it can also be understood as a sample image that is not generated based on the target display image, which will be identified as a generated image sample image by the discrimination model.

[0101] Specifically, each channel image of the first sample image, such as a red channel, a green channel and a blue channel, can be obtained, and each channel image of the first output image, such as a red channel, a green channel and a blue channel, can be obtained; the red channel, the green channel and the blue channel of the first sample image and the first output image are spliced respectively to obtain a spliced false sample image. The false sample image can be understood as a generated image obtained after the processing sample is processed based on the generation model, and the sample image generated based on the generated image and the processing sample, in other words, it can also be understood as a sample image that is not generated based on the target display image, which will be identified as a generated image sample image by the discrimination model.

[0102] Of course, the splicing method in the above embodiment is only one optional sample construction method, and the false sample image and the true sample image can also be constructed by other methods in the present embodiment, and the sample construction method in the above embodiment is not limited.

[0103] Specifically, the trained discrimination model is obtained, and the obtained false sample image and true sample image are input into the trained discrimination model respectively to obtain the model discrimination result output by the model, and the adversarial loss of the generative adversarial network is calculated based on the model discrimination result.

[0104] Optionally, the method of calculating the adversarial loss of the generative adversarial network according to the model discrimination result can comprise: determining the adversarial discrimination loss of the discrimination model in the generative adversarial network according to the model discrimination result of the discrimination model on the false sample image and the true sample image and the expected mask image corresponding to the hair region of the expected image; determining the adversarial generation loss of the discrimination model in the generative adversarial network according to the model discrimination result of the discrimination model on the false sample image and the expected mask image corresponding to the hair region of the expected image; and calculating the adversarial loss of the generative adversarial network according to the adversarial discrimination loss and the adversarial generation loss.

[0105] Illustratively, the adversarial discrimination loss of the discrimination model in the generative adversarial network can be calculated based on the following formula:

[0106]

[0107] wherein, L Drepresents the adversarial discriminant loss of the discriminant model in the generative adversarial network, x represents a true sample image, D(x) represents a model discriminant result of the discriminant model on the true sample image, E x [log(1-D(x))] represents a model discriminant loss of the discriminant model on the true sample image, m represents an expected mask image corresponding to a hair region of an expected image, represents a false sample image, represents a model discriminant loss of the discriminant model on the false sample image, represents a model discriminant result of the discriminant model on the false sample image, and represents an exclusive or operation.

[0108] Specifically, in the above embodiment, the model discriminant loss of the discriminant model on the true sample image is generated based on the discriminant result of the discriminant model on the true sample image, the model discriminant loss of the discriminant model on the false sample image is generated based on the discriminant result of the discriminant model on the false sample image and the expected mask image, and the adversarial discriminant loss of the discriminant model in the generative adversarial network is generated based on the model discriminant loss of the discriminant model on the true sample image and the model discriminant loss of the discriminant model on the false sample image, so as to improve the learning ability of the discriminant model in discriminating the false sample image in the process of training the generative adversarial network, so that the display image output by the trained hairstyle changing model is more realistic, and the display effect of the target display image is improved.

[0109] Illustratively, the adversarial generation loss of the discriminant model in the generative adversarial network can be calculated based on the following formula:

[0110]

[0111] wherein, L G represents the adversarial generation loss of the generative model in the generative adversarial network.

[0112] It should be noted that the adversarial generation loss of the discriminant model in the generative adversarial network is determined based on the model discriminant result of the discriminant model on the false sample image and the expected mask image corresponding to the hair region of the expected image, so as to make the false sample image generated by the generative network more realistic in the process of training the generative adversarial network, so that the display image output by the trained hairstyle changing model is more realistic, and the display effect of the target display image is improved.

[0113] Illustratively, the adversarial discriminant loss of the discriminant model in the generative adversarial network can be calculated based on the following formula:

[0114] L GAN = ω1L D + ω2L G

[0115] wherein, LGAN denotes an adversarial loss of the generative adversarial network, ω1 denotes a weight of L D , and ω2 is a weight of L G .

[0116] It should be noted that, based on the above manner, the generative adversarial network is trained based on the adversarial discriminant loss and the adversarial generative loss of the generative adversarial network, so that the display image output by the hairstyle changing model after training is more realistic, and the display effect of the target display image is improved.

[0117] On the basis of the above embodiment, the model parameters of the generation model are adjusted according to the first model loss.

[0118] S350, calculating a second model loss of the generation model according to the second output image and the expected image, and adjusting the model parameters of the generation model according to the second model loss to obtain a hairstyle changing model.

[0119] In the embodiments of the present disclosure, the second model loss includes a second regression loss and / or a second perception loss; optionally, the method of calculating the second model loss of the generation model based on the second output image and the expected image can include: calculating a second perception loss between the second output image and the expected image based on a preset perception loss function; and / or, calculating a second regression loss between the second output image and the expected image based on a Gram matrix and / or a mean absolute error function, and generating the second model loss based on the second regression loss and / or the second perception loss.

[0120] Specifically, the method of constructing the second perception loss can include: inputting the second output image and the expected image into a pre-trained convolutional neural network for feature extraction to obtain first image features and second image features, and calculating the second perception loss based on the first image features and the second image features based on a preset perception loss function. Optionally, the pre-trained convolutional neural network can include but is not limited to VGG16 network. Optionally, the method of calculating the second perception loss based on the first image features and the second image features based on the preset perception loss function can include: using an L1 loss function to calculate the loss function of the first image features and the second image features to obtain the second perception loss. Exemplarily, the expression of the L1 loss function can be as follows:

[0121]

[0122] wherein y represents the first image features, represents the second image features; and m represents the number of pixel points.

[0123] It should be noted that the preset perceptual loss function is only used as an exemplary introduction in the embodiment, and other perceptual loss functions can be used to calculate the second perceptual loss of the first image feature and the second image feature in the embodiment, which is not limited in the embodiment.

[0124] In the embodiment, the second regression loss includes a first regression sub-loss and a second regression sub-loss. Optionally, the method for calculating the second regression loss between the second output image and the expected image based on the Gram matrix and / or the mean absolute error function can include: calculating the first regression sub-loss between the second output image and the expected image based on the Gram matrix, and calculating the second regression sub-loss between the second output image and the expected image based on the mean absolute error function; and the second regression loss is determined based on the first regression sub-loss and / or the second regression sub-loss.

[0125] Specifically, the method for calculating the first regression sub-loss between the second output image and the expected image based on the Gram matrix can include: inputting the second output image and the expected image into a pre-trained convolutional neural network for feature extraction to obtain a first image feature and a second image feature, and calculating the first regression sub-loss between the first image feature and the second image feature based on the Gram matrix. Optionally, the pre-trained convolutional neural network can include but is not limited to a VGG16 network.

[0126] Specifically, the method for calculating the second regression sub-loss between the second output image and the expected image based on the mean absolute error function can include: substituting the image data of the second output image and the image data of the expected image into the mean absolute error function, and determining the second regression sub-loss between the output image and the expected image based on the mean absolute error function. Optionally, the mean absolute error function includes but is not limited to an L1 loss function. For example, the L1 loss function is used to calculate the loss function between the second output image and the expected image to obtain the second regression sub-loss. For example, the expression of the L1 loss function can be as follows:

[0127]

[0128] wherein y represents the first output image, represents the expected image; and m represents the number of pixel points.

[0129] It is worth noting that the above-mentioned implementation of determining the first regression sub-loss based on the Gram matrix and determining the second regression sub-loss based on the mean absolute error function is only an optional implementation, and the technical solution of the present embodiment can also determine the first regression sub-loss based on the mean absolute error function and determine the second regression sub-loss based on the Gram matrix, and can also determine the first regression sub-loss and the second regression sub-loss based on the Gram matrix at the same time, or determine the first regression sub-loss and the second regression sub-loss based on the mean absolute error function at the same time. The present embodiment does not limit the way of determining the second regression loss.

[0130] On the basis of the above-mentioned embodiments, the model parameters of the generation model are adjusted according to the second model loss until the trained generation model meets the preset condition, and a trained hairstyle changing model is obtained.

[0131] S360, acquire a to-be-processed image including first hairstyle information, and determine a target mask image corresponding to a hair region of the to-be-processed image.

[0132] S370, splice the to-be-processed image and the target mask image to obtain a target processing image.

[0133] S380, input the target processing image into a pre-trained hairstyle changing model to obtain a target display image corresponding to the to-be-processed image and containing second hairstyle information.

[0134] In the embodiments of the present disclosure, by acquiring a first sample image and a second sample image, calculating a first model loss and a second model loss of a generation model of a pre-established generative adversarial network based on the first sample image and the second sample image, and adjusting the model parameters of the generation model based on the first model loss and the second model loss respectively, until the generation model is trained, a hairstyle changing model is obtained, and the trained hairstyle changing model is used to change the hairstyle based on the to-be-processed image and the mask image, to obtain an image including the changed hairstyle. In the above operation, the trained hairstyle changing model based on the sample mask image and the expected image and the expected mask image corresponding to the expected image is used to change the hairstyle, which improves the recognition of the hair region and the hairstyle processing, so that the hairstyle effect in the changed hairstyle image is more realistic, thereby improving the effect experience of the user for the hairstyle preview.

[0135] Figure 4 The flowchart of another image processing method provided by the embodiments of the present disclosure, based on any optional technical solution in the embodiments of the present disclosure, the hairstyle changing model includes an input layer, a hidden layer and an output layer, and the output layer is constructed based on a attention mechanism carrying a guided filter.

[0136] Wherein the explanations of the same or corresponding terms in the above embodiments are not repeated here. SeeFigure 4 The image processing method provided in the embodiment includes:

[0137] S410, obtaining a to-be-processed image including first hairstyle information, and determining a target mask image corresponding to a hair region of the to-be-processed image.

[0138] S420, splicing the to-be-processed image and the target mask image to obtain a target processing image.

[0139] S430, inputting the target processing image into a pre-trained hairstyle changing model to obtain a target display image including second hairstyle information corresponding to the to-be-processed image, wherein the hairstyle changing model includes an input layer, a hidden layer and an output layer, and the output layer is constructed based on an attention mechanism carrying a guided filter.

[0140] In the embodiment of the present disclosure, the hairstyle changing model includes an input layer, a hidden layer and an output layer, and the output layer is constructed based on an attention mechanism carrying a guided filter; on this basis, the method of inputting the target processing image into the pre-trained hairstyle changing model to obtain the target display image including the second hairstyle information corresponding to the to-be-processed image can include: inputting the target processing image through the input layer of the hairstyle changing model, performing hairstyle transformation processing via the hidden layer to obtain an intermediate processing image, wherein the intermediate processing image includes at least a red channel, a green channel, a blue channel and a transparent channel; processing the intermediate processing image based on the attention mechanism carrying a guided filter of the output layer of the hairstyle changing model to obtain the target display image including the second hairstyle information corresponding to the to-be-processed image.

[0141] Specifically, the target processing image obtained based on the above embodiment is input into the input layer of the hairstyle changing model, and the image data output by the input layer is transmitted to the hidden layer for hairstyle transformation processing to obtain the intermediate processing image output by the hidden layer. In the embodiment, the intermediate processing image includes at least a red channel, a green channel, a blue channel and a transparent channel; wherein the number of channels of the transparent channel can include at least one layer of transparent channel.

[0142] Specifically, on the basis of obtaining the intermediate processing image, the intermediate processing image is transmitted to the output layer, and the intermediate processing image is processed based on the attention mechanism carrying a guided filter in the output layer to obtain the target display image including the second hairstyle information corresponding to the to-be-processed image.

[0143] Optionally, the method for processing the intermediate processing image based on the attention mechanism carrying the guided filtering of the output layer of the hairstyle changing model can include: obtaining a color input image composed of a red channel, a green channel and a blue channel in the target processing image; performing grayscale processing on the color input image to obtain a target grayscale image; performing filtering processing on the target grayscale image and a transparent image composed of a transparent channel based on a preset guided filtering algorithm to obtain a first transparent image; multiplying the color processing image composed of the red channel, the green channel and the blue channel in the intermediate processing image and the first transparent image to obtain a first fusion image; multiplying the color input image and a second transparent image to obtain a second fusion image, wherein the transparency value in the second transparent image is obtained by subtracting the transparency value of the first transparent image from a preset transparency value; and adding the first fusion image and the second fusion image to obtain the target display image corresponding to the processing image and containing the second hairstyle information.

[0144] It should be explained that the preset transparency value in the embodiment can be a maximum range value of image pixel values, for example, 255; the preset transparency value can also be set to 1 according to a binary classification manner; the preset transparency value can also be determined as the maximum pixel value in the pixel data of each pixel point in the target display image; of course, other values can also be determined as the preset transparency value based on actual specific conditions, and the determination manner of the transparency value is not limited in the embodiment.

[0145] The technical scheme of the embodiment of the present disclosure specifically includes obtaining a processing image containing first hairstyle information, and determining a target mask image corresponding to the hair region of the processing image; splicing the processing image and the target mask image to obtain a target processing image; inputting the target processing image into a hairstyle changing model that has been pre-trained to obtain a target display image corresponding to the processing image and containing second hairstyle information; the hairstyle changing model includes an input layer, a hidden layer and an output layer, and the output layer is constructed based on an attention mechanism carrying guided filtering. The above technical scheme obtains the processing image containing the hairstyle before changing and the mask image of the hair region in the image, processes the processing image and the mask image based on the attention mechanism carrying the guided filtering of the output layer of the hairstyle changing model, and obtains the image of the changed hairstyle corresponding to the processing image. In the above operation, the hairstyle changing is assisted based on the mask image, the recognition of the hair region and the hairstyle processing are improved, so that the hairstyle effect in the image of the changed hairstyle is more realistic, thereby improving the effect experience of the user for the hairstyle preview.

[0146] Figure 5 A structure schematic diagram of an image processing device provided by the embodiment of the present disclosure is shown in Figure 5As shown, the apparatus comprises: an image acquisition module 510, an image splicing module 520, and a hairstyle changing module 530; wherein,

[0147] The image acquisition module 510 is configured to acquire a to-be-processed image comprising first hairstyle information, and determine a target mask image corresponding to a hair region of the to-be-processed image;

[0148] The image splicing module 520 is configured to splice the to-be-processed image and the target mask image to obtain a target processing image;

[0149] The hairstyle changing module 530 is configured to input the target processing image into a pre-trained hairstyle changing model to obtain a target display image comprising second hairstyle information corresponding to the to-be-processed image.

[0150] The hairstyle changing model is trained based on a sample original image comprising a hair region, a sample mask image corresponding to the hair region of the sample original image, an expected image corresponding to the sample original image, and an expected mask image corresponding to the hair region of the expected image.

[0151] On the basis of the above embodiments, the apparatus can further comprise:

[0152] The first sample image obtaining module is configured to construct a first sample image according to a sample original image comprising a hair region and a sample mask image corresponding to the hair region of the sample original image.

[0153] The second sample image obtaining module is configured to construct a second sample image according to the sample mask image, an expected image corresponding to the sample original image, and an expected mask image corresponding to the hair region of the expected image.

[0154] The hairstyle changing model training module is configured to train a generative model of a pre-established generative adversarial network according to the first sample image and the second sample image, to obtain the hairstyle changing model.

[0155] On the basis of the above embodiments, the second sample image obtaining module can comprise:

[0156] The differentiating region determining unit is configured to determine a differentiating region between the sample mask image and the expected mask image corresponding to the hair region of the sample original image and the expected image.

[0157] The second sample image obtaining unit is configured to process the expected image according to the differentiating region to obtain the second sample image.

[0158] On the basis of each of the above embodiments, optionally, the second sample image obtaining unit comprises:

[0159] The second sample image obtaining subunit is configured to perform random occlusion processing on the partial region in the differentiated region in the expected image to obtain a second sample image.

[0160] On the basis of each of the above embodiments, optionally, the hairstyle changing model training module comprises:

[0161] The output image obtaining sub-module is configured to input the first sample image and the second sample image into a generative model of a pre-established generative adversarial network for training to obtain a first output image and a second output image.

[0162] The first model loss determining sub-module is configured to calculate a first model loss of the generative model according to the first sample image, the first output image and the expected image, and adjust model parameters of the generative model according to the first model loss.

[0163] The second model loss determining sub-module is configured to calculate a second model loss of the generative model according to the second output image and the expected image, and adjust model parameters of the generative model according to the second model loss.

[0164] On the basis of each of the above embodiments, optionally, the first model loss comprises a first generative loss and an adversarial loss; the first generative loss comprises a first regression loss and / or a first perception loss, the first regression loss comprises a first regression sub-loss and a second regression sub-loss; and the second model loss comprises a second regression loss and / or a second perception loss.

[0165] On the basis of each of the above embodiments, optionally, the calculation of the first model loss of the generative model according to the first sample image, the first output image and the expected image comprises at least one of the following:

[0166] calculating a first perception loss between the first output image and the expected image based on a preset perception loss function;

[0167] calculating a first regression loss between the first output image and the expected image based on a Gram matrix and / or a mean absolute error function.

[0168] On the basis of each of the above embodiments, optionally, the first model loss determining sub-module comprises:

[0169] The false sample image obtaining unit is configured to splice the first sample image and the first output image to obtain a false sample image.

[0170] A true sample image obtaining unit is configured to splice the first sample image and the expected image to obtain a true sample image.

[0171] An adversarial loss generating unit is configured to perform discrimination on the false sample image and the true sample image based on a pre-trained discrimination model to obtain a model discrimination result, and calculate an adversarial loss of the generative adversarial network according to the model discrimination result.

[0172] On the basis of the above embodiments, the discrimination unit can optionally include:

[0173] An adversarial discrimination loss generating subunit is configured to determine an adversarial discrimination loss of the discrimination model in the generative adversarial network according to the model discrimination result of the discrimination model on the false sample image and the true sample image and an expected mask image corresponding to the hair region of the expected image.

[0174] An adversarial generation loss generating subunit is configured to determine an adversarial generation loss of the discrimination model in the generative adversarial network according to the model discrimination result of the discrimination model on the false sample image and the expected mask image corresponding to the hair region of the expected image.

[0175] An adversarial loss generating subunit is configured to calculate an adversarial loss of the generative adversarial network according to the adversarial discrimination loss and the adversarial generation loss.

[0176] On the basis of the above embodiments, the hairstyle changing model can include an input layer, a hidden layer and an output layer, and the output layer is constructed based on a guidance filter-carrying attention mechanism. The hairstyle changing module 510 includes:

[0177] An intermediate processing image obtaining sub-module is configured to input the target processing image into the input layer of the hairstyle changing model, perform hairstyle transformation processing via the hidden layer, and obtain an intermediate processing image, wherein the intermediate processing image includes at least a red channel, a green channel, a blue channel and a transparent channel.

[0178] A hairstyle changing sub-module is configured to process the intermediate processing image based on the guidance filter-carrying attention mechanism of the output layer of the hairstyle changing model to obtain a target display image corresponding to the to-be-processed image and containing second hairstyle information.

[0179] On the basis of the above embodiments, the hairstyle changing sub-module can include:

[0180] A gray processing unit is configured to obtain a color input image composed of a red channel, a green channel and a blue channel in the target processing image, perform gray processing on the color input image, and obtain a target gray image.

[0181] The filtering processing unit is configured to perform filtering processing on the target gray image and a transparent image composed of the transparent channel based on a preset guided filtering algorithm, to obtain a first transparent image.

[0182] The first fusion image obtaining unit is configured to multiply the color processing image composed of the red channel, the green channel and the blue channel in the intermediate processing image and the first transparent image, to obtain a first fusion image.

[0183] The second fusion image obtaining unit is configured to multiply the color input image and a second transparent image, to obtain a second fusion image, wherein the transparency value in the second transparent image is obtained by subtracting the transparency value of the first transparent image from a preset transparency value.

[0184] The hairstyle unit module is configured to add the first fusion image and the second fusion image, to obtain a target display image corresponding to the to-be-processed image and containing second hairstyle information.

[0185] The technical scheme of the embodiment of the present disclosure specifically comprises obtaining a to-be-processed image containing first hairstyle information, and determining a target mask image corresponding to a hair region of the to-be-processed image; splicing the to-be-processed image and the target mask image, to obtain a target processing image; inputting the target processing image into a hairstyle changing model trained in advance, to obtain a target display image corresponding to the to-be-processed image and containing second hairstyle information; wherein the hairstyle changing model is trained based on a sample original image containing a hair region, a sample mask image corresponding to the hair region of the sample original image, an expected image corresponding to the sample original image, and an expected mask image corresponding to the hair region of the expected image. The above technical scheme obtains a to-be-processed image containing a hairstyle before changing, and a mask image of the hair region in the image, performs hairstyle changing based on the to-be-processed image and the mask image, and obtains an image containing a changed hairstyle. In the above operation, the hairstyle changing is assisted based on the mask image, the recognition of the hair region and the hairstyle processing are improved, so that the hairstyle effect in the image after changing is more realistic, thereby improving the effect experience of the user for hairstyle preview.

[0186] The image processing apparatus provided by the embodiment of the present disclosure can execute the image processing method provided by any of the embodiments of the present disclosure, and has the corresponding function modules and beneficial effects of the execution method.

[0187] It should be noted that each unit and module included in the above apparatus is only divided according to the function logic, but is not limited to the above division, as long as the corresponding function can be realized; in addition, the specific names of each functional unit are only for convenient mutual distinction, and are not used to limit the protection scope of the embodiments of the present disclosure.

[0188] Figure 6 A structural diagram of an electronic device according to an embodiment of the disclosure is provided. Hereinafter, a structural diagram of an electronic device (e.g., a terminal device or a server) 500 suitable for implementing an embodiment of the disclosure is described with reference to FIG. 1. Figure 6 FIG. 1 illustrates a structural diagram of an electronic device (e.g., a terminal device or a server) 500 suitable for implementing an embodiment of the disclosure. The terminal device in an embodiment of the disclosure can include, but is not limited to, a mobile terminal such as a mobile phone, a notebook computer, a digital broadcast receiver, a PDA (Personal Digital Assistant), a PAD (Tablet Personal Computer), a PMP (Portable Multimedia Player), a car terminal (e.g., a car navigation terminal), and the like, as well as a stationary terminal such as a digital TV, a desktop computer, and the like. Figure 6 The electronic device shown is merely an example and should not impose any limitation on the functions and use range of an embodiment of the disclosure. Figure 6

[0189] As shown in FIG. 1, the electronic device 500 can include a processing device (e.g., a central processing unit, a graphic processing unit, etc.) 501 that can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 502 or a program loaded from a storage device 508 into a random access memory (RAM) 503. In the RAM 503, various programs and data required for the operation of the electronic device 500 are also stored. The processing device 501, the ROM 502, and the RAM 503 are connected to each other through a bus 504. An input / output (I / O) interface 505 is also connected to the bus 504. Figure 6

[0190] Generally, the following devices can be connected to the I / O interface 505: an input device 506 including, for example, a touch screen, a touch pad, a keyboard, a mouse, a camera, a microphone, an accelerometer, a gyroscope, and the like; an output device 507 including, for example, a liquid crystal display (LCD), a speaker, a vibrator, and the like; a storage device 508 including, for example, a magnetic tape, a hard disk, and the like; and a communication device 509. The communication device 509 can allow the electronic device 500 to communicate with other devices wirelessly or by wire to exchange data. Although Figure 6 The electronic device 500 having various devices is shown, but it is understood that all the shown devices are not required to be implemented or provided. More or less devices can be alternatively implemented or provided.

[0191] ​​In particular, according to embodiments of the present disclosure, the processes described above with reference to the flowcharts can be implemented as a computer software program. For example, embodiments of the present disclosure include a computer program product comprising a computer program carried on a non-transitory computer-readable medium, the computer program containing program code for executing the methods illustrated by the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network by the communication apparatus 509, or installed from the storage apparatus 508, or installed from the ROM 502. When the computer program is executed by the processing apparatus 501, the above-mentioned functions defined in the methods of the embodiments of the present disclosure are executed.

[0192] The names of the messages or information exchanged between the plurality of devices in the embodiments of the present disclosure are only for illustrative purposes, and are not intended to limit the scope of the messages or information.

[0193] The electronic device provided by the embodiments of the present disclosure and the image processing method provided by the above-mentioned embodiments belong to the same inventive concept, and the technical details not described in detail in the present embodiment can be referred to the above-mentioned embodiments, and the present embodiment has the same beneficial effects as the above-mentioned embodiments.

[0194] The embodiments of the present disclosure provide a computer storage medium, which stores a computer program, and the program is executed by a processor to implement the image processing method provided by the above-mentioned embodiments.

[0195] It should be noted that the computer-readable medium described above can be a computer-readable signal medium or a computer-readable storage medium or any combination thereof. The computer-readable storage medium, for example, can be, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus or device, or any suitable combination of the foregoing. More specific examples of the computer-readable storage medium can include, but are not limited to, an electrical connection having one or more wires, a portable computer diskette, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or Flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing. In the disclosure, the computer-readable storage medium can be any tangible medium that contains or stores a program used or used in conjunction with an instruction execution system, apparatus or device. In the disclosure, the computer-readable signal medium can include a data signal carried in a baseband or as a part of a carrier wave, which carries computer-readable program code. Such a propagated data signal can take various forms, including, but not limited to, an electromagnetic signal, an optical signal, or any suitable combination of the foregoing. The computer-readable signal medium can also be any computer-readable medium other than the computer-readable storage medium, which can send, propagate or transmit a program for use by or in conjunction with an instruction execution system, apparatus or device. The program code contained in the computer-readable medium can be transmitted by any suitable medium, including, but not limited to, a wire, a cable, a RF (radio frequency), or the like, or any suitable combination of the foregoing.

[0196] In some embodiments, the client, server, or both can communicate using any current known or future developed network protocol, such as HTTP (HyperText Transfer Protocol), and can be interconnected with any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include local area networks ("LAN"), wide area networks ("WAN"), the Internet, and peer-to-peer networks (e.g., ad hoc peer-to-peer networks), as well as any current known or future developed networks.

[0197] The computer-readable medium described above can be included in the electronic device described above; or can exist separately from the electronic device and not be assembled into the electronic device.

[0198] The computer-readable medium described above carries one or more programs, when the one or more programs are executed by the electronic device, the electronic device:

[0199] The computer readable medium carries one or more programs, when the one or more programs are executed by the electronic device, the electronic device is caused to: acquire a to-be-processed image including first hairstyle information, and determine a target mask image corresponding to a hair region of the to-be-processed image;

[0200] stitch the to-be-processed image and the target mask image to obtain a target processing image;

[0201] input the target processing image into a hairstyle changing model trained in advance to obtain a target display image including second hairstyle information corresponding to the to-be-processed image;

[0202] The hairstyle changing model is trained based on a sample original image including a hair region, a sample mask image corresponding to the hair region of the sample original image, an expected image corresponding to the sample original image, and an expected mask image corresponding to the hair region of the expected image.

[0203] Computer program code for carrying out operations of the present disclosure can be written in any one or more of a number of programming languages or combinations of languages including an object-oriented programming language such as Java, Smalltalk, C++ or the like and conventional procedural programming languages such as C or similar programming languages. The program code can execute entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer or entirely on the remote computer or server. In the latter scenario, the remote computer can be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or the connection can be made to an external computer (for example, through the Internet using an Internet Service Provider).

[0204] The flow and block diagrams in the drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods and computer program products according to various embodiments of the present disclosure. In this regard, each block in the flow and block diagrams can represent a module, segment, or portion of code, which comprises one or more executable instructions for implementing the specified logical functions. It should also be noted that in some alternative implementations, the functions noted in the blocks can occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently or the blocks can sometimes be executed in the reverse order, depending upon the functionality involved. It will also be noted that each block of the block diagrams and / or flowchart illustrations, and combinations thereof, can be implemented by a dedicated hardware-based system that performs the specified functions or operations, or can be implemented by a combination of dedicated hardware and computer instructions.

[0205] The units described in the embodiments of the present disclosure can be implemented by software, or by hardware. In some cases, the name of the unit does not constitute a limitation on the unit itself. For example, the first obtaining unit can also be described as a unit for obtaining at least two Internet protocol addresses.

[0206] The functions described above in the specification can be implemented in part or in whole by one or more hardware logic components. For example, and without limitation, illustrative types of hardware logic components that can be used include Field-programmable Gate Arrays (FPGAs), Program-specific Integrated Circuits (ASICs), Program-specific Standard Products (ASSPs), System-on-a-chip systems (SOCs), Complex Programmable Logic Devices (CPLDs), etc.

[0207] In the context of the present disclosure, a machine-readable medium can be a tangible medium that contains or stores a program for use by or in connection with an instruction execution system, apparatus, or device. The machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium can include but is not limited to an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any suitable combination of the foregoing. More specific examples of the machine-readable storage medium will include one or more of: a portable computer diskette, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or Flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0208] According to one or more embodiments of the present disclosure, Example One provides an image processing method, the method comprising:

[0209] obtaining a to-be-processed image including first hairstyle information, and determining a target mask image corresponding to a hair region of the to-be-processed image;

[0210] stitching the to-be-processed image and the target mask image to obtain a target processing image;

[0211] inputting the target processing image into a pre-trained hairstyle changing model to obtain a target display image including second hairstyle information corresponding to the to-be-processed image;

[0212] The hairstyle changing model is trained based on a sample original image including a hair region, a sample mask image corresponding to the hair region of the sample original image, an expected image corresponding to the sample original image, and an expected mask image corresponding to the hair region of the expected image.

[0213] According to one or more embodiments of the present disclosure, Example Two provides an image processing method, which includes:

[0214] constructing a first sample image according to a sample original image including a hair region and a sample mask image corresponding to the hair region of the sample original image;

[0215] constructing a second sample image according to the sample mask image, an expected image corresponding to the sample original image, and an expected mask image corresponding to the hair region of the expected image;

[0216] training a generative model of a pre-established generative adversarial network according to the first sample image and the second sample image to obtain the hairstyle changing model.

[0217] According to one or more embodiments of the present disclosure, Example Three provides an image processing method, which includes:

[0218] The constructing a second sample image according to the sample mask image, an expected image corresponding to the sample original image, and an expected mask image corresponding to the hair region of the expected image includes:

[0219] determining a differential region between the sample mask image and the expected mask image corresponding to the hair region of the sample original image;

[0220] processing the expected image according to the differential region to obtain a second sample image.

[0221] According to one or more embodiments of the present disclosure, Example Four provides an image processing method, which includes:

[0222] The processing of the expected image according to the differential region obtains a second sample image, and the processing of the expected image according to the differential region includes:

[0223] The part of the differential region in the expected image is randomly occluded to obtain a second sample image.

[0224] According to one or more embodiments of the present disclosure, Example Five provides an image processing method, which includes:

[0225] The training of the pre-established generative adversarial network according to the first sample image and the second sample image includes:

[0226] The first sample image and the second sample image are respectively input into a generative model of the pre-established generative adversarial network for training to obtain a first output image and a second output image.

[0227] A first model loss of the generative model is calculated according to the first sample image, the first output image and the expected image, and a model parameter of the generative model is adjusted according to the first model loss.

[0228] A second model loss of the generative model is calculated according to the second output image and the expected image, and a model parameter of the generative model is adjusted according to the second model loss.

[0229] According to one or more embodiments of the present disclosure, Example Six provides an image processing method, which includes:

[0230] The first model loss includes a first generative loss and an adversarial loss; the first generative loss includes a first regression loss and / or a first perception loss, the first regression loss includes a first regression sub-loss and a second regression sub-loss; and the second model loss includes a second regression loss and / or a second perception loss.

[0231] According to one or more embodiments of the present disclosure, Example Seven provides an image processing method, which includes:

[0232] The calculation of the first model loss of the generative model according to the first sample image, the first output image and the expected image includes at least one of the following:

[0233] The first perception loss between the first output image and the expected image is calculated based on a preset perception loss function;

[0234] The first regression loss between the first output image and the expected image is calculated based on a Gram matrix and / or a mean absolute error function.

[0235] According to one or more embodiments of the present disclosure, Example Eight provides an image processing method, which comprises:

[0236] The first model loss of the generative model is calculated according to the first sample image, the first output image and the expected image, comprising:

[0237] The first sample image and the first output image are spliced to obtain a fake sample image;

[0238] The first sample image and the expected image are spliced to obtain a real sample image;

[0239] The fake sample image and the real sample image are discriminated based on a pre-trained discriminant model to obtain a model discrimination result, and an adversarial loss of the generative adversarial network is calculated according to the model discrimination result.

[0240] According to one or more embodiments of the present disclosure, Example Nine provides an image processing method, which comprises:

[0241] The adversarial loss of the generative adversarial network is calculated according to the model discrimination result, comprising:

[0242] The adversarial discrimination loss of the discriminant model in the generative adversarial network is determined according to the model discrimination result of the fake sample image and the real sample image by the discriminant model and the expected mask image corresponding to the hair region of the expected image;

[0243] The adversarial generation loss of the discriminant model in the generative adversarial network is determined according to the model discrimination result of the fake sample image by the discriminant model and the expected mask image corresponding to the hair region of the expected image;

[0244] The adversarial loss of the generative adversarial network is calculated according to the adversarial discrimination loss and the adversarial generation loss.

[0245] According to one or more embodiments of the present disclosure, Example Ten provides an image processing method, which comprises:

[0246] The hairstyle changing model comprises an input layer, a hidden layer and an output layer, and the output layer is constructed based on a attention mechanism carrying guided filtering; the target processing image is input into the pre-trained hairstyle changing model to obtain a target display image corresponding to the to-be-processed image and containing second hairstyle information, comprising:

[0247] The target processing image is input through the hairstyle changing model input layer, and a hairstyle changing process is performed via a hidden layer to obtain an intermediate processing image, wherein the intermediate processing image at least includes a red channel, a green channel, a blue channel and a transparent channel;

[0248] The intermediate processing image is processed based on an attention mechanism carrying a guided filter of an output layer of the hairstyle changing model to obtain a target display image corresponding to the to-be-processed image and containing second hairstyle information.

[0249] According to one or more embodiments of the present disclosure, example eleven provides an image processing method, which comprises:

[0250] A color input image composed of a red channel, a green channel and a blue channel in the target processing image is acquired, and a target grayscale image is obtained by performing grayscale processing on the color input image;

[0251] A target grayscale image and a transparent image composed of the transparent channel are filtered based on a preset guided filter algorithm to obtain a first transparent image;

[0252] A color processing image composed of a red channel, a green channel and a blue channel in the intermediate processing image is multiplied by the first transparent image to obtain a first fusion image;

[0253] The color input image is multiplied by a second transparent image to obtain a second fusion image, wherein a transparency value in the second transparent image is obtained by subtracting a transparency value of the first transparent image from a preset transparency value;

[0254] The first fusion image and the second fusion image are added to obtain a target display image corresponding to the to-be-processed image and containing second hairstyle information.

[0255] According to one or more embodiments of the present disclosure, example twelve provides an image processing device, which comprises:

[0256] An image acquisition module is configured to acquire a to-be-processed image including first hairstyle information and determine a target mask image corresponding to a hair region of the to-be-processed image;

[0257] An image splicing module is configured to splice the to-be-processed image and the target mask image to obtain a target processing image;

[0258] A hairstyle changing module is configured to input the target processing image into a pre-trained hairstyle changing model to obtain a target display image corresponding to the to-be-processed image and containing second hairstyle information;

[0259] The hairstyle changing model is trained based on a sample original image including a hair region, a sample mask image corresponding to the hair region of the sample original image, an expected image corresponding to the sample original image, and an expected mask image corresponding to the hair region of the expected image.

[0260] The above description is merely that of the preferred embodiments of the present disclosure and the principles of the technology employed, and it is understood that the scope of the disclosure involved is not limited to the technical solutions formed by the specific combinations of the technical features described above, and it should also cover other technical solutions formed by the combinations of the technical features described above or their equivalent features without departing from the above disclosed concept. For example, the technical solutions formed by the mutual replacement of the above features and the technical features with similar functions disclosed in the present disclosure (but not limited to) form the technical solutions.

[0261] Further, while operations are depicted in a particular, chronological sequence, this should not be understood as requiring such order unless otherwise specified. Multi-tasking and parallel processing can be advantageous. Likewise, while specific implementation details are contained in the above discussion, these should not be construed as limiting the scope of the disclosure. Some features described in the context of separate embodiments can also be implemented together in a single embodiment. Conversely, various features described in the context of a single embodiment can also be implemented in multiple embodiments separately or in any suitable sub-combination.

[0262] Although the subject matter has been described in language specific to structural features and / or methodological acts, it is to be understood that the subject defined in the appended claims is not necessarily limited to the specific features or acts described above. Rather, the specific features and acts described above are disclosed as example forms of implementing the claims.

Claims

1. An image processing method, characterized by, The method comprises the following steps: constructing a first sample image according to a sample original image comprising a hair region and a sample mask image corresponding to the hair region of the sample original image; constructing a second sample image according to the sample mask image, an expected image corresponding to the sample original image, and an expected mask image corresponding to the hair region of the expected image; training a generative model of a pre-established generative adversarial network according to the first sample image and the second sample image, to obtain a hairstyle changing model; obtaining a to-be-processed image comprising first hairstyle information, and determining a target mask image corresponding to the hair region of the to-be-processed image; splicing the to-be-processed image and the target mask image to obtain a target processing image; inputting the target processing image into the pre-trained hairstyle changing model to obtain a target display image comprising second hairstyle information corresponding to the to-be-processed image; wherein the hairstyle changing model is trained based on a sample original image comprising a hair region, a sample mask image corresponding to the hair region of the sample original image, an expected image corresponding to the sample original image, and an expected mask image corresponding to the hair region of the expected image.

2. The image processing method of claim 1, wherein, The method comprises the following steps: determining a difference region between the sample mask image and the expected mask image corresponding to the hair region of the expected image corresponding to the sample original image; processing the expected image according to the difference region to obtain a second sample image.

3. The image processing method of claim 2, wherein, The method comprises the following steps: randomly occluding part of the difference region in the expected image to obtain a second sample image.

4. The image processing method of claim 1, wherein, The method comprises the following steps: inputting the first sample image and the second sample image into the generative model of the pre-established generative adversarial network for training to obtain a first output image and a second output image; calculating a first model loss of the generative model according to the first sample image, the first output image, and the expected image, and adjusting model parameters of the generative model according to the first model loss; calculating a second model loss of the generative model according to the second output image and the expected image, and adjusting model parameters of the generative model according to the second model loss.

5. The image processing method of claim 4, wherein, The first model loss comprises a first generation loss and an adversarial loss; the first generation loss comprises a first regression loss and / or a first perception loss, the first regression loss comprises a first regression sub-loss and a second regression sub-loss; the second model loss comprises a second regression loss and / or a second perception loss.

6. The image processing method of claim 5, wherein, The method comprises at least one of the following steps: calculating a first perceptual loss between the first output image and the expected image based on a preset perceptual loss function; calculating a first regression loss between the first output image and the expected image based on a Gram matrix and / or a mean absolute error function.

7. The image processing method of claim 5, wherein, The calculating of the first model loss of the generative model according to the first sample image, the first output image and the expected image comprises: splicing the first sample image and the first output image to obtain a fake sample image; splicing the first sample image and the expected image to obtain a real sample image; performing discrimination on the fake sample image and the real sample image based on a pre-trained discriminator model to obtain a model discrimination result, and calculating an adversarial loss of the generative adversarial network according to the model discrimination result.

8. The image processing method of claim 7, wherein, The calculating of the adversarial loss of the generative adversarial network according to the model discrimination result comprises: determining an adversarial discrimination loss of the discriminator model in the generative adversarial network according to the model discrimination result of the discriminator model on the fake sample image and the real sample image and an expected mask image corresponding to a hair region of the expected image; determining an adversarial generation loss of the discriminator model in the generative adversarial network according to the model discrimination result of the discriminator model on the fake sample image and the expected mask image corresponding to the hair region of the expected image; calculating the adversarial loss of the generative adversarial network according to the adversarial discrimination loss and the adversarial generation loss.

9. The image processing method of claim 1, wherein, The hairstyle changing model comprises an input layer, a hidden layer and an output layer, and the output layer is constructed based on an attention mechanism carrying a guided filter; the inputting of the target processing image into the pre-trained hairstyle changing model to obtain a target display image corresponding to the to-be-processed image and containing second hairstyle information comprises: inputting the target processing image through the input layer of the hairstyle changing model, performing hairstyle transformation processing via the hidden layer to obtain an intermediate processing image, wherein the intermediate processing image at least comprises a red channel, a green channel, a blue channel and a transparent channel; performing processing on the intermediate processing image based on the attention mechanism carrying a guided filter of the output layer of the hairstyle changing model to obtain the target display image corresponding to the to-be-processed image and containing the second hairstyle information.

10. The image processing method of claim 9, wherein, The performing of processing on the intermediate processing image based on the attention mechanism carrying a guided filter of the output layer of the hairstyle changing model to obtain the target display image corresponding to the to-be-processed image and containing the second hairstyle information comprises: obtaining a color input image composed of a red channel, a green channel and a blue channel in the target processing image, performing grayscale processing on the color input image to obtain a target grayscale image; performing filtering processing on the target grayscale image and a transparent image composed of the transparent channel based on a preset guided filter algorithm to obtain a first transparent image; multiplying a color processing image composed of a red channel, a green channel and a blue channel in the intermediate processing image and the first transparent image to obtain a first fusion image; multiplying the color input image and a second transparent image to obtain a second fusion image, wherein a transparency value in the second transparent image is obtained by subtracting the transparency value of the first transparent image from a preset transparency value; adding the first fusion image and the second fusion image to obtain a target display image corresponding to the to-be-processed image and containing second hairstyle information.

11. An image processing apparatus characterized by comprising: The method comprises the following steps: a first sample image obtaining module is configured to construct a first sample image according to a sample original image containing a hair region and a sample mask image corresponding to the hair region of the sample original image; a second sample image obtaining module is configured to construct a second sample image according to the sample mask image, an expected image corresponding to the sample original image and an expected mask image corresponding to a hair region of the expected image; a hairstyle changing model training module is configured to train a generative model of a pre-established generative adversarial network according to the first sample image and the second sample image, to obtain a hairstyle changing model; an image obtaining module is configured to obtain a to-be-processed image containing first hairstyle information and determine a target mask image corresponding to a hair region of the to-be-processed image; an image splicing module is configured to splice the to-be-processed image and the target mask image to obtain a target processing image; a hairstyle changing module is configured to input the target processing image into the pre-trained hairstyle changing model to obtain a target display image corresponding to the to-be-processed image and containing second hairstyle information. The hairstyle changing model is trained based on a sample original image containing a hair region, a sample mask image corresponding to the hair region of the sample original image, an expected image corresponding to the sample original image and an expected mask image corresponding to a hair region of the expected image.

12. An electronic device, comprising: The electronic device comprises: one or more processors; a storage device configured to store one or more programs, when the one or more programs are executed by the one or more processors, the one or more processors implement the image processing method of any one of claims 1-10.

13. A storage medium containing computer-executable instructions, wherein: The computer executable instructions, when executed by a computer processor, are used to perform the image processing method of any one of claims 1-10. The computer executable instructions, when executed by a computer processor, are used to perform the image processing method of any one of claims 1-10.

Images