Hair transformation model generation method and device, hair transformation method and device
By training the hair transformation model, using multiple pairs of sample character images and color guide maps, the problem of poor color consistency after hair transformation in the prior art is solved, and a more natural hair color transformation effect is achieved.
Patent Information
- Application Number
- CN202010802416.0
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2020-08-11
- Publication Date
- 2024-11-22
- Estimated Expiration
- 2040-08-11
AI Technical Summary
In the hair transformation function of the prior art, the hair color consistency before and after the transformation is poor and not natural enough.
By obtaining multiple pairs of sample characters, the hairstyles of the characters in each pair of samples are inconsistent, and corresponding color guide maps are generated. The first sample character image and color guide map are used as training data, and the second sample character image is used as supervision information. Get a hair transformation model to ensure that the color of the changed hair is consistent with the color before the change.
The color consistency between the hair after the transformation and before the transformation is achieved, and the color of the hair after the transformation is more natural, avoiding the unnatural color caused by outliers in the prior art.
Smart Images

Figure CN114078083B_ABST
Abstract
Description
Technical Field
[0001] The disclosed embodiments relate to the field of image processing, and in particular, to a hair transformation model generation method and device, and a hair transformation method and device. Background Art
[0002] At present, in order to increase users' interest in video applications, most video applications have a hair transformation function, that is, the hair of the user in the photo or video provided by the user is changed into another hairstyle. The implementation of this function in the prior art specifically includes the following two steps: (1) using a preset hairstyle transformation model to transform the original hairstyle of the person in the photo or video provided by the user into a preset hairstyle (e.g., long wavy hair). (2) using a preset color transformation method according to the color of the original hairstyle, transform the color of the preset hairstyle converted in step (1) into a color close to the hair color of the person in the photo or video provided by the user. Among them, two methods are currently used in step (2): table lookup and mean variance color transformation. Among them, the need for table lookup (Lookup Table) is based on the experience of the designer, which is difficult to apply to complex application scenarios where the current users have different hair colors, and the color of the hair after transformation is not consistent with the color before transformation. In the implementation process, mean variance color transformation is easily affected by abnormal values in the RGB values corresponding to the colors in the original hairstyle (e.g., the influence of external highlights on the color), resulting in the color of the preset hairstyle after transformation being unnatural.
[0003] In summary, when the prior art realizes the hair transformation function, the hair color consistency before and after the transformation is poor and not natural enough. Summary of the invention
[0004] The present invention relates to a hair transformation model generation method and device, and a hair transformation method and device, which can realize a hair transformation function in which the hair of a character portrait has a more consistent and natural color after transformation than before transformation.
[0005] To achieve the above objectives, the embodiments of the present disclosure adopt the following technical solutions:
[0006] In a first aspect, a method for generating a hair transformation model is provided, comprising: obtaining a first sample character image and a second sample character image corresponding to the first sample character image; the second sample character image is obtained based on the first sample character image, and the hairstyle of the character in the second sample character image is inconsistent with that in the first sample character image; determining a first color guidance map based on the first sample character image and the second sample character image; the first color guidance map includes a hair generation area that is the same as the hair segmentation area in the second sample character image; the color of the hair generation area in the first color guidance map is obtained based on the color of the hair in the first sample character image; using the first sample character image and the first color guidance map as training data and the second sample character image as supervisory information, training to obtain a hair transformation model.
[0007] Based on the above technical solution, when training the hair transformation model, firstly, a plurality of pairs of samples are obtained, in which the hairstyles of the first sample person image and the second sample person image in each pair of samples are inconsistent, and the second sample person image is obtained from the first sample person image; then, according to these sample pairs, a corresponding first color guidance map is generated, and the hair generation area in the first color guidance map is the same as the hair segmentation area in the second sample person image, and its color is obtained from the color of the hair in the first sample person image. Finally, the hair transformation model can be trained by using the first sample person image and the first color guidance map as training data and the second sample person image as supervisory information. In this way, the obtained hair transformation model can output the image of the person after the hair transformation after receiving a person image and the information indicating the corresponding hairstyle. Because the samples used for the whole model training are large in number, and the first color guidance map whose color is determined by the first sample person image is used, and the second sample person image with the same hairstyle as that in the first color guidance map is used as supervisory information, the model finally obtained can make the hair after the transformation and the hair before the transformation have better color consistency when completing the hair transformation function.
[0008] Optionally, determining the first color guide map according to the first sample character image and the second sample character image includes:
[0009] Determine a target hair mask map of the second sample character image; and determine a first color guide map according to the color value of the hair in the first sample character image, the target hair mask map, and the color value of the hair corresponding to the target hair mask map.
[0010] Further optionally, determining the first color guide map according to the color value of the hair in the first sample person image, the target hair mask map, and the color value of the hair corresponding to the target hair mask map includes:
[0011] Calculate the statistical value of the color value of the hair in the first sample person image; replace the color values of all pixels of the hair in the target hair mask image with the statistical value of the color value of the hair in the first sample person image to obtain a first color guide map.
[0012] Optionally, in a case where the hair area in the second sample character image is larger than the hair area in the first sample character image, the color of the hair generation area in the first color guidance image is obtained according to the hair color in the first sample character image, including: the color of the hair generation area in the first color guidance image is obtained according to the hair color in the first sample character image and the hair color in the second sample character image;
[0013] Determining the first color guide map according to the color value of the hair in the first sample person image, the target hair mask map, and the color value of the hair corresponding to the target hair mask map includes:
[0014] Calculate the statistical value of the color value of the hair in the first sample person image and the statistical value of the color value of the hair corresponding to the second area in the target hair mask image except the first area corresponding to the area of the hair in the first sample person image;
[0015] The color values of all pixels in the first area are replaced with the statistical values of the color values of the hair in the first sample character image, and the color values of all pixels in the second area are replaced with the statistical values of the color values of the hair corresponding to the second area to obtain a first color guide map.
[0016] Optionally, when the hair area in the first sample character image is larger than the hair area in the second sample character image, determining the first color guide map according to the hair color value in the first sample character image, the target hair mask map, and the hair color value corresponding to the target hair mask map includes:
[0017] Calculate the statistical value of the color value of the hair in the third area of the first sample character image corresponding to the area of the hair in the target hair mask image; replace the color values of all pixels in the area of the hair in the target hair mask image with the statistical value of the hair color value in the third area to obtain a first color guide map.
[0018] Optionally, the first sample person image and the color guide map are used as training data, and the second sample person image is used as supervision information. After the hair transformation model is obtained through training, the following steps are further included:
[0019] Acquire multiple third sample character images; acquire at least one preset color guide map; each preset color guide map corresponds to a hair generation area; calculate the statistical value of the color value of the hair in the third sample character image; replace the color values of all pixels of the hair in the preset color guide map with the statistical value of the hair color value in the third sample character image to obtain a second color guide map; input the third sample character image and the second color guide map into a hair transformation model to obtain a fourth sample character image; use the third sample character image and hairstyle variables as training data, and the fourth sample character image corresponding to the hairstyle variables as supervision information to train and obtain a target hair transformation model; the hairstyle variables correspond to the hair generation area one-to-one; and update the hair transformation model to the target hair transformation model.
[0020] Optionally, calculating the statistical value of the hair color value includes: arranging the RGB values of all pixels of the hair color in ascending order according to color and value, calculating the average value of the RGB values corresponding to the colors and values whose order is within a preset order range, and taking the average value of the RGB values corresponding to the colors and values whose order is within the preset order range as the statistical value of the hair color value; the color sum is the sum of the R value, the G value and the B value in the RGB value.
[0021] In a second aspect, a hair transformation method is provided, comprising:
[0022] A source person image and hairstyle indication information are obtained; the hairstyle indication information is used to indicate a target hair generation area; the source person image and the hairstyle indication information are input into a hair transformation model to obtain a target person image; the hair transformation model is obtained by the hair transformation model generation method provided in the first aspect; the hairstyle of the person in the target person image is the hairstyle corresponding to the target hair generation area indicated by the hairstyle indication information, and the similarity between the hair color of the person image in the target person image and the hair color of the person image in the source person image is greater than a preset value.
[0023] Based on the above technical solution, a hair transformation device that needs to realize the hair transformation function can use only the hair transformation model generated by the technical solution provided by the first aspect to use the source character image to obtain a target character image whose hairstyle and hair color meet the requirements, thereby avoiding the problem of poor color consistency after hair transformation as in the prior art.
[0024] Optionally, the hairstyle indication information includes a preset color guide map, and the preset color guide map includes a target hair generation area; inputting the source person image and the hairstyle indication information into the hair transformation model to obtain the target person image includes:
[0025] Calculate the statistical value of the color value of the hair in the source character image; replace the color values of all pixels in the target hair generation area in the preset color guidance map with the statistical value of the color value of the hair in the source character image to obtain the target color guidance map; input the source character image and the target color guidance map into the hair transformation model to obtain the target character image.
[0026] Optionally, when the hair transformation model is updated to a target hair transformation model obtained by the hair transformation model generation method provided by the first aspect, the hairstyle indication information includes a target hairstyle variable corresponding to a target hair generation area; inputting the source person image and the hairstyle indication information into the hair transformation model to obtain the target person image includes:
[0027] The source person image and the target hairstyle variable are input into the updated hair transformation model to obtain the target person image; the hairstyle of the person in the target person image corresponds to the target hair generation area.
[0028] In a third aspect, a hair transformation model generation device is provided, including: an acquisition module, a processing module, and a training module. The acquisition module is configured to acquire a first sample character image and a second sample character image corresponding to the first sample character image; the second sample character image is obtained based on the first sample character image, and the character's hairstyle in the second sample character image is inconsistent with the first sample character image; the processing module is configured to determine a first color guidance map based on the first sample character image and the second sample character image acquired by the acquisition module; the first color guidance map includes a hair generation area that is the same as the hair segmentation area in the second sample character image; the color of the hair generation area in the first color guidance map is obtained based on the color of the hair in the first sample character image; the training module is configured to use the first sample character image acquired by the acquisition module and the first color guidance map determined by the processing module as training data, and the second sample character image acquired by the acquisition module as supervisory information, to train and obtain a hair transformation model.
[0029] Optionally, the processing module specifically includes a first determining unit and a second determining unit;
[0030] The first determination unit is configured to determine the target hair mask map of the second sample character image acquired by the acquisition module; the second determination unit is configured to determine the first color guide map based on the color value of the hair in the first sample character image acquired by the acquisition module, the target hair mask map determined by the first determination unit, and the color value of the hair corresponding to the target hair mask map.
[0031] Further optionally, the second determination unit is specifically configured to: calculate the statistical value of the color value of the hair in the first sample character image obtained by the acquisition module; replace the color values of all pixels of the hair in the target hair mask image determined by the first determination unit with the statistical value of the color value of the hair in the first sample character image to obtain a first color guide map.
[0032] Further optionally, in a case where the hair area in the second sample character image is larger than the hair area in the first sample character image, the color of the hair generation area in the first color guidance map is obtained according to the hair color in the first sample character image, including: the color of the hair generation area in the first color guidance map is obtained according to the hair color in the first sample character image and the hair color in the second sample character image;
[0033] The second determination unit is specifically configured to: calculate the statistical value of the color value of the hair in the first sample character image obtained by the acquisition module, and the statistical value of the color value of the hair corresponding to the second area in the target hair mask image determined by the first determination unit except the first area corresponding to the area of the hair in the first sample character image; replace the color values of all pixels in the first area with the statistical value of the color value of the hair in the first sample character image, and replace the color values of all pixels in the second area with the statistical value of the color value of the hair corresponding to the second area, so as to obtain a first color guide map.
[0034] Further optionally, when the hair area in the first sample character image is larger than the hair area in the second sample character image, the second determination unit is specifically configured to: calculate the statistical value of the hair color value in the third area of the first sample character image acquired by the acquisition module corresponding to the hair area in the target hair mask image determined by the first determination unit; replace the color values of all pixels in the hair area in the target hair mask image with the statistical value of the hair color value in the third area to obtain a first color guide map.
[0035] Further optionally, after the training module obtains the hair transformation model through training, the acquisition module is further configured to obtain multiple third sample character images; the acquisition module is further configured to obtain at least one preset color guidance map; each preset color guidance map corresponds to a hair generation area; the second determination unit is further configured to calculate the statistical value of the hair color value in the third sample character image obtained by the acquisition module; the second determination unit is further configured to replace the color values of all pixels of the hair in the preset color guidance map obtained by the acquisition module with the statistical value of the hair color value in the third sample character image to obtain a second color guidance map; the acquisition module is further configured to input the third sample character image into the hair transformation model obtained by the training module through the second color guidance map obtained by the second determination unit to obtain a fourth sample character image; the training module is further configured to use the third sample character image and hairstyle variable obtained by the acquisition module as training data, and the fourth sample character image corresponding to the hairstyle variable as supervision information, to train and obtain a target hair transformation model; the hairstyle variable corresponds one-to-one to the hair generation area; the training module is further configured to update the hair transformation model to the target hair transformation model.
[0036] Optionally, the second determination unit is specifically configured to: arrange the RGB values of all pixels of the hair color in ascending order according to the color and value, calculate the average value of the RGB values corresponding to the colors and values whose order is within a preset order range, and take the average value of the RGB values corresponding to the colors and values whose order is within the preset order range as the statistical value of the hair color value; the color sum value is the sum of the R value, G value and B value in the RGB value.
[0037] In a fourth aspect, a hair transformation device is provided, comprising: an acquisition module and a processing module. The acquisition module is configured to acquire a source person image and hairstyle indication information; the hairstyle indication information is used to indicate a target hair generation area;
[0038] The processing module is configured to input the source character image and hairstyle indication information obtained by the acquisition module into the hair transformation model to obtain a target character image; the hair transformation model is obtained by the hair transformation model generation method provided in the first aspect; the hairstyle of the character in the target character image is the hairstyle corresponding to the target hair generation area indicated by the hairstyle indication information, and the similarity between the hair color of the character image in the target character image and the hair color of the character image in the source character image is greater than a preset value.
[0039] Optionally, the hairstyle indication information includes a preset color guide map, and the preset color guide map includes a target hair generation area; the processing module is specifically configured to: calculate the statistical value of the color value of the hair in the source character image obtained by the acquisition module; replace the color values of all pixels in the target hair generation area in the preset color guidance map obtained by the acquisition module with the statistical value of the color value of the hair in the source character image to obtain the target color guidance map; input the source character image and the target color guidance map obtained by the acquisition module into the hair transformation model to obtain the target character image.
[0040] Optionally, when the hair transformation model is updated to the target hair transformation model obtained by the hair transformation model generation method provided by the first aspect, the hairstyle indication information includes a target hairstyle variable corresponding to the target hair generation area; the processing module is specifically configured to: input the source character image and the target hairstyle variable acquired by the acquisition module into the updated hair transformation model to obtain the target character image; the hairstyle of the character in the target character image corresponds to the target hair generation area.
[0041] In a fifth aspect, an electronic device is provided, comprising: a processor; and a memory for storing instructions executable by the processor; wherein the processor is configured to execute the instructions to implement the hair transformation model generation method provided in the first aspect.
[0042] According to a sixth aspect, an electronic device is provided, comprising: a processor; and a memory for storing instructions executable by the processor; wherein the processor is configured to execute the instructions to implement the hair transformation method provided in the second aspect.
[0043] In a seventh aspect, a computer storage medium is provided. When instructions in the computer storage medium are executed by a processor of an electronic device, the electronic device is enabled to execute the hair transformation model generation method provided in the first aspect.
[0044] In an eighth aspect, a computer storage medium is provided. When instructions in the computer storage medium are executed by a processor of an electronic device, a server is enabled to execute the hair transformation method provided in the second aspect.
[0045] According to a ninth aspect, a computer program product is provided, comprising instructions, which, when executed on a computer, enable the computer to execute the hair transformation model generation method provided in the first aspect.
[0046] According to a tenth aspect, a computer program product is provided, comprising instructions, which, when executed on a computer, causes the computer to execute the hair transformation method provided in the second aspect.
[0047] It can be understood that the solutions of the third aspect to the tenth aspect provided above are all used to execute the corresponding methods provided in the first aspect and the second aspect above. Therefore, the beneficial effects that can be achieved can refer to the beneficial effects in the corresponding methods provided above and will not be repeated here.
[0048] It should be understood that in the present application, the names of the above-mentioned hair transformation model generation device, hair transformation device and electronic device do not limit the devices or functional modules themselves. In actual implementation, these devices or functional modules may appear with other names. As long as the functions of each device or functional module are similar to the present invention, they belong to the scope of the claims of the present disclosure and their equivalent technologies. In addition, it should be understood that the above general description and the detailed description below are only exemplary and explanatory, and cannot limit the present disclosure. BRIEF DESCRIPTION OF THE DRAWINGS
[0049] Figure 1 A schematic diagram of a hair transformation model generation method and an implementation environment of the hair transformation method provided by an embodiment of the present disclosure;
[0050] Figure 2 A schematic diagram of a lut diagram;
[0051] Figure 3 A schematic diagram of a hair transformation model generation method provided in an embodiment of the present disclosure Figure 1 ;
[0052] Figure 4 A schematic diagram of a pair of sample person images and a corresponding first color guide map provided by an embodiment of the present disclosure;
[0053] Figure 5 A schematic diagram of a hair transformation model generation method provided in an embodiment of the present disclosure Figure 2 ;
[0054] Figure 6 A schematic diagram of a pair of target hair mask images provided for an embodiment of the present disclosure;
[0055] Figure 7 A schematic diagram of a hair transformation model generation method provided in an embodiment of the present disclosure Figure 3 ;
[0056] Figure 8 A schematic diagram of a first color guide map provided in an embodiment of the present disclosure;
[0057] Fig. 9 A schematic diagram of a hair transformation model generation method provided in an embodiment of the present disclosure Figure 4 ;
[0058] Fig.10A schematic diagram of another first color guide map provided in an embodiment of the present disclosure;
[0059] Fig.11 A schematic diagram of a hair transformation model generation method provided in an embodiment of the present disclosure Figure 5 ;
[0060] Fig.12 A schematic diagram of another first color guide map provided in an embodiment of the present disclosure;
[0061] Fig.13 A schematic diagram of a hair transformation model generation method provided in an embodiment of the present disclosure Figure 6 ;
[0062] Fig.14 A schematic diagram of a preset color guide map provided in an embodiment of the present disclosure;
[0063] Fig.15 A schematic diagram of a hair transformation scene provided by an embodiment of the present disclosure;
[0064] Fig.16 A schematic diagram of a flow chart of a hair transformation method provided by an embodiment of the present disclosure;
[0065] Fig.17 A schematic flow chart of another hair transformation method provided by an embodiment of the present disclosure;
[0066] Fig.18 A schematic flow chart of another hair transformation method provided in an embodiment of the present disclosure;
[0067] Fig.19 A structural schematic diagram of a hair transformation device provided by an embodiment of the present disclosure;
[0068] Fig. 20 A schematic diagram of the structure of a hair transformation model generating device provided by an embodiment of the present disclosure;
[0069] Fig.21 A schematic diagram of the structure of an electronic device provided in an embodiment of the present disclosure. DETAILED DESCRIPTION
[0070] In order to enable ordinary persons in the art to better understand the technical solutions of the present disclosure, the technical solutions in the embodiments of the present disclosure will be clearly and completely described below in conjunction with the accompanying drawings.
[0071] It should be noted that the terms "first", "second", etc. in the specification and claims of the present disclosure and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged where appropriate, so that the embodiments of the present disclosure described herein can be implemented in an order other than those illustrated and described herein. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the present disclosure. Instead, they are merely examples of devices and methods consistent with some aspects of the present disclosure as detailed in the appended claims.
[0072] In addition, in the description of the embodiments of the present disclosure, unless otherwise specified, " / " means or, for example, A / B can mean A or B; "and / or" in this article is only a description of the association relationship of associated objects, indicating that there can be three relationships, for example, A and / or B can mean: A exists alone, A and B exist at the same time, and B exists alone. In addition, in the description of the embodiments of the present disclosure, "multiple" means two or more than two.
[0073] First, the technical terms involved in this disclosure are introduced:
[0074] RGB value: RGB color mode (also translated as "red, green, and blue") is a color standard in the industry. It obtains various colors by changing the three color channels of red, green, and blue and superimposing them on each other. RGB represents the colors of the three channels of red, green, and blue. This standard covers almost all colors that human vision can perceive and is one of the most widely used color systems. RGB values refer to the three channels of a pixel and are represented by integers. Usually, there are 256 RGB values, ranging from 0, 1, 2... to 255. Among them, although the highest number is 255, 0 is also one of the values, so there are 256 in total. For example, the RGB value of a pixel can be (122, 255, 0).
[0075] The following is an introduction to the application scenarios of the technical solutions provided by this disclosure:
[0076] Figure 1 FIG. 1 is a schematic diagram of a hair transformation model generation method and an implementation environment of the hair transformation method according to an exemplary embodiment. The hair transformation model generation method and the hair transformation method provided in the embodiment of the present disclosure can be applied to the environment of the embodiment. Figure 1 As shown, the implementation environment may include a terminal 01 and a server 02, and the terminal 01 communicates with the server 02 via a wired communication method or a wireless communication method.
[0077] Exemplarily, the terminal 01 in the embodiment of the present disclosure may be a device with image processing function, such as a mobile phone, tablet computer, desktop, laptop, handheld computer, notebook computer, ultra-mobile personal computer (UMPC), netbook, as well as cellular phone, personal digital assistant (PDA), augmented reality (AR) / virtual reality (VR) device, and the embodiment of the present application does not impose any special restrictions on the specific form of the terminal. It can interact with the user through one or more methods such as keyboard, touchpad, touch screen, remote control, voice interaction or handwriting device.
[0078] Exemplarily, the server 02 in the present disclosure may be a single server, or a server cluster consisting of multiple servers, or a cloud computing service center, and the present disclosure does not limit this.
[0079] The server 02 may be mainly used for processing the character image. After processing the character image, the server 02 may send the processed character image to the terminal 01 via a wired or wireless method.
[0080] At present, in order to realize the hair transformation function (converting the hairstyle in a character image into another hairstyle and making the hair color in the transformed character image close to the hair color in the character image before the transformation), the existing technology includes converting the hairstyle in the character image according to the hairstyle transformation model and adjusting the hair color in the converted character image to be close to the hair color in the source character image. The process of adjusting the hair color can be called the image post-processing process, which generally uses two methods: lookup table and mean variance color transformation. Among them, the need for a lookup table is based on the experience of the designer to design a Figure 2The LUT (display lookup table) diagram shown in the figure is then used to convert the RGB value of each pixel in the converted character image into a coordinate according to a preset formula, and then the corresponding RGB value is found from the LUT diagram, and finally the RGB value found is filled into the pixel with the highest color. Because the entire solution needs to be based on character experience, it is difficult to adapt to the complex application scenarios of today's users with different hair colors, so the consistency between the color of the transformed hair and the color of the hair before the transformation is poor. The mean-variance color transformation requires calculating the mean and variance of the RGB values corresponding to the color of the original hair, as well as the mean and variance of the RGB values corresponding to the color of the preset hair, and then further performing color transformation based on the mean and variance of the two. The calculation amount is large, and it is easily affected by the abnormal values in the RGB values corresponding to the color of the original hair (such as the influence of external highlights on the color), making the color of the final transformed preset hair unnatural.
[0081] In view of the above problems, the disclosed embodiments provide a method for generating a hair transformation model, which can realize a hair transformation function in which the hair of a character avatar has a more consistent and natural color after the hair transformation than before the transformation. The specific execution subject is a hair transformation model generation device, which can be an electronic device, which can be a terminal in the above implementation environment, or a server in the above implementation environment, depending on the actual situation.
[0082] Reference Figure 3 As shown, the embodiment of the present disclosure provides a hair transformation method applied to a hair transformation model generation device, and the method includes 301-303.
[0083] 301. Obtain a first sample person image and a second sample person image corresponding to the first sample person image.
[0084] Among them, the second sample character image is obtained based on the first sample character image, and specifically can be obtained by using a preset deep learning algorithm. The hairstyle of the character in the second sample character image is inconsistent with that in the first sample character image, and the similarity between the hair color of the character in the second sample character image and the hair color of the character in the first sample character image is less than or equal to a preset value.
[0085] Exemplarily, the first sample person image can be directly obtained from the network, and the second sample person image can be obtained by processing the first sample person image using the CycleGAN neural network. Figure 4 As shown, Figure 4 The first sample character image is shown in a. Figure 4 In the figure, b is the second sample person image after being processed by the CycleGAN neural network. The difference between the two is that the hairstyle is different and the hair color is also different.
[0086] It should be noted that, because the actual CycleGAN neural network is large in size, requires more computing resources, and cannot be controlled in detail, if the CycleGAN neural network is used to transform the hairstyle in a certain character image, the hairstyle of the character in the transformed character image will be random and the hair color will be quite different from the hair color of the character in the character image before the transformation, so the CycleGAN neural network is used in the present disclosure to generate sample data for the hair transformation model. At the same time, according to the different requirements of the present disclosure for the hair transformation model (for example, short hair style to long hair style, long hair style to short hair style, transformation of different short hair styles, transformation of different long hair styles), the CycleGAN neural network can be controlled in a general direction to generate corresponding sample data, or the CycleGAN neural network can be controlled to generate a large number of character image pairs with different hairstyles, and the corresponding sample data can be selected from them. The embodiments of the present application do not make specific restrictions on this.
[0087] Of course, the present disclosure is not limited to using the CycleGAN neural network to obtain sample data, but may also be any other feasible method.
[0088] 302. Determine a first color guide map according to the first sample character image and the second sample character image.
[0089] The first color guide map includes a hair generation area that is the same as the hair segmentation area in the second sample character image; the color of the hair generation area in the first color guide map is obtained according to the color of the hair in the first sample character image.
[0090] Exemplarily, the first color guide map is as follows Figure 4 As shown in c, it includes a hair generation area that is the same as the hair segmentation area (the hairstyle of the person's hair) in the second sample person image, and the determining factor of the color of the hair generation area includes the color of the hair in the first sample person image.
[0091] 303. Using the first sample person image and the first color guide map as training data and the second sample person image as supervisory information, a hair transformation model is obtained through training.
[0092] The supervised learning algorithm used in step 303 may be any feasible algorithm and is not specifically limited here.
[0093] When the hair generation area in the first color guidance map and the hair segmentation area in the second sample character image are the same, the determining factor of the color of the hair generation area in the first color guidance map includes the color of the hair in the first sample character image. After the hair transformation model is trained using the first sample character image and the first color guidance map as training data and the second sample character image as supervisory information, if the hair transformation model needs to be used to transform the hair in the source character image, a preset color guidance map with a target hairstyle (target hair generation area) and hair color similar to the hair color in the first sample character image can be determined using the source character image. Then, by inputting the source character image and the preset color guidance map into the hair transformation model, a target character image with a hairstyle that is a target hairstyle and hair color close to the hair color in the source character image can be obtained.
[0094] In the above disclosed embodiment, when training the hair transformation model, multiple pairs of samples are first obtained, and the hairstyles of the first sample person image and the second sample person image in each pair of samples are inconsistent, and the second sample person image is obtained from the first sample person image; then, according to these sample pairs, a corresponding first color guidance map is generated, and the hair generation area in the first color guidance map is the same as the hair segmentation area in the second sample person image, and its color is obtained from the color of the hair in the first sample person image. Finally, the hair transformation model can be trained by using the first sample person image and the first color guidance map as training data and the second sample person image as supervisory information. In this way, the obtained hair transformation model can output a person image after hair transformation after receiving a person image and information indicating the corresponding hairstyle. Because the samples used for the entire model training are large in number, and the first color guidance map whose color is determined by the first sample person image is used, and the second sample person image whose hairstyle is the same as that in the first color guidance map is used as supervisory information, the model finally obtained can make it possible to achieve a better consistency in the color of the transformed hair and the hair before transformation when completing the hair transformation function.
[0095] In one implementation of the present disclosure, in combination with Figure 3 ,like Figure 5 As shown, the above step 302 includes 3021 and 3022.
[0096] 3021. Determine a target hair mask image of the second sample person image.
[0097] Exemplarily, step 3021 can be implemented by segmenting the hair in the second sample person image according to a preset hair segmentation network. Figure 6 Taking the image shown in a as an example, the target hair mask is Figure 6 As shown in b.
[0098] 3022. Determine a first color guide map according to the color value of the hair in the first sample character image, the target hair mask map, and the color value of the hair corresponding to the target hair mask map.
[0099] The technical solution provided by the above-mentioned disclosed embodiment can separately segment the hair in the second sample character image to form a corresponding target hair mask map, and then determine the first color guide map according to the color value of the hair in the first sample character image, the target hair mask map and the color value of the hair corresponding to the target hair mask map, so that the first color guide map can fully reflect the hairstyle of the hair in the second sample character image and the color of the hair in the first sample character image, and is not affected by the hairstyle of the hair in the first sample character image and the color of the pixels in the area other than the hair in the second sample character image, thereby improving the efficiency of the entire model training process.
[0100] In one implementation of the present disclosure, Figure 5 , refer to Figure 7 As shown, the above step 3022 includes SA1 and SA2.
[0101] SA1. Calculate the statistical value of the hair color in the target hair mask image.
[0102] SA2. Replace the color values of all pixels of the hair in the target hair mask image with the statistical values of the color values of the hair in the first sample person image to obtain a first color guide map.
[0103] For example, the first sample person image is as follows Figure 8 As shown in a, the second sample character image is as follows Figure 8 As shown in b, the first color guide map is finally obtained as Figure 8 As shown in c.
[0104] The technical solution provided by the above-mentioned disclosed embodiment can replace the color values of all pixels of the hair in the target hair mask image with the statistical values of the color values of the hair in the first sample character image, so that the color of the hair in the target hair mask image is statistically the same as the color value of the hair in the first sample character image. This target hair mask image can be used as the first color guide image, and it is close to the color of the hair in the first sample character image, thereby ensuring the correlation between the two sample data and enhancing the effect of the subsequently obtained hair transformation model.
[0105] Optional, because in practice, the role of the hair transformation model may be to change the short hair style in the source character image into a long hair style, and the second sample character image obtained by using the first sample character image using cyclegan for this purpose is likely to have a different color of hair than the first sample character image. Therefore, if only using Figure 7 The method for generating the first model guidance map in the above-mentioned manner may result in that the hair color of the target person image obtained by the hair transformation model finally trained is not very similar to the hair color of the source person image due to the large difference between the supervision information and the training samples. Therefore, in one achievable manner, when the hair area in the second sample person image is larger than the hair area in the first sample person image, the color of the hair generation area in the first color guidance map in the related statement of the aforementioned step 302 is obtained according to the hair color in the first sample person image, including: the color of the hair generation area in the first color guidance map is obtained according to the hair color in the first sample person image and the hair color in the second sample person image, combined with Figure 5 , refer to Fig. 9 As shown, the above step 3022 includes SB1 and SB2.
[0106] SB1. Calculate the statistical value of the hair color value in the first sample person image and the statistical value of the hair color value corresponding to the second area in the target hair mask image except the first area corresponding to the area of the hair in the first sample person image.
[0107] SB2. Replace the color values of all pixels in the first area with the statistical values of the hair color values in the first sample character image, and replace the color values of all pixels in the second area with the statistical values of the hair color values corresponding to the second area to obtain a first color guide map.
[0108] For example, the first sample person image is as follows Fig.10 As shown in a, the statistical value of the hair color value in the first sample character image is Fig.10 The statistical value corresponding to the hair color shown in a in the figure, the second sample character image is as follows Fig.10 As shown in c, the first color guide map is finally obtained as Fig.10 In addition, it can be seen that compared with Fig.10 As shown in b, Fig.10 The hairstyle shown in c is missing some bangs, so when filling, there is no need to fill these bangs areas. The same applies to the subsequent situations.
[0109] Based on the above technical scheme, when the purpose of the hair transformation model generation method is to generate a hair transformation model that changes the short hair style in the source character image into a long hair style, the first color guide map can not only reflect the overall color of the hair in the first sample character image, but also reflect the gradual change of the hair color when the hair is long, so that the overall distribution of the hair color in the first color guide map is more realistic and natural, so that the final hair transformation model not only has good color consistency but also looks more natural when changing the short hair style into the long hair style.
[0110] Optional, because in practice, the role of the hair transformation model may be to change the long hair style in the source character image into a short hair style, and the color of the hair that is more than the second sample character image obtained by using cyclegan for this point is likely to be different from the color of the hair in the first sample character image. Therefore, if the user needs the color of the target character image to be similar to the color of the hair of only the short hair style (target hairstyle) in the source character image, then if you use Figure 7 The first model guidance map is generated in the manner described in the above. Although the hair color output by the hair transformation model is more uniform than the hair color of the source person image, due to the differences between the supervision information and the training samples, the hair color of the target person image obtained by the hair transformation model is similar to the overall hair color of the source person image, but the hair color of the area corresponding to the short hair style in the source person image is not similar, which may not meet the user's needs. Therefore, in order to avoid this, a feasible method is to combine Figure 5 , refer to Fig.11 As shown, the above step 3022 includes SC1 and SC2.
[0111] SC1. Calculate a statistical value of a color value of hair in a third region of the first sample person image that corresponds to a region of hair in a target hair mask image.
[0112] SC2. Replace the color values of all pixels in the hair region of the target hair mask image with the statistical values of the hair color values in the third region to obtain a first color guide image.
[0113] For example, the first sample person image is as follows Fig.12 As shown in a, the second sample character image is as follows Fig.12 Taking the example shown in b, the first color guide map is finally obtained as follows Fig.12 As shown in c.
[0114] In one implementation of the present disclosure, Figure 3 , refer to Fig.13 As shown, the above step 303 also includes steps 304-310.
[0115] 304. Obtain a plurality of third sample person images.
[0116] 305. Obtain at least one preset color guide image.
[0117] The preset color guide map only includes hair with a target hairstyle corresponding to the preset color guide map, that is, each preset color guide map corresponds to a hair generation area. Fig.14 As shown in a, b and c, there are three different preset color guide maps; the color of the hair in each preset color guide map can be white.
[0118] 306. Calculate the statistical value of the hair color value in the third sample person image.
[0119] 307 . Replace the color values of all pixels of the hair in the preset color guide map with the statistical values of the color values of the hair in the third sample person image to obtain a second color guide map.
[0120] 308. Input the third sample character image and the second color guide map into the hair transformation model to obtain a fourth sample character image.
[0121] For example, Fig.15 a is the third sample character image, and the second color guide map is Fig.14 For example, the fourth sample character image is as follows Fig.15 As shown in b.
[0122] 309. Using the third sample person image and the hairstyle variable as training data and the fourth sample person image corresponding to the hairstyle variable as supervision information, a target hair transformation model is obtained through training.
[0123] The hairstyle variables correspond to the target hairstyles one by one. Fig.14 For example, the hairstyle variables can be 1, 2, and 3, which can correspond to Fig.14 a, b and c in .
[0124] 310. Update the hair transformation model to the target hair transformation model.
[0125] The above disclosed embodiment can utilize a large number of third sample character images according to the hair transformation model obtained through the aforementioned training to produce a large number of new sample data pairs (third sample character images and fourth sample character images) corresponding to different target hairstyles, and then based on these new third sample character image pairs and hairstyle variables, as well as the fourth sample character images corresponding to the hairstyle variables, a target character image having a target hairstyle corresponding to the selected hairstyle variables can be trained based on the source character image and the selected hairstyle variables, and the hair color of the target character image is similar to the hair color of the source character image. When the hair transformation model is used to transform the hair, compared to the prior art, the color of the transformed hair can be made more consistent with the color of the hair before the transformation. Furthermore, compared Figure 3 As for the technical solution provided, the technical solution provided by the above-mentioned public embodiment does not need to generate a color guide map. After receiving the source character image and hairstyle variable, the corresponding target character image can be directly input, and the entire processing process is more efficient; at the same time, because the hairstyle variable can be determined by the user (for example, after the user selects one of the multiple optional hairstyles, the hairstyle variable corresponding to the selected hairstyle is used as the hairstyle variable determined by the user), target character images with different target hairstyles can be generated according to user needs, thereby improving user experience.
[0126] Optionally, in order to prevent the abnormal values (large R value, G value or B value) in the RGB value of the hair color of the person in the source person image due to factors such as the external environment (highlight) during imaging or the person's own hair treatment (highlighting) from affecting the color distribution of the entire hair, the statistical value of the hair color value is calculated in all the above embodiments as follows: after arranging the RGB values of all pixels of the hair color in ascending order according to the color and value, the average value of the RGB values corresponding to the colors and values in the preset order range is calculated, and the average value of the RGB values corresponding to the colors and values in the preset order range is used as the statistical value of the hair color value; the color and value is the sum of the R value, G value and B value in the RGB value. Among them, the preset order range can be 20%-30% of the total order number (only for example, depending on the actual situation). In this way, because the abnormal values caused by highlights or highlights are unlikely to be within the preset order range, even if some exist, they will only be a small part, so the final statistical value is also avoided from being affected by these abnormal values. For example, there are 10 pixels in a part of hair, and the RGB values corresponding to the 10 pixels are pixel 1 (122, 254, 0), pixel 2 (123, 253, 12), pixel 3 (125, 254, 0), pixel 4 (122, 254, 10), pixel 5 (122, 250, 0), pixel 6 (126, 254, 0), pixel 7 (222, 255, 255), pixel 8 (120, 254, 0), pixel 9 (122, 254, 30), pixel 10 (255, 254, 0). The colors and values of the ten pixels are as follows: The order is 376, 376, 379, 386, 382, 380, 732, 374, 376, 509; the ascending order is: pixel 8, pixel 1, pixel 2, pixel 9, pixel 3, pixel 6, pixel 5, pixel 4, pixel 10, pixel 7; the average value of the RGB values corresponding to the colors and values whose order is within the preset order range (20%-30%) is ((122+123) / 2, (254+253) / 2, 6)=(122.5, 253.5, 6); the statistical value of the color value of this part of hair is (122.5, 253.5, 6).
[0127] The hair transformation model generated by the hair transformation model generation method of the aforementioned embodiment is referred to as Fig.16 As shown, the embodiment of the present disclosure provides a hair transformation method applied to a hair transformation device. The hair transformation device and the hair transformation model generating device in the above-mentioned embodiment can be the same device; the hair transformation device can be an electronic device, and the electronic device can be Figure 1 The method includes 401 and 402.
[0128] 401. Obtain source person image and hairstyle indication information.
[0129] The source person image may be a person image captured by the hair transformation device in response to the user's shooting operation through a camera device set by the hair transformation device, or may be a person image obtained from other devices. The source person image may be a single person photo or a frame of a person image in a video, and is not specifically limited here.
[0130] The hairstyle indication information is used to indicate the target hair generation area; that is, the hairstyle indication information is used to determine the hairstyle of the target person image.
[0131] 402. Input the source person image and the hairstyle indication information into the hair transformation model to obtain a target person image.
[0132] Among them, the hair transformation model is obtained by the hair transformation model generation method provided by the aforementioned embodiment; the hairstyle of the person in the target person image is the hairstyle corresponding to the target hair generation area indicated by the hairstyle indication information, and the hair color of the person image in the target person image and the hair color of the person image in the source person image are more similar than a preset value.
[0133] Exemplarily, the above-mentioned preset values can be determined according to actual needs; the similarity of hair color can refer to the difference between the relevant statistical values (such as mean, variance, standard deviation, etc.) of the hair color values of the target person image and the relevant statistical values of the hair color values of the source person image in a statistical sense. The smaller the difference, the greater the similarity, and the greater the difference, the smaller the similarity.
[0134] For example, the source character image is Fig.15 For example, the character image shown in a in the figure has short hair. After using the above step 402, Fig.15 In the image shown in b, the hairstyle of the person in this image becomes long hair, and its color and Fig.15 The hair colors of the people in the image shown in (a) are similar.
[0135] In the above disclosed embodiments, when the hair transformation device needs to perform a hair transformation function, because the hair transformation model is generated using the technical solution provided by the above embodiments, a target person image can be obtained using the source person image, in which the hair color after the hairstyle transformation is more consistent with the hair color of the source person image. Furthermore, because the entire process only needs to use the hair transformation model, it avoids the image post-processing process with a large amount of calculation after the hairstyle transformation as in the prior art, thereby reducing the demand for computing resources by the hair transformation device.
[0136] In one implementation of the present disclosure, Figure 3 For the corresponding embodiments, refer to Fig.17As shown, when the hairstyle indication information includes a preset color guide map, and the preset color guide map includes a target hair generation area, the above step 402 specifically includes 402A1-402A3.
[0137] 402A1. Calculate the statistical value of the hair color value in the source person image.
[0138] The specific calculation method can refer to the description in the above embodiment and will not be repeated here.
[0139] 402A2. Replace the color values of all pixels in the target hair generation area in the preset color guide map with the statistical values of the color values of the hair in the source person image to obtain a target color guide map.
[0140] The preset color guide map only includes hair with a target hairstyle, and the color of the hair may not exist (transparent or white); the preset color guide map may be pre-stored.
[0141] 402A3. Input the source character image and the target color guide map into the hair transformation model to obtain the target character image.
[0142] In the above disclosed embodiment, after determining a target color guide map using a source character image, the source character image and the target color guide map can be input into the hair transformation model generated by the technical solution provided by the above embodiment to obtain the desired target character image. Compared with the prior art, because the hair transformation model is generated using the technical solution provided by the above embodiment, the source character image can be used to obtain a target character image whose hair color after the hairstyle transformation is more consistent with the hair color of the source character image. Furthermore, because the entire process only requires the use of the hair transformation model after the source character image and the preset color guide map are processed accordingly to obtain the target color guide map, the target character image can be obtained, thereby avoiding the image post-processing process with a large amount of calculation after the hairstyle transformation as in the prior art, thereby reducing the demand for computing resources by the hair transformation device.
[0143] In one implementation of the present disclosure, Fig.13 In a corresponding embodiment, that is, when the hair transformation model is updated to the target hair transformation model obtained by the hair transformation model generation method provided in the above embodiment, the hairstyle indication information includes a target hairstyle variable corresponding to the target hair generation area; Fig.18 As shown, the above step 402 specifically includes: inputting the source character image and the target hairstyle variable into the updated hair transformation model to obtain the target character image; the hairstyle of the character in the target character image corresponds to the target hair generation area.
[0144] The target hairstyle variable is any one of the hairstyle variables, and the hairstyle of the person in the target person image is the target hairstyle corresponding to the target hairstyle variable. Exemplarily, when the hair transformation device is an electronic device, the target hairstyle variable can be obtained by displaying a plurality of hairstyle variables on a display screen of the electronic device, and in response to a click operation on the target hairstyle variable, the target hairstyle variable is obtained.
[0145] For example, the source character image is Fig.15 In the image shown in a, the target hairstyle variable is 1, and the target hairstyle corresponding to the target hairstyle variable is Fig.14 As shown in a, the target person image finally obtained by 402 is Fig.15 As shown in b.
[0146] In the above disclosed embodiment, after obtaining a target hairstyle variable, the source character image and the target hairstyle variable can be input into the updated hair transformation model (target hair transformation model) to obtain the desired target character image. Fig.17 For the corresponding embodiment, the entire processing flow is faster, and because the target hairstyle variable can be determined by the user, a target character image with a hairstyle that meets the user's requirements can be generated, thereby improving the user experience.
[0147] The above embodiments mainly introduce the solutions provided by the embodiments of the present disclosure from the perspective of the hair transformation device (electronic device (terminal / server)). It can be understood that in order to implement the above methods, the hair transformation model generation device includes hardware structures and / or software modules corresponding to the execution of each method flow, and these hardware structures and / or software modules corresponding to the execution of each method flow can constitute a hair transformation model generation device, and the same is true for the hair transformation device. It should be easy for those skilled in the art to realize that, in combination with the algorithm steps of each example described in the embodiments disclosed herein, the present disclosure can be implemented in the form of hardware or a combination of hardware and computer software. Whether a certain function is executed in the form of hardware or computer software driving hardware depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to exceed the scope of the present disclosure.
[0148] The embodiment of the present disclosure can divide the hair transformation device into functional modules according to the above method example. For example, the hair transformation model generation device or the hair transformation device can divide each functional module corresponding to each function, or two or more functions can be integrated into one processing module. The above integrated module can be implemented in the form of hardware or in the form of software functional modules. It should be noted that the division of modules in the embodiment of the present disclosure is schematic and is only a logical functional division. There may be other division methods in actual implementation.
[0149] In the case of dividing each functional module into corresponding functional modules, Fig.19 A possible structural diagram of a hair transformation model generating device 03 is shown. The hair transformation model generating device 03 comprises: an acquisition module 51 , a processing module 52 and a training module 53 .
[0150] Among them, the acquisition module 51 is configured to acquire a first sample character image and a second sample character image corresponding to the first sample character image; the second sample character image is obtained based on the first sample character image, and the hairstyle of the character in the second sample character image is inconsistent with the first sample character image; the processing module 52 is configured to determine a first color guidance map based on the first sample character image and the second sample character image acquired by the acquisition module 51; the first color guidance map includes a hair generation area that is the same as the hair segmentation area in the second sample character image; the color of the hair generation area in the first color guidance map is obtained based on the color of the hair in the first sample character image; the training module 53 is configured to use the first sample character image acquired by the acquisition module 51 and the first color guidance map determined by the processing module 52 as training data, and the second sample character image acquired by the acquisition module 51 as supervision information, to train a hair transformation model.
[0151] Optionally, the processing module 52 specifically includes a first determining unit 521 and a second determining unit 522;
[0152] The first determination unit 521 is configured to determine the target hair mask map of the second sample character image acquired by the acquisition module 51; the second determination unit 522 is configured to determine the first color guide map based on the color value of the hair in the first sample character image acquired by the acquisition module 51, the target hair mask map determined by the first determination unit 521, and the color value of the hair corresponding to the target hair mask map.
[0153] Further optionally, the second determination unit 522 is specifically configured to: calculate the statistical value of the color value of the hair in the first sample character image obtained by the acquisition module 51; replace the color values of all pixels of the hair in the target hair mask image determined by the first determination unit 521 with the statistical value of the color value of the hair in the first sample character image to obtain a first color guide map.
[0154] Further optionally, in a case where the hair area in the second sample character image is larger than the hair area in the first sample character image, the color of the hair generation area in the first color guidance map is obtained according to the hair color in the first sample character image, including: the color of the hair generation area in the first color guidance map is obtained according to the hair color in the first sample character image and the hair color in the second sample character image;
[0155] The second determination unit 522 is specifically configured to: calculate the statistical value of the color value of the hair in the first sample character image obtained by the acquisition module 51, and the statistical value of the color value of the hair corresponding to the second area in the target hair mask image determined by the first determination unit 521 except the first area corresponding to the area of the hair in the first sample character image; replace the color values of all pixels in the first area with the statistical value of the color value of the hair in the first sample character image, and replace the color values of all pixels in the second area with the statistical value of the color value of the hair corresponding to the second area, so as to obtain a first color guide map.
[0156] Further optionally, when the hair area in the first sample character image is larger than the hair area in the second sample character image, the second determination unit 522 is specifically configured to: calculate the statistical value of the hair color value in the third area of the first sample character image acquired by the acquisition module 51 corresponding to the hair area in the target hair mask image determined by the first determination unit 521; replace the color values of all pixels in the hair area in the target hair mask image with the statistical value of the hair color value in the third area to obtain a first color guide map.
[0157] Further optionally, after the training module 53 obtains the hair transformation model through training, the acquisition module 51 is further configured to obtain a plurality of third sample character images; the acquisition module 51 is further configured to obtain at least one preset color guide map; each preset color guide map corresponds to a hair generation area; the second determination unit 522 is further configured to calculate the statistical value of the color value of the hair in the third sample character image obtained by the acquisition module 51; the second determination unit 522 is further configured to replace the color values of all pixels of the hair in the preset color guide map obtained by the acquisition module 51 with the statistical value of the color value of the hair in the third sample character image, so as to obtain a second color guidance map; the acquisition module 51 is further configured to input the third sample character image and the second color guidance map obtained by the second determination unit 522 into the hair transformation model obtained through training by the training module 53, so as to obtain a fourth sample character image; the training module 53 is further configured to use the third sample character image and the hairstyle variable obtained by the acquisition module 51 as training data, and the fourth sample character image corresponding to the hairstyle variable as supervision information, to train and obtain a target hair transformation model; the hairstyle variable corresponds one-to-one to the hair generation area; the training module 53 is further configured to update the hair transformation model to the target hair transformation model.
[0158] Optionally, the second determination unit 522 is specifically configured to: arrange the RGB values of all pixels of the hair color in ascending order according to the color and value, calculate the average value of the RGB values corresponding to the colors and values whose order is within a preset order range, and take the average value of the RGB values corresponding to the colors and values whose order is within the preset order range as the statistical value of the hair color value; the color sum value is the sum of the R value, the G value and the B value in the RGB value.
[0159] Regarding the hair transformation model generation device in the above embodiment, the specific manner in which each module performs operations and the corresponding beneficial effects have been described in detail in the embodiment of the hair transformation model generation method mentioned above, and will not be repeated here.
[0160] In the case of dividing each functional module into corresponding functional modules, Fig. 20 A possible structural schematic diagram of the hair transformation device 04 is shown. The hair transformation device 04 includes: an acquisition module 61 and a processing module 62 .
[0161] The acquisition module 61 is configured to acquire a source person image and hairstyle indication information; the hairstyle indication information is used to indicate a target hair generation area;
[0162] The processing module 62 is configured to input the source character image and hairstyle indication information obtained by the acquisition module 61 into the hair transformation model to obtain a target character image; the hair transformation model is obtained by the hair transformation model generation method provided in the first aspect; the hairstyle of the character in the target character image is the hairstyle corresponding to the target hair generation area indicated by the hairstyle indication information, and the similarity between the hair color of the character image in the target character image and the hair color of the character image in the source character image is greater than a preset value.
[0163] Optionally, the hairstyle indication information includes a preset color guide map, and the preset color guide map includes a target hair generation area; the processing module 62 is specifically configured to: calculate the statistical value of the color value of the hair in the source character image obtained by the acquisition module 61; replace the color values of all pixels in the target hair generation area in the preset color guide map obtained by the acquisition module 61 with the statistical value of the color value of the hair in the source character image to obtain the target color guidance map; input the source character image and the target color guidance map obtained by the acquisition module 61 into the hair transformation model to obtain the target character image.
[0164] Optionally, when the hair transformation model is updated to the target hair transformation model obtained by the hair transformation model generation method provided in the first aspect, the hairstyle indication information includes a target hairstyle variable corresponding to the target hair generation area; the processing module 62 is specifically configured to: input the source character image and the target hairstyle variable obtained by the acquisition module 61 into the updated hair transformation model to obtain the target character image; the hairstyle of the character in the target character image corresponds to the target hair generation area.
[0165] Regarding the hair transformation device in the above embodiment, the specific manner in which each module performs operations and the corresponding beneficial effects have been described in detail in the above embodiment of the hair transformation method, and will not be repeated here.
[0166] Fig.21 FIG. 1 is a possible structural diagram of an electronic device according to an exemplary embodiment. The electronic device may be the hair transformation model generating device 03 and / or the hair transformation device 04 described above. Fig.21 As shown, the electronic device includes a processor 71 and a memory 72. The memory 72 is used to store instructions executable by the processor 71, and the processor 71 can implement the functions of each module in the hair transformation model generation device 03 and / or the hair transformation device 04 in the above embodiments. The memory 72 stores at least one instruction, and the at least one instruction is loaded and executed by the processor 71 to implement the hair transformation model generation and / or hair transformation method provided by the above method embodiments.
[0167] In a specific implementation, as an embodiment, the processor 71 (71-1 and 71-2) may include one or more CPUs, such as Fig.21 As an embodiment, the electronic device may include a plurality of processors 71, for example Fig.21 The processor 71-1 and the processor 71-2 shown in FIG. Each CPU in these processors 71 may be a single-core processor (Single-CPU) or a multi-core processor (Multi-CPU). The processor 71 here may refer to one or more devices, circuits, and / or processing cores for processing data (e.g., computer program instructions).
[0168] The memory 72 may be a read-only memory (ROM) or other types of static storage devices that can store static information and instructions, a random access memory (RAM) or other types of dynamic storage devices that can store information and instructions, or an electrically erasable programmable read-only memory (EEPROM), a compact disc read-only memory (CD-ROM) or other optical disc storage, optical disc storage (including compressed optical disc, laser disc, optical disc, digital versatile disc, Blu-ray disc, etc.), a disk computer storage medium or other magnetic storage device, or any other medium that can be used to carry or store the desired program code in the form of instructions or data structures and can be accessed by a computer, but is not limited thereto. The memory 72 may exist independently and be connected to the processor 71 via a bus 73. The memory 72 may also be integrated with the processor 71.
[0169] The bus 73 may be an industry standard architecture (ISA) bus, a peripheral component interconnect (PCI) bus, or an extended industry standard architecture (EISA) bus. The bus 73 may be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Fig.21 Only one thick line is used in the diagram, but this does not mean that there is only one bus or only one type of bus.
[0170] In addition, in order to facilitate information exchange between the electronic device and other devices (for example, when the electronic device is a terminal, the electronic device exchanges information with a server, or when the electronic device is a server, the electronic device exchanges information with a terminal), the electronic device includes a communication interface 74. The communication interface 74 uses any transceiver-like device for communicating with other devices or communication networks, such as a control system, a radio access network (RAN), a wireless local area network (WLAN), etc. The communication interface 74 may include a receiving unit to implement a receiving function, and a sending unit to implement a sending function.
[0171] In some embodiments, when the electronic device is a terminal, it may optionally include: a peripheral device interface 75 and at least one peripheral device. The processor 71, the memory 72 and the peripheral device interface 75 may be connected via a bus 73 or a signal line. Each peripheral device may be connected to the peripheral device interface 75 via a bus 73, a signal line or a circuit board. Specifically, the peripheral device includes: at least one of a radio frequency circuit 76, a display screen 77, a camera 78, an audio circuit 79, a positioning component 80 and a power supply 81.
[0172] The peripheral device interface 75 can be used to connect at least one peripheral device related to I / O (input / output) to the processor 71 and the memory 72. In some embodiments, the processor 71, the memory 72 and the peripheral device interface 75 are integrated on the same chip or circuit board; in some other embodiments, any one or two of the processor 71, the memory 72 and the peripheral device interface 75 can be implemented on a separate chip or circuit board, which is not limited in this embodiment.
[0173] The radio frequency circuit 76 is used to receive and transmit RF (radio frequency) signals, also known as electromagnetic signals. The radio frequency circuit 76 communicates with the communication network and other communication devices through electromagnetic signals. The radio frequency circuit 76 converts the electrical signal into an electromagnetic signal for transmission, or converts the received electromagnetic signal into an electrical signal. Optionally, the radio frequency circuit 404 includes: an antenna system, an RF transceiver, one or more amplifiers, a tuner, an oscillator, a digital signal processor, a codec chipset, a user identity module card, etc. The radio frequency circuit 76 can communicate with other devices through at least one wireless communication protocol. The wireless communication protocol includes but is not limited to: a metropolitan area network, various generations of mobile communication networks (2G, 3G, 4G and 5G), a wireless local area network and / or a Wi-Fi (wireless fidelity) network. In some embodiments, the radio frequency circuit 76 may also include circuits related to NFC (near field communication), which is not limited in the present disclosure.
[0174] The display screen 77 is used to display a UI (user interface). The UI may include graphics, text, icons, videos, and any combination thereof. When the display screen 77 is a display screen, the display screen 77 also has the ability to collect touch signals on the surface or above the surface of the display screen 77. The touch signal can be input as a control signal to the processor 71 for processing. At this time, the display screen 77 can also be used to provide virtual buttons and / or virtual keyboards, also known as soft buttons and / or soft keyboards. In some embodiments, the display screen 77 can be a front panel of an electronic device; the display screen 77 can be made of materials such as LCD (liquid crystal display), OLED (organic light-emitting diode), etc.
[0175] The camera assembly 78 is used to capture images or videos. Optionally, the camera assembly 78 includes a front camera and a rear camera. Usually, the front camera is set on the front panel of the electronic device, and the rear camera is set on the back of the electronic device. The audio circuit 79 may include a microphone and a speaker. The microphone is used to collect sound waves from the user and the environment, and convert the sound waves into electrical signals and input them into the processor 71 for processing, or input them into the radio frequency circuit 76 to achieve voice communication. For the purpose of stereo acquisition or noise reduction, there can be multiple microphones, which are respectively set in different parts of the electronic device. The microphone can also be an array microphone or an omnidirectional acquisition microphone. The speaker is used to convert the electrical signal from the processor 71 or the radio frequency circuit 76 into sound waves. The speaker can be a traditional film speaker or a piezoelectric ceramic speaker. When the speaker is a piezoelectric ceramic speaker, it can not only convert the electrical signal into a sound wave audible to humans, but also convert the electrical signal into a sound wave that cannot be heard by humans for purposes such as ranging. In some embodiments, the audio circuit 79 may also include a headphone jack.
[0176] The positioning component 80 is used to locate the current geographic location of the electronic device to achieve navigation or LBS (location based service). The positioning component 80 can be a positioning component based on the US GPS (global positioning system), China's Beidou system, Russia's Grenas system or the European Union's Galileo system.
[0177] The power supply 81 is used to power various components in the electronic device. The power supply 81 can be an alternating current, a direct current, a disposable battery, or a rechargeable battery. When the power supply 81 includes a rechargeable battery, the rechargeable battery can support wired charging or wireless charging. The rechargeable battery can also be used to support fast charging technology.
[0178] In some embodiments, the electronic device further includes one or more sensors 710. The one or more sensors 710 include but are not limited to: an acceleration sensor, a gyroscope sensor, a pressure sensor, a fingerprint sensor, an optical sensor, and a proximity sensor.
[0179] The acceleration sensor can detect the magnitude of acceleration on the three coordinate axes of the coordinate system established by the electronic device. The gyroscope sensor can detect the body direction and rotation angle of the electronic device. The gyroscope sensor can cooperate with the acceleration sensor to collect the user's 3D actions on the electronic device. The pressure sensor can be set on the side frame of the electronic device and / or the lower layer of the display screen 77. When the pressure sensor is set on the side frame of the electronic device, the user's holding signal of the electronic device can be detected. The fingerprint sensor is used to collect the user's fingerprint. The optical sensor is used to collect the ambient light intensity. The proximity sensor, also known as the distance sensor, is usually set on the front panel of the electronic device. The proximity sensor is used to collect the distance between the user and the front of the electronic device.
[0180] Those skilled in the art will understand that Fig.21 The structure shown in the figure does not constitute a limitation on the electronic device, and may include more or fewer components than shown in the figure, or combine certain components, or adopt different component arrangements. The embodiment of the present disclosure also provides a computer storage medium, which stores instructions. When the instructions on the computer storage medium are executed by a processor of an electronic device, the electronic device can execute the hair transformation method provided in the above embodiment.
[0181] The embodiments of the present disclosure also provide a computer storage medium having instructions stored thereon. When the instructions on the computer storage medium are executed by a processor of an electronic device, the electronic device is enabled to execute the hair transformation model generation method provided in the aforementioned embodiments.
[0182] The embodiment of the present disclosure also provides a computer program product including instructions, which, when executed on a computer, enables the computer to execute the hair transformation model generation method provided in the aforementioned embodiment.
[0183] The embodiment of the present disclosure also provides a computer program product including instructions, which, when executed on a computer, enables the computer to execute the hair transformation method provided by the aforementioned embodiment.
[0184] Those skilled in the art will readily appreciate other embodiments of the present disclosure after considering the specification and practicing the invention disclosed herein. The present disclosure is intended to cover any variations, uses or adaptations of the present disclosure that follow the general principles of the present disclosure and include common knowledge or customary techniques in the art that are not disclosed in the present disclosure. The description and examples are to be considered exemplary only, and the true scope and spirit of the present disclosure are indicated by the following claims.
[0185] It should be understood that the present disclosure is not limited to the exact structures that have been described above and shown in the drawings, and that various modifications and changes may be made without departing from the scope thereof. The scope of the present disclosure is limited only by the appended claims.
Claims
1. A method for generating a hair transformation model, It is characterized in that include: Acquire a first sample character image and a second sample character image corresponding to the first sample character image; The second sample person image is obtained based on the first sample person image, and the hairstyle of the person in the second sample person image is inconsistent with that in the first sample person image; Determine a first color guide map according to the first sample person image and the second sample person image; the first color guide map includes a hair generation area that is the same as the hair segmentation area in the second sample person image; The color of the hair generation area in the first color guide image is obtained according to the color of the hair in the first sample person image; The hair transformation model is trained by using the first sample character image and the first color guide map as training data and the second sample character image as supervision information.
2. The method for generating a hair transformation model according to claim 1, It is characterized in that Determining a first color guide map according to the first sample person image and the second sample person image includes: Determining a target hair mask image of the second sample person image; The first color guide map is determined according to the color value of the hair in the first sample character image, the target hair mask map, and the color value of the hair corresponding to the target hair mask map.
3. The method for generating a hair transformation model according to claim 2, It is characterized in that The determining the first color guide map according to the color value of the hair in the first sample person image, the target hair mask map, and the color value of the hair corresponding to the target hair mask map comprises: Calculating a statistical value of the color value of the hair in the first sample person image; The color values of all pixels of hair in the target hair mask image are replaced with statistical values of the color values of hair in the first sample person image to obtain the first color guide map.
4. The method for generating a hair transformation model according to claim 2, It is characterized in that In the case where the hair area in the second sample person image is larger than the hair area in the first sample person image, The color of the hair generation area in the first color guide map is obtained according to the color of the hair in the first sample character image, including: the color of the hair generation area in the first color guide map is obtained according to the color of the hair in the first sample character image and the color of the hair in the second sample character image; The determining the first color guide map according to the color value of the hair in the first sample person image, the target hair mask map, and the color value of the hair corresponding to the target hair mask map comprises: Calculate the statistical value of the color value of the hair in the first sample person image and the statistical value of the color value of the hair corresponding to the second area in the target hair mask image except the first area corresponding to the area of the hair in the first sample person image; The color values of all pixels in the first area are replaced with the statistical values of the color values of the hair in the first sample person image, and the color values of all pixels in the second area are replaced with the statistical values of the color values of the hair corresponding to the second area to obtain the first color guide map.
5. The method for generating a hair transformation model according to claim 2, It is characterized in that In a case where the hair area in the first sample person image is larger than the hair area in the second sample person image, determining the first color guide map according to the color value of the hair in the first sample person image, the target hair mask map, and the color value of the hair corresponding to the target hair mask map includes: Calculate a statistical value of a color value of hair in a third area of the first sample person image corresponding to an area of hair in the target hair mask image; The color values of all pixels in the hair region in the target hair mask image are replaced with the statistical values of the hair color values in the third region to obtain the first color guide image.
6. The method for generating a hair transformation model according to claim 1, It is characterized in that The first sample person image and the color guide map are used as training data, and the second sample person image is used as supervision information. After the hair transformation model is obtained through training, the method further includes: Acquire a plurality of third sample person images; Obtain at least one preset color guide map; each of the preset color guide maps corresponds to a hair generation area; Calculate the statistical value of the color value of the hair in the third sample person image; Replacing the color values of all pixels of hair in the preset color guide map with the statistical values of the color values of hair in the third sample person image to obtain a second color guide map; Inputting the third sample character image and the second color guide map into the hair transformation model to obtain a fourth sample character image; The third sample character image and the hairstyle variable are used as training data, and the fourth sample character image corresponding to the hairstyle variable is used as supervision information, and a target hair transformation model is obtained through training; the hairstyle variable corresponds to the hair generation area one by one; The hair transformation model is updated to the target hair transformation model.
7. The method for generating a hair transformation model according to any one of claims 3 to 6, It is characterized in that The calculating of the statistical value of the hair color value comprises: After arranging the RGB values of all pixels of the hair color in ascending order according to the color and value, calculating the average value of the RGB values corresponding to the colors and values whose order is within a preset order range, and taking the average value of the RGB values corresponding to the colors and values whose order is within the preset order range as the statistical value of the hair color value; the color and value is the sum of the R value, the G value and the B value in the RGB value.
8. A hair transformation method, It is characterized in that include: Obtain source character image and hairstyle indication information; The hairstyle indication information is used to indicate a target hair growth area; The source person image and the hairstyle indication information are input into a hair transformation model to obtain a target person image; the hair transformation model is obtained by the hair transformation model generation method according to any one of claims 1 to 7; the hairstyle of the person in the target person image is the hairstyle corresponding to the target hair generation area indicated by the hairstyle indication information, and the similarity between the hair color of the person image in the target person image and the hair color of the person image in the source person image is greater than a preset value; The hairstyle indication information includes a preset color guide map, and the preset color guide map includes the target hair generation area; The step of inputting the source person image and the hairstyle indication information into the hair transformation model to obtain the target person image comprises: Calculate the statistical value of the hair color value in the source person image; Replacing the color values of all pixels in the target hair generation area in the preset color guide map with the statistical values of the color values of the hair in the source person image to obtain a target color guide map; Inputting the source character image and the target color guide map into the hair transformation model to obtain the target character image; In the case where the hair transformation model is updated to a target hair transformation model obtained by the method for generating a hair transformation model according to claim 6, the hairstyle indication information includes a target hairstyle variable corresponding to a target hair generation area; and the step of inputting the source person image and the hairstyle indication information into the hair transformation model to obtain the target person image includes: The source character image and the target hairstyle variable are input into the updated hair transformation model to obtain a target character image; the hairstyle of the character in the target character image corresponds to the target hair generation area.
9. A hair transformation model generation device, It is characterized in that The device comprises: an acquisition module, configured to acquire a first sample character image and a second sample character image corresponding to the first sample character image; the second sample character image is obtained based on the first sample character image, and the hairstyle of the character in the second sample character image is inconsistent with that in the first sample character image; a processing module configured to determine a first color guidance map according to the first sample person image and the second sample person image acquired by the acquisition module; the first color guidance map includes a hair generation area that is the same as the hair segmentation area in the second sample person image; the color of the hair generation area in the first color guidance map is obtained according to the color of the hair in the first sample person image; The training module is configured to use the first sample character image obtained by the acquisition module and the first color guide map determined by the processing module as training data, and the second sample character image obtained by the acquisition module as supervision information to train a hair transformation model.
10. The hair transformation model generating device according to claim 9, It is characterized in that The processing module specifically includes a first determining unit and a second determining unit; The first determining unit is configured to determine a target hair mask image of the second sample person image acquired by the acquiring module; The second determination unit is configured to determine the first color guide map according to the color value of the hair in the first sample character image acquired by the acquisition module, the target hair mask map determined by the first determination unit, and the color value of the hair corresponding to the target hair mask map.
11. The hair transformation model generating device according to claim 10, It is characterized in that The second determining unit is specifically configured to: Calculating a statistical value of the color value of the hair in the first sample person image acquired by the acquisition module; The color values of all pixels of hair in the target hair mask image determined by the first determining unit are replaced with statistical values of the color values of hair in the first sample person image to obtain the first color guide map.
12. The hair transformation model generating device according to claim 10, It is characterized in that In the case where the hair area in the second sample person image is larger than the hair area in the first sample person image, The color of the hair generation area in the first color guide map is obtained according to the color of the hair in the first sample character image, including: the color of the hair generation area in the first color guide map is obtained according to the color of the hair in the first sample character image and the color of the hair in the second sample character image; The second determining unit is specifically configured to: Calculate the statistical value of the hair color value in the first sample person image acquired by the acquisition module and the statistical value of the hair color value corresponding to the second area in the target hair mask image determined by the first determination unit except the first area corresponding to the hair in the first sample person image; The color values of all pixels in the first area are replaced with the statistical values of the color values of the hair in the first sample person image, and the color values of all pixels in the second area are replaced with the statistical values of the color values of the hair corresponding to the second area to obtain the first color guide map.
13. The hair transformation model generating device according to claim 10, It is characterized in that In a case where the hair area in the first sample person image is larger than the hair area in the second sample person image, the second determining unit is specifically configured as follows: Calculate a statistical value of the color value of hair in a third area of the first sample person image acquired by the acquisition module corresponding to the area of hair in the target hair mask image determined by the first determination unit; The color values of all pixels in the hair region in the target hair mask image are replaced with the statistical values of the hair color values in the third region to obtain the first color guide image.
14. The hair transformation model generating device according to claim 10, It is characterized in that After the training module obtains the hair transformation model through training, The acquisition module is further configured to acquire a plurality of third sample person images; The acquisition module is further configured to acquire at least one preset color guide map; each of the preset color guide maps corresponds to a hair growth area; The second determining unit is further configured to calculate a statistical value of the color value of the hair in the third sample person image acquired by the acquisition module; The second determining unit is further configured to replace the color values of all pixels of hair in the preset color guide map acquired by the acquisition module with the statistical values of the color values of hair in the third sample person image to obtain a second color guide map; The acquisition module is further configured to input the third sample person image and the second color guide map obtained by the second determination unit into the hair transformation model trained by the training module to obtain a fourth sample person image; The training module is further configured to use the third sample character image and hairstyle variable obtained by the acquisition module as training data, and the fourth sample character image corresponding to the hairstyle variable as supervision information, to train and obtain a target hair transformation model; the hairstyle variable corresponds to the hair generation area one by one; The training module is further configured to update the hair transformation model to the target hair transformation model.
15. The hair transformation model generating device according to any one of claims 11 to 14, It is characterized in that The second determining unit is specifically configured to: After arranging the RGB values of all pixels of the hair color in ascending order according to the color and value, calculating the average value of the RGB values corresponding to the colors and values whose order is within a preset order range, and taking the average value of the RGB values corresponding to the colors and values whose order is within the preset order range as the statistical value of the hair color value; the color and value is the sum of the R value, the G value and the B value in the RGB value.
16. A hair transformation device, It is characterized in that include: An acquisition module, configured to acquire a source person image and hairstyle indication information; The hairstyle indication information is used to indicate a target hair growth area; A processing module, configured to input the source person image and the hairstyle indication information acquired by the acquisition module into a hair transformation model to obtain a target person image; the hair transformation model is obtained by the hair transformation model generation method according to any one of claims 1 to 7; the hairstyle of the person in the target person image is the hairstyle corresponding to the target hair generation area indicated by the hairstyle indication information, and the similarity between the hair color of the person image in the target person image and the hair color of the person image in the source person image is greater than a preset value; The hairstyle indication information includes a preset color guide map, and the preset color guide map includes the target hair generation area; the processing module is specifically configured to: Calculating the statistical value of the hair color value in the source person image acquired by the acquisition module; Replacing the color values of all pixels in the target hair generation area in the preset color guide map acquired by the acquisition module with the statistical values of the color values of the hair in the source person image to obtain a target color guide map; Inputting the source character image and the target color guide map acquired by the acquisition module into the hair transformation model to obtain the target character image; In the case where the hair transformation model is updated to a target hair transformation model obtained by the method for generating a hair transformation model according to claim 6, the hairstyle indication information includes a target hairstyle variable corresponding to a target hair generation area; and the processing module is specifically configured as follows: The source character image and the target hairstyle variable acquired by the acquisition module are input into the updated hair transformation model to obtain a target character image; the hairstyle of the character in the target character image corresponds to the target hair generation area.
17. An electronic device, It is characterized in that include: processor; a memory for storing instructions executable by the processor; The processor is configured to execute the instructions to implement the method according to any one of claims 1 to 8.
18. A computer storage medium having stored thereon instructions, It is characterized in that When the instructions in the computer storage medium are executed by a processor of an electronic device, the electronic device is enabled to perform the method as claimed in any one of claims 1 to 8.
19. A computer program product, It is characterized in that The method comprises instructions, which, when executed on a computer, cause the computer to execute the method according to any one of claims 1 to 8.
