Virtual hair dyeing method, device and equipment and storage medium

By generating a network model, virtual light dyeing and replacement treatment of hair areas is solved, and the problem of poor virtual hair dyeing effect in the prior art is improved, the accuracy and picture quality of hair dyeing are reduced, and storage pressure and deviation are reduced.

CN118918015APending Publication Date: 2024-11-08XIAMEN MEITUZHIJIA TECH
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202310502150.1
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2023-05-06
Publication Date
2024-11-08

AI Technical Summary

Technical Problem

The existing virtual hair dyeing methods cannot be automatically adjusted according to the user's hair color, resulting in a large deviation from the desired color, and there is a problem of storage pressure and missing hair details.

Method used

By generating a network model, the hair region is preprocessed, the hair region is extracted and virtual light dyeing prediction is performed, the color of the hair region is converted into a reference color, the replacement process is performed based on the reference color image, and the new pending image is dyed , reduce the number of LUT tables to reduce storage pressure and mapping processing speed.

Benefits of technology

It improves the accuracy and picture quality of virtual hair dyeing, reduces the deviation between the actual color results and the expected hair color, improves the user's visual experience, and effectively reduces the number of LUT tables, solving the problem of missing hair details.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118918015A_ABST
    Figure CN118918015A_ABST
Patent Text Reader

Abstract

The invention provides a virtual hair dyeing method and device, equipment and a storage medium, and the method comprises the steps: carrying out the image preprocessing of an input image or video, carrying out the light dyeing of a hair region image through employing a generative network model, and obtaining a hair region reference color image, the method comprises the following steps: firstly, obtaining a hair area reference color image, then fusing the hair area reference color image with an original to-be-processed image to obtain a lightly dyed image, and finally dyeing the lightly dyed image, thereby reducing the influence of the original hair color on a dyeing result, reducing the deviation between an actual coloring result and an expected hair color, improving the dyeing accuracy, and improving the impression experience of a user.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present application relates to the field of image processing technology, and in particular to a virtual hair dyeing method, device, equipment and storage medium. Background Art

[0002] With the continuous development and progress of beauty tools, more and more people are accustomed to using beauty tools to edit their makeup and hair style, and hairstyle and hair color are the key points in makeup editing. Virtual hair dyeing technology can help people quickly experience different hair colors, reduce the cost of trial colors, and greatly improve the user's viewing experience.

[0003] The existing virtual hair dyeing method is to perform dyeing adjustment on the picture, map the original hair color to the target hair color, and save this pixel mapping relationship as a color look up table (LUT) for batch hair dyeing.

[0004] However, existing virtual hair dyeing methods cannot automatically adjust according to the user's hair color, resulting in a large deviation between the coloring result and the expected hair color. In order to ensure the hair dyeing effect, it is usually necessary to use multiple LUT tables to dye different hair colors, and the LUT tables need to be compressed to reduce storage pressure and mapping processing speed. Therefore, there are still problems of storage pressure and lack of hair details. Summary of the invention

[0005] The purpose of the present application is to provide a virtual hair dyeing method, device, equipment and storage medium to address the deficiencies in the above-mentioned prior art, so as to solve the problem of poor hair dyeing effect in the prior art.

[0006] To achieve the above objectives, the technical solutions adopted in this application are as follows:

[0007] In a first aspect, the present application provides a virtual hair dyeing method, the method comprising:

[0008] Extracting a hair region to be processed from the image to be processed according to the hair mask, and performing image preprocessing on the hair region to obtain a hair region image;

[0009] Inputting the hair region image and the hair mask into a generative network model, performing virtual light dyeing prediction on the hair region image to convert the color of the hair region in the hair region image into a reference color, and obtaining a hair region reference color image;

[0010] Based on the hair region reference color image, performing hair region replacement processing on the image to be processed to obtain the new image to be processed;

[0011] The hair region of the new image to be processed is dyed to obtain a dyed image.

[0012] Optionally, extracting a hair region to be processed from the image to be processed according to the hair mask, and performing image pre-processing on the hair region to obtain a hair region image, includes:

[0013] Traversing the pixel values ​​in the hair mask to determine the minimum rectangular area containing the hair area;

[0014] Perform image pre-processing on the minimum rectangular area to obtain a hair area image.

[0015] Optionally, performing image pre-processing on the minimum rectangular area to obtain a hair area image includes:

[0016] Filling the edges of the minimum rectangular area to obtain an initial area image;

[0017] The initial region image is subjected to color space conversion and normalization processing to obtain a hair region image.

[0018] Optionally, the generative network model includes an encoder and a decoder, the decoder includes: a plurality of groups of sequentially connected feature processing modules and a plurality of groups of sequentially connected fusion convolution modules, wherein each group of feature processing modules includes a deconvolution layer and a decoding convolution layer, and each group of the fusion convolution modules includes a prediction convolution layer and an upsampling layer;

[0019] The step of inputting the hair region image and the hair mask into a generative network model, and performing virtual light dyeing prediction on the hair region image to convert the color of the hair region in the hair region image into a reference color to obtain a hair region reference color image includes:

[0020] Inputting the hair region image and the hair mask into the encoder for convolution processing to obtain a coding feature map, and performing downsampling processing on the coding feature map by the encoder to obtain target region features of the hair region image;

[0021] Input the target region features into the first group of feature processing modules, and each group of feature processing modules performs feature processing based on the input features in turn to obtain the region features output by each group of feature processing modules, and input each region feature into the corresponding fusion convolution module for prediction fusion processing and output fusion features;

[0022] The regional features output by the last set of feature processing modules are connected with the fused features output by the last set of fused convolution modules to obtain the hair region reference color image.

[0023] Optionally, the feature processing by each group of feature processing modules based on the input features includes:

[0024] The deconvolution layer in the feature processing module performs deconvolution processing on the input features, and outputs the deconvolution features to the decoding convolution layer;

[0025] The decoding convolution layer in the feature processing module performs decoding convolution processing on the deconvolution feature to obtain a decoding convolution feature, and the decoding convolution feature is input as a regional feature into the next feature processing module and the corresponding fusion convolution module respectively.

[0026] Optionally, the step of inputting the features of each region into a corresponding fusion convolution module for prediction fusion processing and outputting fusion features includes:

[0027] Inputting the regional features into the prediction convolution layer of the fusion convolution module for prediction processing to obtain prediction features;

[0028] Fusing the predicted features with the output result of the upsampling layer in the previous fusion convolution module to obtain the fusion result;

[0029] The fusion result is input into the upsampling layer of the fusion convolution module for upsampling processing to obtain and output the fusion feature.

[0030] Optionally, performing hair region replacement processing on the image to be processed based on the hair region reference color image to obtain the new image to be processed includes:

[0031] Performing data conversion on the hair region reference color image to obtain a converted hair region reference color image;

[0032] Performing hair smoothing processing on the converted hair region reference color image to obtain a middle region image;

[0033] Performing image fusion and image enhancement on the intermediate region image to obtain an enhanced region image;

[0034] The enhanced region image is pasted back into the image to be processed according to the hair mask to replace the hair region in the image to be processed, thereby obtaining the new image to be processed.

[0035] In a second aspect, the present application provides a virtual hair dyeing device, the device comprising:

[0036] A pre-processing module, used for extracting a hair region to be processed from the image to be processed according to the hair mask, and performing image pre-processing on the hair region to obtain a hair region image;

[0037] A prediction module, configured to input the hair region image and the hair mask into a generation network model, and perform virtual light dyeing prediction on the hair region image to convert the color of the hair region in the hair region image into a reference color, thereby obtaining a hair region reference color image;

[0038] A post-processing module, configured to perform hair region replacement processing on the image to be processed based on the hair region reference color image to obtain the new image to be processed;

[0039] The dyeing module is used to dye the hair area of ​​the new image to be processed to obtain a dyed image.

[0040] Optionally, the pre-processing module is specifically used for:

[0041] Traversing the pixel values ​​in the hair mask to determine the minimum rectangular area containing the hair area;

[0042] Perform image pre-processing on the minimum rectangular area to obtain a hair area image.

[0043] Optionally, the pre-processing module is further specifically used for:

[0044] Filling the edges of the minimum rectangular area to obtain an initial area image;

[0045] The initial region image is subjected to color space conversion and normalization processing to obtain a hair region image.

[0046] Optionally, the generative network model includes an encoder and a decoder, the decoder includes: a plurality of groups of sequentially connected feature processing modules and a plurality of groups of sequentially connected fusion convolution modules, wherein each group of feature processing modules includes a deconvolution layer and a decoding convolution layer, and each group of the fusion convolution modules includes a prediction convolution layer and an upsampling layer;

[0047] The prediction module is specifically used for:

[0048] Inputting the hair region image and the hair mask into the encoder for convolution processing to obtain a coding feature map, and performing downsampling processing on the coding feature map by the encoder to obtain target region features of the hair region image;

[0049] Input the target region features into the first group of feature processing modules, and each group of feature processing modules performs feature processing based on the input features in turn to obtain the region features output by each group of feature processing modules, and input each region feature into the corresponding fusion convolution module for prediction fusion processing and output fusion features;

[0050] The regional features output by the last set of feature processing modules are connected with the fused features output by the last set of fused convolution modules to obtain the hair region reference color image.

[0051] Optionally, the prediction module is further specifically used for:

[0052] The deconvolution layer in the feature processing module performs deconvolution processing on the input features, and outputs the deconvolution features to the decoding convolution layer;

[0053] The decoding convolution layer in the feature processing module performs decoding convolution processing on the deconvolution feature to obtain a decoding convolution feature, and the decoding convolution feature is input as a regional feature into the next feature processing module and the corresponding fusion convolution module respectively.

[0054] Optionally, the prediction module is further specifically used for:

[0055] Inputting the regional features into the prediction convolution layer of the fusion convolution module for prediction processing to obtain prediction features;

[0056] Fusing the predicted features with the output result of the upsampling layer in the previous fusion convolution module to obtain the fusion result;

[0057] The fusion result is input into the upsampling layer of the fusion convolution module for upsampling processing to obtain and output the fusion feature.

[0058] Optionally, the post-processing module is specifically used for:

[0059] Performing data conversion on the hair region reference color image to obtain a converted hair region reference color image;

[0060] Performing hair smoothing processing on the converted hair region reference color image to obtain a middle region image;

[0061] Performing image fusion and image enhancement on the intermediate region image to obtain an enhanced region image;

[0062] The enhanced region image is pasted back into the image to be processed according to the hair mask to replace the hair region in the image to be processed, thereby obtaining the new image to be processed.

[0063] In a third aspect, the present application provides an electronic device, comprising: a processor, a storage medium and a bus, wherein the storage medium stores machine-readable instructions executable by the processor, and when the electronic device is running, the processor communicates with the storage medium via the bus, and the processor executes the machine-readable instructions to perform the steps of the virtual hair dyeing method as described above.

[0064] In a fourth aspect, the present application provides a computer-readable storage medium having a computer program stored thereon, and when the computer program is executed by a processor, the steps of the virtual hair dyeing method as described above are executed.

[0065] The beneficial effects of the present application are as follows: by pre-processing the image of the hair area, the generative network model can be focused on the area that needs to be dyed, reducing the interference of the background area, reducing the difficulty of model learning, and thus obtaining a clearer hair area image. By generating a network model to lightly dye the hair area of ​​the image to be processed, when the original hair color is a special hair color, the influence of the original hair color in the image on the hair dyeing result can be reduced, and the deviation between the actual dyeing result and the expected hair color can be reduced. By replacing the image to be processed based on the hair area reference image and dyeing the hair area of ​​the new image to be processed, the quality of the image after hair dyeing can be improved, and the user's viewing experience can be improved. In addition, by generating a network model to lightly dye the hair area first, the number of LUT tables can be effectively reduced, thereby reducing storage pressure and mapping processing speed, and solving the problem of missing hair details. BRIEF DESCRIPTION OF THE DRAWINGS

[0066] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings required for use in the embodiments will be briefly introduced below. It should be understood that the following drawings only show certain embodiments of the present application and therefore should not be regarded as limiting the scope. For ordinary technicians in this field, other related drawings can be obtained based on these drawings without paying creative work.

[0067] Figure 1 A schematic diagram showing an application scenario provided by an embodiment of the present application is shown;

[0068] Figure 2 A flowchart of a virtual hair dyeing method provided in an embodiment of the present application is shown;

[0069] Figure 3 A flowchart of image pre-processing provided by an embodiment of the present application is shown;

[0070] Figure 4 A flowchart of another image pre-processing provided by an embodiment of the present application is shown;

[0071] Figure 5 A schematic diagram of a structure of a generated network model provided in an embodiment of the present application is shown;

[0072] Figure 6 A flowchart of generating a reference color image of a hair area provided by an embodiment of the present application is shown;

[0073] Figure 7A flowchart of internal processing of generating a network model provided by an embodiment of the present application is shown;

[0074] Figure 8 A flow chart for obtaining fusion features provided by an embodiment of the present application is shown;

[0075] Fig. 9 A schematic diagram of generating an intermediate output result of a network model provided in an embodiment of the present application is shown;

[0076] Fig.10 A flowchart of image replacement provided by an embodiment of the present application is shown;

[0077] Fig.11 A schematic structural diagram of a virtual hair dyeing device provided in an embodiment of the present application is shown;

[0078] Fig.12 A schematic structural diagram of an electronic device provided in an embodiment of the present application is shown. DETAILED DESCRIPTION

[0079] To make the purpose, technical scheme and advantages of the embodiments of the present application clearer, the technical scheme in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. It should be understood that the drawings in the present application only serve the purpose of explanation and description and are not used to limit the scope of protection of the present application. In addition, it should be understood that the schematic drawings are not drawn in real proportion. The flowchart used in this application shows the operations implemented according to some embodiments of the present application. It should be understood that the operations of the flowchart can be implemented out of order, and the steps without logical context can be reversed in order or implemented simultaneously. In addition, those skilled in the art can add one or more other operations to the flowchart under the guidance of the content of the present application, or remove one or more operations from the flowchart.

[0080] In addition, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments. The components of the embodiments of the present application described and shown in the drawings here can be arranged and designed in various configurations. Therefore, the following detailed description of the embodiments of the present application provided in the drawings is not intended to limit the scope of the application claimed for protection, but merely represents the selected embodiments of the present application. Based on the embodiments of the present application, all other embodiments obtained by those skilled in the art without making creative work belong to the scope of protection of the present application.

[0081] It should be noted that the term "comprising" will be used in the embodiments of the present application to indicate the existence of the features declared thereafter, but does not exclude the addition of other features.

[0082] Existing virtual dyeing methods usually perform dyeing adjustments on a picture, establish a mapping relationship between the hair color in the picture and the target hair color, then save this mapping relationship as a LUT table, and perform batch hair dyeing according to the LUT table.

[0083] However, the hair colors of users vary in depth. Therefore, for users with different hair colors, in order to obtain more accurate coloring results, it is necessary to re-establish the color mapping relationship through the LUT table. In order to save a large number of LUT tables, the LUT tables need to be compressed to reduce storage pressure and mapping processing speed. This method will lead to the problem of missing hair details. If the user's hair color is not adjusted accordingly, the coloring result will deviate greatly from the expected hair color.

[0084] It can be seen that how to improve the efficiency and accuracy of virtual hair dyeing has become an urgent problem to be solved.

[0085] Based on the above problems, the present application proposes a virtual hair dyeing method, which generates a network model to lightly dye the hair area in the portrait, and then dyes the lightly dyed hair. This can effectively solve the problem of a large deviation between the coloring result and the expected hair color caused by the different shades of hair, and improve the accuracy of virtual hair dyeing.

[0086] The virtual hair dyeing method of the present application can be applied to Figure 1 In the virtual hair dyeing scene shown, when the user uploads a picture, video, or performs real-time video recording, the electronic device can identify and lightly dye the hair area of ​​the user in the picture or video based on the method of the present application, and change the color of the hair area to the target hair color specified by the user.

[0087] Next, combine Figure 2 , the virtual hair dyeing method of the present application is further described, and the execution subject of the method can be an electronic device, such as Figure 2 As shown, the method includes:

[0088] S201: extracting a hair region to be processed from the image to be processed according to the hair mask, and performing image pre-processing on the hair region to obtain a hair region image.

[0089] Optionally, the hair mask can be obtained by the electronic device by segmenting the image to be processed through a front-end hair segmentation module.

[0090] Optionally, the image to be processed may be a picture or video uploaded by a user that needs to be dyed hair. It should be understood that a real-time video of the user may also be used as the image to be processed.

[0091] It should be noted that in order to improve the accuracy of the dyeing details of the hair edges, it is necessary to help the subsequent neural network model learn the edge transition of the hair. Therefore, the hair and the background information near the hair edges can be used as the hair area to be processed, and this part of the area can be pre-processed to obtain the hair area image.

[0092] Optionally, the image pre-processing can be edge filling, color space conversion and normalization of the hair area. Among them, by edge filling of the hair area, hair area images of different resolutions can be converted into images of the same specifications; the color space conversion step can convert the image to a unified color space, such as Lab color space for processing; the normalization step can convert the image pixel value to a specific range, thereby effectively accelerating the convergence of the model.

[0093] S202: Inputting the hair region image and the hair mask into the generative network model, performing virtual light dyeing prediction on the hair region image to convert the color of the hair region in the hair region image into a reference color, and obtaining a hair region reference color image.

[0094] In an embodiment of the present application, the hair area image and the hair mask may be stacked in the channel dimension, and the stacking result may be input into a generative network model to obtain a hair area reference color image.

[0095] Optionally, the reference color may be a light color in the Lab color space. The Lab value mainly includes three dimensions: the brightness of the object, the red-green color of the object, and the yellow-blue color of the object. For example, a color with a brightness value of 70-100 may be regarded as a light color.

[0096] Optionally, the generative network model may include an encoder and a decoder, the encoder being used to extract features from an input image, and the decoder being able to perform virtual dyeing prediction on the color of the hair area image based on the image features extracted by the encoder, so as to convert the color of the hair area in the hair area image into a reference color to obtain a hair area reference color image.

[0097] The hair region reference color image may refer to a light-colored image of the hair region image in the Lab color space.

[0098] S203: Based on the hair region reference color image, hair region replacement processing is performed on the image to be processed to obtain a new image to be processed.

[0099] Optionally, the hair region in the image to be processed may be replaced with the hair region reference color image obtained in step S202, thereby obtaining an image to be processed with a light-colored hair region, ie, a new image to be processed.

[0100] It should be noted that the hair region replacement process for the image to be processed may include the following steps: data conversion, hair smoothing, image fusion, image enhancement, and portrait pasting. Among them, data conversion may be transferring the output of the generative network model from the GPU to the CPU, hair smoothing, image fusion, and image enhancement may improve the authenticity of the generated hair region, and portrait pasting may be fused with the generated light-colored hair region and the original image to be processed to obtain the result of lightly dyed hair in the image to be processed.

[0101] In the embodiment of the present application, by generating a network model, the hair area of ​​the image to be processed is first lightly dyed. This can reduce the influence of the original hair color on the hair dyeing result when the hair color is darker, and reduce the deviation between the actual dyeing result and the expected hair color.

[0102] S204: Dyeing the hair region of the new image to be processed to obtain a dyed image.

[0103] Optionally, a LUT table may be used to dye the hair region of a new image to be processed to obtain a dyed image.

[0104] In the embodiment of the present application, by performing image pre-processing on the hair area, the generative network model can be concentrated on the area where hair dyeing is required, reducing the interference of the background area, reducing the difficulty of model learning, and thus obtaining a clearer hair area image. By generating a network model to lightly dye the hair area of ​​the image to be processed, the original hair color in the image can be reduced in the case where the original hair color is a special hair color. The influence of the original hair color on the hair dyeing result can be reduced, and the deviation between the actual coloring result and the expected hair color can be reduced. By replacing the image to be processed based on the hair area reference image and dyeing the hair area of ​​the new image to be processed, the quality of the image after hair dyeing can be improved, and the user's viewing experience can be improved. In addition, by generating a network model to lightly dye the hair area first, the number of LUT tables can be effectively reduced, thereby reducing storage pressure and mapping processing speed, solving the problem of missing hair details, and improving the effect of virtual hair dyeing.

[0105] Next, in the above step S201, the hair region to be processed is extracted from the image to be processed according to the hair mask, and the hair region is pre-processed to obtain the hair region image. Figure 3 As shown, the above step S201 includes:

[0106] S301: Traverse the pixel values ​​in the hair mask to determine the minimum rectangular area containing the hair area.

[0107] Optionally, by traversing each pixel value in the hair mask, the maximum and minimum values ​​in the height direction and the width direction of the hair region can be obtained, thereby obtaining a minimum rectangular region containing the entire hair region.

[0108] Furthermore, in order to speed up the hair region cropping, the hair region can be extracted by using a jump detection method. For example, traversing the entire hair mask matrix with a step length of 10 can reduce the number of pixels that need to be judged for hair region cropping by 100 times. At the same time, the length and width of the detected minimum rectangular region can be enlarged by 1.1 times to include the background information near the hair edge, thereby helping the subsequent neural network model to learn edge transitions.

[0109] For example, assuming that the value of an element that can be identified as hair is 127, each element of the hair mask can be traversed to determine whether the element value is greater than 127, and the minimum and maximum horizontal and vertical coordinates of all points whose element values ​​are greater than 127 are saved. If all values ​​in the hair mask are not greater than 127, it is considered that there is no hair data in the image, and the original image is directly returned as the dyeing result image. Otherwise, the center point, width and height are calculated. Then the width and height are expanded by 1.1 times to obtain the new width and height, and the new width and height and the center point are used for cropping to obtain the minimum rectangular area after expansion and the hair mask of the corresponding area.

[0110] S302: Perform image pre-processing on the minimum rectangular area to obtain a hair area image.

[0111] Optionally, after determining the minimum rectangular area, in order to improve the processing efficiency of generating the network model, image pre-processing may be performed on the minimum rectangular area to obtain a hair area image.

[0112] Among them, image pre-processing may include edge filling, color space conversion and normalization processing.

[0113] Next, combine Figure 4 , the above minimum rectangular area is subjected to image preprocessing to obtain the hair area image for further explanation, such as Figure 4 As shown, the above step S302 includes:

[0114] S401: Fill the edges of the minimum rectangular area to obtain an initial area image.

[0115] Optionally, edge filling refers to scaling the extracted hair rectangular area to a fixed size according to the long side, and performing numerical padding on both ends of the short side to make the long and short sides of the hair area the same length, thereby obtaining an initial area image. This ensures that even if images of different resolutions are input, the generated network model can still process them.

[0116] For example, when edge filling is performed, the hair rectangular area can be scaled in proportion to the long side to a resolution that can be processed by the generative network model, and then different initialization data matrices are created according to the category of the hair rectangular area. If the data is a hair rectangular area, a white image with RGB values ​​of all 255 is constructed, and if the data is a hair mask, a mask matrix with values ​​of all 0 is constructed. The scaled data is then copied to the created matrix and placed in the center of the matrix.

[0117] S402: Perform color space conversion and normalization processing on the initial region image to obtain a hair region image.

[0118] It should be noted that commonly used color spaces for images include RGB, Lab, and HSV. The color space conversion of this application refers to converting the image to the Lab color space for processing. Combined with previous virtual hair dyeing experience, an important reason why some dark hair dyeing effects are not good is that the original hair color brightness is relatively low. Even if the hair tone changes, but the brightness remains unchanged, the hair color will still be darker. Based on the above considerations, this application converts the image color to Lab format, so that the model can learn brightness transformation more easily.

[0119] Normalization is to convert the image pixel values ​​to a specific range in a specific way, which can effectively prevent gradient guarantee and accelerate the convergence of the model. For example, for the initial region image, each element can be divided by 255 and then subtracted from 255 to constrain it to between -0.5 and 0.5. For the hair mask, it can be divided by 255 to complete the normalization process.

[0120] In the embodiment of the present application, images of different resolutions can be converted into images of the same specifications through image pre-processing, thereby improving the universality of the generative network model in processing different images.

[0121] like Figure 5 As shown, it is a schematic diagram of the structure of a generation network model given in this application, referring to Figure 5 The generative network model of the present application includes an encoder 501 and a decoder 502, and the decoder 502 includes: multiple groups of feature processing modules 503 connected in sequence and multiple groups of fused convolution modules 504 connected in sequence, wherein each group of feature processing modules 503 includes a deconvolution layer and a decoding convolution layer, and each group of fused convolution modules 504 includes a prediction convolution layer and an upsampling layer.

[0122] Reference Figure 5 , skip connections can be used to pass the features extracted by the encoding convolutional layer of the encoder to the decoding convolutional layer of the decoder to improve the accuracy of the decoder's prediction.

[0123] Next, combine Figure 5 The generative network model structure of the hair region image and the hair mask are input into the generative network model, and the hair region image is predicted to be virtually lightly dyed to convert the color of the hair region in the hair region image into a reference color to obtain the hair region reference color image. Figure 6 As shown, the above step S202 includes:

[0124] S601: The hair region image and the hair mask are input into the encoder for convolution processing to obtain a coding feature map, and the encoder performs downsampling processing on the coding feature map to obtain target region features of the hair region image.

[0125] In this application, the hair area image and the hair mask can be stacked in the channel dimension first, and then the stacking result is used as the input of the encoder of the generative network model, and the encoder performs convolution processing.

[0126] Optionally, the encoder part can use different pre-trained models as feature extractors to speed up model training and improve the effect, such as mobilenet, resnet, etc. Take resnet18 as an example, refer to Figure 5 The encoder can include two encoding convolutional layers and four downsampling layers. The resolution of the input image can be any integer multiple of 16. In the encoder, the image first undergoes two convolutions and then four downsampling operations. The jump connection after the two convolutions will pass the processed feature map to the decoder, and the result of the decoder convolution processing will be directly added as the new input data.

[0127] Optionally, the target region feature may be a feature map of a hair region image obtained after encoder convolution and downsampling.

[0128] S602: Input the target area features into the first group of feature processing modules, and each group of feature processing modules performs feature processing based on the input features in turn to obtain the area features output by each group of feature processing modules, and input each area feature into the corresponding fusion convolution module for prediction fusion processing and output fusion features.

[0129] Reference Figure 5 The feature processing module includes a deconvolution layer and a decoding convolution layer, wherein the deconvolution layer can increase the size of the feature map, and the decoding convolution layer can perform feature processing on the feature map to obtain regional features, and input the regional features into the corresponding fusion convolution module.

[0130] Optionally, the fused convolution module may include a prediction convolution layer and an upsampling layer, wherein the prediction convolution layer can predict the output result of the regional features, and the upsampling layer can improve the resolution of the output result of the prediction convolution layer and fuse it with the output result of the prediction convolution layer of the next fused convolution module.

[0131] S603: Connect the regional features output by the last set of feature processing modules and the fused features output by the last set of fused convolution modules to obtain a hair region reference color image.

[0132] Reference Figure 5 , the regional features output by the last set of feature processing modules can be added to the fusion features output by the last set of fusion convolution modules to obtain the hair region reference color image output by the generative network model.

[0133] The following is a description of the steps of feature processing performed by each group of feature processing modules based on the input features. Figure 7 As shown, the above step S602 includes:

[0134] S701: The deconvolution layer in the feature processing module performs deconvolution processing on the input features and outputs the deconvolution features to the decoding convolution layer.

[0135] Optionally, the deconvolution layer can perform deconvolution on the input feature map to increase the size of the feature map.

[0136] For example, refer to Figure 5 Assuming that an input image with 4 channels and a resolution of 256×256 is subjected to two convolutions and four downsamplings by the encoder, a feature map with 512 channels and a resolution of 16×16 is obtained. Then, after deconvolution by the deconvolution layer 1 of the decoder, a feature map with 256 channels and a resolution of 32×32 is obtained, and the feature map is input into the decoding convolution layer for convolution processing.

[0137] S702: The decoding convolution layer in the feature processing module performs decoding convolution processing on the deconvolution feature to obtain a decoding convolution feature, and the decoding convolution feature is input as a regional feature into the next feature processing module and the corresponding fusion convolution module.

[0138] Optionally, the decoding convolution layer can perform convolution processing on the feature map output by the deconvolution layer to obtain a feature map of a corresponding size of the deconvolution layer, namely, a decoding convolution feature. The decoding convolution feature can characterize the hair color characteristics at different sizes.

[0139] Reference Figure 5, the decoded convolution features output by the decoding convolution layer 2 can be input into the deconvolution layer 3 and the prediction convolution layer 1 respectively for feature fusion and feature prediction of the next size.

[0140] Among them, the output of the predicted convolution layer is 3-channel image data of different scales, and the output result of the current scale is the predicted convolution layer output result * 0.5 + the upsampling layer output result * 0.5, thereby realizing the fusion of multi-scale information.

[0141] Furthermore, in the above step S702, the features of each region are input into the corresponding fusion convolution module for prediction, and the result of the previous scale is fused, such as Figure 8 As shown, including:

[0142] S801: Input the regional features into the prediction convolution layer of the fusion convolution module for prediction processing to obtain prediction features.

[0143] Reference Figure 5 , the prediction convolution layer 1 can perform prediction processing on the regional features of the decoding convolution layer input to obtain the prediction features.

[0144] It should be noted that Figure 5 The first fused convolution module in does not receive the fused features input from the previous fused convolution module, so the prediction convolution layer 1 can only predict the regional features output by the decoding convolution layer 1.

[0145] S802: Fuse the predicted features with the output result of the upsampling layer in the previous fusion convolution module to obtain a fusion result.

[0146] For example, Figure 5 The regional features output by the decoding convolution layer 2 are sent to the prediction convolution layer 2 to obtain a feature map with 3 channels and a size of 64*64, i.e., the prediction features. The prediction convolution layer 2 will fuse the output results of the upsampling layer 1 and the output results of the prediction convolution layer 2 at a ratio of 0.5, i.e., the fusion result is the output result of the prediction convolution layer 2 * 0.5 + the output result of the upsampling layer 1 * 0.5, and the fusion result is obtained, and the fusion result is sent to the upsampling layer 2 for processing.

[0147] It should be noted that this application only uses a ratio of 0.5 as an example to illustrate the fusion process of the output results of the prediction convolution layer and the upsampling layer. Those skilled in the art can also set the ratio of the output results of the prediction convolution layer and the upsampling layer according to actual conditions, and this application does not limit this.

[0148] S803: Input the fusion result into the upsampling layer of the fusion convolution module for upsampling processing to obtain and output fusion features.

[0149] Reference Figure 5 The upsampling layer 2 can upsample the fused features output by the prediction convolution layer 2 to improve the resolution of the output image, and continue to output the fused features after upsampling to the prediction convolution layer 3, which continues to perform feature fusion based on the output results of the upsampling layer 2 and the output results of the decoding convolution layer 3, and continues to upsample the fused results output by the prediction convolution layer 3.

[0150] like Fig. 9 As shown, it is another structural schematic diagram given in this application, in Fig. 9 The output of each layer of the input feature map in the generative network model is shown in Fig. 9 The input image has 4 channels and a size of 256×256. After the encoder performs two convolutions and four downsamplings, the obtained feature map is input into the decoder. The feature processing module and fusion convolution module of the decoder predict and fuse the input feature map, and finally obtain an output feature map with 3 channels and a size of 256×256.

[0151] Among them, the result of convolution of the feature map in the encoder can be passed to the decoder through jump connection to improve the accuracy of the decoder prediction.

[0152] In an embodiment of the present application, before using the generative network model to lightly dye the hair area of ​​the image to be processed, the generative network model can be trained first. The following is an explanation of the training process of the generative network model.

[0153] The initial generative network model may include a generator and a discriminator. During the training phase, the generator will continuously generate light-stained images of labeled samples. At the same time, the discriminator will obtain a discrimination result based on the light-stained images generated by the generator and the label information of the labeled samples, and continuously optimize the generator and the discriminator based on the discrimination results to improve the authenticity of the images generated by the generator.

[0154] The discriminator network structure can be composed of a pre-trained feature extractor and a fully connected network, which is used to predict the authenticity of the input image. In order to avoid the problem of mode collapse during training, the structure of the feature extractor in the discriminator network structure can be the same as the network structure of the encoder in the generator network.

[0155] The loss functions used in model generator training can include content loss, GAN loss, perceptual loss, total variation loss, and orientation loss. For real-time and video tasks, temporal consistency loss can also be used to perform frame stabilization training on the video.

[0156] When calculating content loss, we can first downsample the label samples by a factor of 2 multiple times to obtain label results of different resolutions, and then calculate the mean square error with the different resolution results generated by the model. The mean square error is used to measure the content difference between the generated results and the target results.

[0157] The GAN loss can use the softplus function to activate the output probability results of the discriminator network, thereby guiding the direction of model generation.

[0158] Perceptual loss is used to maintain the overall texture consistency of the hair in the generated image and the target image. By calculating the similarity of the feature maps of the target image and the generated image at different network layers, the model can keep the generated image and the target image more semantically similar.

[0159] The total variation loss measures the smoothness of the image by calculating the mean absolute error between adjacent pixels. The total variation loss can reduce the noise in the generated image and produce a smoother result.

[0160] Directional loss is mainly used to enhance the similarity between the generated hair color and the target hair color and improve the generation effect.

[0161] The temporal consistency loss uses the absolute error between pixels of two adjacent frames to measure the similarity of adjacent frames and minimizes the L1 loss between the predicted results and the true labels at multiple scales.

[0162] In addition, in order to protect the model's ability to make predictions based on neighboring pixels, only the minimum difference between pixels in adjacent frames can be penalized.

[0163] After determining the content loss, GAN loss, perceptual loss, total variation loss, and orientation loss, they can be added together according to their respective weight coefficients. If the labeled sample is a video, the temporal consistency loss also needs to be calculated.

[0164] After the training of the generated network model is completed based on the label samples, the hair area reference color image can be obtained through the above steps S201 and S202, and then the hair area is replaced on the image to be processed based on the hair area reference color image to obtain a new image to be processed, such as Fig.10 As shown, the above step S203 includes:

[0165] S1001: performing data conversion on the hair region reference color image to obtain a converted hair region reference color image.

[0166] Optionally, data conversion includes two aspects: denormalization and color space conversion. Denormalization converts the model output data into image data for subsequent processing, and converts the image data pixel value to 0 to 255 by adding 0.5 to each element value of the hair area reference color image and multiplying by 255. Color space conversion converts the image data in Lab color space into RGB color space.

[0167] For example, each element in the hair region reference color image may be multiplied by 255 and then added with 127, and then converted from the LAB color space to the RGB color space using a color conversion tool such as opencv.

[0168] S1002: Perform hair smoothing processing on the converted hair region reference color image to obtain a middle region image.

[0169] Optionally, the hair smoothing process may be to first blur the surface of the image, and then fuse the blurred image with the generated image of the generative network model according to the hair mask, so as to obtain a result with smoother hair.

[0170] For example, each element in the hair mask can be multiplied by the soft hair ratio to obtain a processed hair mask. Then, the hair region reference color image is surface blurred, and the final result image is equal to the result of the surface blur image multiplied by the processed hair mask plus the hair region reference color image multiplied by 1 minus the processed hair mask.

[0171] S1003: performing image fusion and image enhancement on the middle area image to obtain an enhanced area image.

[0172] Optionally, image fusion refers to using the hair mask to integrate the brightness information of the original image into the generated hair, so as to ensure that the hair edge transition is natural enough. Morphological dilation and multiple Gaussian filtering of the hair mask can make the mask transition smoother, and then the processed mask is used to fuse the brightness information of the original image into the generated image.

[0173] Optionally, image enhancement mainly uses high pass and soft light processing to sharpen the image and enhance the details of the generated hair.

[0174] High contrast first performs Gaussian filtering on the original image to obtain a blurred image, then adds 128 pixels to the blurred image and subtracts it from the original image to obtain a high contrast retained result image containing image edge information.

[0175] Soft light processing is to integrate the extracted high contrast retention results into the generated hair, and darken or brighten the color channel value of the base layer according to the source color. For each color channel value, if the source color is brighter than 50% gray, the result color is brighter than the base color. If the source color is darker than 50% gray, the result color is darker than the base color.

[0176] Exemplarily, the undyed hair input image can first be Gaussian filtered to obtain a blurred image, then the input image plus 127 minus the blurred image can be used to obtain a high-contrast retained result image, and then the high-contrast retained result image and the dyed hair result image can be softly superimposed to obtain an enhanced result.

[0177] S1004: Paste the enhanced region image back into the image to be processed according to the hair mask to replace the hair region in the image to be processed, thereby obtaining a new image to be processed.

[0178] Optionally, the enhanced area image is pasted back to the image to be processed, that is, the processed hair area head portrait is pasted back to the original image to ensure that other information in the non-head portrait area does not change. First, the filling area is removed using the edge filling parameters in the pre-processing, and then the image is restored to the size before scaling, and finally the hair area is pasted back to the original image through the hair mask to obtain the final generated result. At the same time, in order to further reduce the unnatural phenomenon caused by fusion, image fusion can be used again after pasting back.

[0179] In the embodiment of the present application, by replacing and fusing the light-colored hair area generated by the generative network model with the original image to be processed, the authenticity of the finally generated light-dyed image can be further improved, thereby enhancing the user's viewing experience.

[0180] Based on the same inventive concept, a virtual hair dyeing device corresponding to the virtual hair dyeing method is also provided in the embodiment of the present application. Since the principle of solving the problem by the device in the embodiment of the present application is similar to the above-mentioned virtual hair dyeing method in the embodiment of the present application, the implementation of the device can refer to the implementation of the method, and the repeated parts will not be repeated.

[0181] Reference Fig.11 , which is a schematic diagram of a virtual hair dyeing device provided in an embodiment of the present application, the device comprises: a pre-processing module 1101, a prediction module 1102, a post-processing module 1103 and a dyeing module 1104, wherein:

[0182] The pre-processing module 1101 is used to extract the hair region to be processed from the image to be processed according to the hair mask, and perform image pre-processing on the hair region to obtain a hair region image;

[0183] The prediction module 1102 is used to input the hair region image and the hair mask into the generation network model, perform virtual light dyeing prediction on the hair region image, so as to convert the color of the hair region in the hair region image into a reference color, and obtain a hair region reference color image;

[0184] A post-processing module 1103 is used to perform hair region replacement processing on the image to be processed based on the hair region reference color image to obtain a new image to be processed;

[0185] The dyeing module 1104 is used to dye the hair area of ​​the new image to be processed to obtain a dyed image.

[0186] Optionally, the pre-processing module 1101 is specifically used for:

[0187] Traverse the pixel values ​​in the hair mask to determine the minimum rectangular area containing the hair area;

[0188] Perform image preprocessing on the minimum rectangular area to obtain the hair area image.

[0189] Optionally, the pre-processing module 1101 is further specifically used for:

[0190] Fill the edge of the minimum rectangular area to obtain the initial area image;

[0191] The initial region image is subjected to color space conversion and normalization processing to obtain a hair region image.

[0192] Optionally, the generated network model includes an encoder and a decoder, the decoder includes: a plurality of groups of sequentially connected feature processing modules and a plurality of groups of sequentially connected fusion convolution modules, wherein each group of feature processing modules includes a deconvolution layer and a decoding convolution layer, and each group of fusion convolution modules includes a prediction convolution layer and an upsampling layer;

[0193] The prediction module 1102 is specifically used for:

[0194] The hair region image and the hair mask are input into the encoder for convolution processing to obtain a coding feature map, and the encoder performs downsampling processing on the coding feature map to obtain the target region features of the hair region image;

[0195] Input the target region features into the first group of feature processing modules, and each group of feature processing modules performs feature processing based on the input features in turn to obtain the region features output by each group of feature processing modules, and input each region feature into the corresponding fusion convolution module for prediction fusion processing and output fusion features;

[0196] The regional features output by the last set of feature processing modules are connected with the fused features output by the last set of fusion convolution modules to obtain a reference color image of the hair region.

[0197] Optionally, the prediction module 1102 is further specifically configured to:

[0198] The deconvolution layer in the feature processing module performs deconvolution on the input features and outputs the deconvolution features to the decoding convolution layer;

[0199] The decoding convolution layer in the feature processing module performs decoding convolution processing on the deconvolution features to obtain decoding convolution features, and the decoding convolution features are input into the next feature processing module and the corresponding fusion convolution module as regional features.

[0200] Optionally, the prediction module 1102 is further specifically configured to:

[0201] The regional features are input into the prediction convolution layer of the fusion convolution module for prediction processing to obtain prediction features;

[0202] Fuse the predicted features with the output result of the upsampling layer in the previous fusion convolution module to obtain a fusion result;

[0203] The fusion result is input into the upsampling layer of the fusion convolution module for upsampling processing to obtain and output the fusion feature.

[0204] Optionally, the post-processing module 1103 is specifically used for:

[0205] Performing data conversion on the hair region reference color image to obtain a converted hair region reference color image;

[0206] Performing hair smoothing processing on the converted hair region reference color image to obtain an intermediate region image;

[0207] Perform image fusion and image enhancement on the middle area image to obtain an enhanced area image;

[0208] The enhanced region image is pasted back into the image to be processed according to the hair mask to replace the hair region in the image to be processed, thereby obtaining a new image to be processed.

[0209] For descriptions of the processing flow of each module in the device and the interaction flow between each module, reference may be made to the relevant descriptions in the above method embodiment, which will not be described in detail here.

[0210] The embodiment of the present application performs image pre-processing on the hair area, so that the generative network model can focus on the area that needs hair dyeing, reduce the interference of the background area, and reduce the difficulty of model learning, thereby obtaining a clearer hair area image. By generating a network model to lightly dye the hair area of ​​the image to be processed, the original hair color in the image can be reduced in the case where the original hair color is a special hair color. The deviation between the actual dyeing result and the expected hair color can be reduced. By replacing the image to be processed based on the hair area reference image and dyeing the hair area of ​​the new image to be processed, the quality of the image after hair dyeing can be improved, and the user's viewing experience can be improved.

[0211] The present application also provides an electronic device, such as Fig.12 1 is a schematic diagram of the structure of an electronic device provided in an embodiment of the present application, including: a processor 1201, a memory 1202 and a bus. The memory 1202 stores machine-readable instructions executable by the processor 1201 (for example, Fig.11 In the device, the pre-processing module 1101, the prediction module 1102, the post-processing module 1103 and the execution instructions corresponding to the dyeing module 1104, etc., when the computer device is running, the processor 1201 communicates with the memory 1202 through the bus, and when the machine-readable instructions are executed by the processor 1201, the processing of the above-mentioned virtual hair dyeing method is performed.

[0212] An embodiment of the present application further provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the steps of the above-mentioned virtual hair dyeing method are executed.

[0213] Those skilled in the art can clearly understand that, for the convenience and simplicity of description, the specific working process of the system and device described above can refer to the corresponding process in the method embodiment, and will not be repeated in this application. In the several embodiments provided in this application, it should be understood that the disclosed system, device and method can be implemented in other ways. The device embodiments described above are merely schematic. For example, the division of the modules is only a logical function division. There may be other division methods in actual implementation. For example, multiple modules or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some communication interfaces, indirect coupling or communication connection of devices or modules, which can be electrical, mechanical or other forms.

[0214] In addition, each functional unit in each embodiment of the present application can be integrated into a processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. If the function is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention is essentially or the part that contributes to the prior art or the part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions to enable a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, RandomAccess Memory), disk or optical disk and other media that can store program code.

[0215] The above are only specific implementation methods of the present application, but the protection scope of the present application is not limited thereto. Any technician familiar with the technical field can easily think of changes or substitutions within the technical scope disclosed in the present application, which should be covered by the protection scope of the present application.

Claims

1. A virtual hair dyeing method, It is characterized in that include: Extracting a hair region to be processed from the image to be processed according to the hair mask, and performing image preprocessing on the hair region to obtain a hair region image; Inputting the hair region image and the hair mask into a generative network model, performing virtual light dyeing prediction on the hair region image to convert the color of the hair region in the hair region image into a reference color, and obtaining a hair region reference color image; Based on the hair region reference color image, performing hair region replacement processing on the image to be processed to obtain a new image to be processed; The hair region of the new image to be processed is dyed to obtain a dyed image.

2. The method according to claim 1, It is characterized in that The step of extracting a hair region to be processed from the image to be processed according to the hair mask and performing image pre-processing on the hair region to obtain a hair region image comprises: Traversing the pixel values ​​in the hair mask to determine the minimum rectangular area containing the hair area; Perform image pre-processing on the minimum rectangular area to obtain a hair area image.

3. The method according to claim 2, It is characterized in that The performing image pre-processing on the minimum rectangular area to obtain a hair area image includes: Filling the edges of the minimum rectangular area to obtain an initial area image; The initial region image is subjected to color space conversion and normalization processing to obtain a hair region image.

4. The method according to claim 1, It is characterized in that The generative network model includes an encoder and a decoder, wherein the decoder includes: a plurality of groups of sequentially connected feature processing modules and a plurality of groups of sequentially connected fusion convolution modules, wherein each group of feature processing modules includes a deconvolution layer and a decoding convolution layer, and each group of the fusion convolution modules includes a prediction convolution layer and an upsampling layer; The step of inputting the hair region image and the hair mask into a generative network model, and performing virtual light dyeing prediction on the hair region image to convert the color of the hair region in the hair region image into a reference color to obtain a hair region reference color image includes: Inputting the hair region image and the hair mask into the encoder for convolution processing to obtain a coding feature map, and performing downsampling processing on the coding feature map by the encoder to obtain target region features of the hair region image; Input the target region features into the first group of feature processing modules, and each group of feature processing modules performs feature processing based on the input features in turn to obtain the region features output by each group of feature processing modules, and input each region feature into the corresponding fusion convolution module for prediction fusion processing and output fusion features; The regional features output by the last set of feature processing modules are connected with the fused features output by the last set of fused convolution modules to obtain the hair region reference color image.

5. The method according to claim 4, It is characterized in that The feature processing modules of each group perform feature processing based on the input features, including: The deconvolution layer in the feature processing module performs deconvolution processing on the input features, and outputs the deconvolution features to the decoding convolution layer; The decoding convolution layer in the feature processing module performs decoding convolution processing on the deconvolution feature to obtain a decoding convolution feature, and the decoding convolution feature is input as a regional feature into the next feature processing module and the corresponding fusion convolution module respectively.

6. The method according to claim 4, It is characterized in that The step of inputting the features of each region into the corresponding fusion convolution module for prediction fusion processing and outputting fusion features includes: Inputting the regional features into the prediction convolution layer of the fusion convolution module for prediction processing to obtain prediction features; Fusing the predicted features with the output result of the upsampling layer in the previous fusion convolution module to obtain a fusion result; The fusion result is input into the upsampling layer of the fusion convolution module for upsampling processing to obtain and output the fusion feature.

7. The method according to claim 1, It is characterized in that The step of performing hair region replacement processing on the image to be processed based on the hair region reference color image to obtain the new image to be processed includes: Performing data conversion on the hair region reference color image to obtain a converted hair region reference color image; Performing hair smoothing processing on the converted hair region reference color image to obtain a middle region image; Performing image fusion and image enhancement on the intermediate region image to obtain an enhanced region image; The enhanced region image is pasted back into the image to be processed according to the hair mask to replace the hair region in the image to be processed, thereby obtaining the new image to be processed.

8. A virtual hair dyeing device, It is characterized in that include: A pre-processing module, used for extracting a hair region to be processed from the image to be processed according to the hair mask, and performing image pre-processing on the hair region to obtain a hair region image; A prediction module, configured to input the hair region image and the hair mask into a generation network model, and perform virtual light dyeing prediction on the hair region image to convert the color of the hair region in the hair region image into a reference color, thereby obtaining a hair region reference color image; A post-processing module, configured to perform hair region replacement processing on the image to be processed based on the hair region reference color image to obtain a new image to be processed; The dyeing module is used to dye the hair area of ​​the new image to be processed to obtain a dyed image.

9. An electronic device, It is characterized in that include: A processor, a storage medium and a bus, wherein the storage medium stores program instructions executable by the processor. When the electronic device is running, the processor communicates with the storage medium via the bus, and the processor executes the program instructions to perform the steps of the virtual hair dyeing method as described in any one of claims 1 to 7.

10. A computer-readable storage medium, It is characterized in that The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the virtual hair dyeing method according to any one of claims 1 to 7 are executed.