A hairstyle conversion method, device and storage medium

Abstract

The application discloses a hairstyle conversion method and device and a storage medium, which comprises the following steps: obtaining a source image to be converted and a target hairstyle image, performing alignment processing to obtain a target hairstyle alignment image; inputting the source image to be converted into a first encoder to obtain a first feature vector and a first feature map; inputting the first feature map into a neural network after first hair segmentation to perform image inpainting, and obtaining a third feature vector and a third feature map; inputting the target hairstyle alignment image into the first encoder to obtain a second feature vector and a second feature map; performing second hair segmentation on the second feature map, compressing the second hair segmentation result, and recording the compression result as M; performing hairstyle fusion based on the second feature map, the third feature map and M to obtain a first fusion image; and performing secondary fusion on the first fusion image to obtain a second fusion image. Through the hairstyle conversion method, the hairstyle conversion effect can be improved, and a natural and high-definition hairstyle conversion result can be obtained.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of image processing, in particular to a hairstyle conversion method, a hairstyle conversion device applying the method and a computer readable storage medium. BACKGROUND

[0002] Generally, people will try to change their image and temperament by hairdressing. However, a large part of them hope to see the overall effect of hairdressing before hairdressing, as a reference for the effect before trying hairdressing.

[0003] Or, people also want to try different hairstyles in a short time through simple operation for the purpose of novelty experience. Therefore, some hairstyle (or hairstyle transfer) applications based on image processing technology have emerged. The hairstyle conversion (or transfer) task refers to modifying the hairstyle of the face in the source image to the hairstyle of the face in the target image.

[0004] However, the current hairstyle conversion (or transfer) task has poor implementation effect and unnatural hairstyle conversion effect. For example, other facial attributes except hair will also change, the resolution of the converted image is not high, and the hair part is relatively blurred. SUMMARY

[0005] The main purpose of the present application is to provide a hairstyle conversion method, device and storage medium, which aims to solve the technical problem of poor and unnatural hairstyle conversion effect of the existing hairstyle conversion method.

[0006] To achieve the above purpose, the present application provides a hairstyle conversion method, which includes the following steps: obtaining a source image to be converted and a target hairstyle image, performing alignment processing to obtain a target hairstyle alignment image; inputting the source image to be converted into a first encoder to obtain a first feature vector and a first feature map; performing first hair segmentation on the first feature map and inputting the first feature map into a neural network for image inpainting to obtain a third feature vector and a third feature map; inputting the target hairstyle alignment image into the first encoder to obtain a second feature vector and a second feature map; performing second hair segmentation on the second feature map, compressing the second hair segmentation result, and recording the compression result as M; based on the second feature map, the third feature map and M, performing hairstyle fusion to obtain a first fusion image; performing secondary fusion on the first fusion image to obtain a second fusion image.

[0007] Preferably, the alignment processing further comprises: respectively performing key point detection on the to-be-converted source image and the target hairstyle image to obtain key points of the to-be-converted source image and key points of the target hairstyle image; inputting the target hairstyle image and the key points of the target hairstyle image into a second encoder of the alignment network to perform feature extraction to obtain corresponding high-dimensional features; and inputting the high-dimensional features and the key points of the to-be-converted source image into a generator of the alignment network to generate a target hairstyle alignment image.

[0008] Preferably, the alignment network is trained by a K-hairstyle dataset, and the alignment network adopts a u-net structure.

[0009] Preferably, the first encoder is obtained by pre-training, specifically, a plurality of face images are generated by a Stylegan generator, and a plurality of vectors corresponding to the plurality of face images are saved for training the first encoder.

[0010] Preferably, after the first hair segmentation is performed on the first feature map, the first feature map is input into a neural network for image inpainting to obtain a third feature vector and a third feature map, and the method specifically comprises the following steps: performing first hair segmentation on the first feature map to obtain a source image hairstyle mask to determine a to-be-repaired region; obtaining a direction vector of edited hair length based on the face image, and processing the first feature map based on the direction vector to obtain a guide image for guiding image inpainting; and inputting the first feature map into the neural network, and the neural network performs image inpainting on the first feature map after the first hair segmentation based on the source image hairstyle mask and the guide image.

[0011] Preferably, the direction vector of the edited hair length is obtained based on the face image, and the method specifically comprises the following steps: obtaining a preset long hair threshold and a preset short hair threshold, and classifying the face image by a face attribute discriminator to obtain a picture dataset with a hair length label; training a corresponding feature vector based on the hair length label to obtain an svm classification hyperplane; and obtaining a normal vector of the svm classification hyperplane to obtain the direction vector of the edited hair length.

[0012] Preferably, the second hair segmentation is performed on the second feature map and the second feature map is compressed, and the compression result is denoted as M, and the method specifically comprises the following steps: performing second hair segmentation on the second feature map to obtain a target hairstyle mask; and compressing the target hairstyle mask to the scale of the feature space F, and denoting the compression result as M.

[0013] Preferably, based on the second feature map, the third feature map and M, hairstyle fusion is performed to obtain a first fusion image, and the specific formula is as follows: F merge =M*F align +(1-M)*F short ; wherein F merge represents the first fused high-dimensional feature, which has spatial significance, and F alignF high-dimensional features representing the second feature map, F short F high-dimensional features representing the third feature map; the second fusion is based on a fusion mask of the source hairstyle mask and the target hairstyle mask, the first fusion map and the source image to be converted;

[0014] Corresponding to the hairstyle conversion method, the present application provides a hairstyle conversion device, comprising: an image acquisition module for acquiring a source image to be converted and a target hairstyle image; an alignment processing module for aligning the source image to be converted and the target hairstyle image to obtain a target hairstyle alignment image; a first encoder, the source image to be converted is input into the first encoder to obtain a first feature vector and a first feature map; the target hairstyle alignment image is input into the first encoder to obtain a second feature vector and a second feature map; an image repair module for inputting the first feature map after first hair segmentation into a neural network for image repair to obtain a third feature vector and a third feature map; a hair segmentation module for first hair segmentation and second hair segmentation of the second feature map; a compression module for compressing the second hair segmentation result of the second feature map, and recording the compression result as M; a first fusion module for hairstyle fusion based on the second feature map, the third feature map and M to obtain a first fusion map; a second fusion module for second fusion of the first fusion map to obtain a second fusion map.

[0015] In addition, in order to achieve the above-mentioned purpose, the present application also provides a computer readable storage medium, wherein the computer readable storage medium stores an image processing program, and the image processing program is executed by a processor to realize the steps of the hairstyle conversion method as described above.

[0016] The beneficial effects of the present application are:

[0017] (1) Through alignment processing, the hidden space features of the generation network can be modified according to the posture of the source image to be transferred, and the face angle of the source image to be transferred is automatically matched to obtain a hairstyle with a suitable angle; through the first fusion (i.e. hairstyle fusion), the target hairstyle can be adjusted by referring to the hair color, environmental light and other parameters of the source image to be transferred; through the second fusion, the fusion effect can be improved to obtain a natural and high-definition hairstyle conversion result;

[0018] (2) The alignment network is trained by using the K-hairstyle dataset, so that the target output image has a pixel-level label, and the u-net structure is used to fully utilize the target face posture;

[0019] (3) The image is repaired by guiding the image-guided neural network, which can improve the image repair effect; and the source image hairstyle mask is combined to repair the first feature map, and non-hair is generated in the source image hairstyle mask area, so that the repair result is more realistic, and at the same time, the area of the source image to be converted except the hair is not changed, which improves the image repair effect;

[0020] (4) By compressing the target hairstyle mask to the scale of the feature space F, the F feature level fusion will not appear unnatural transition edge after editing, and at the same time, the consistency of the irrelevant area and the original picture can be preserved, so that the hairstyle conversion effect is more natural;

[0021] (5) Based on the fusion mask of the source image hairstyle mask and the target hairstyle mask, the first fusion image and the third feature image are fused twice, which can avoid the change of the face and body and other non-hair areas due to the hairstyle conversion, and improve the hairstyle conversion effect. BRIEF DESCRIPTION OF DRAWINGS

[0022] The accompanying drawings, which are included to provide a further understanding of the application and are incorporated in and constitute a part of this application, illustrate embodiments of the application and together with the description serve to explain the application. In the drawings:

[0023] Figure 1 It is a flowchart of the hairstyle conversion method of the present application;

[0024] Figure 2 It is a schematic diagram of an embodiment of the hairstyle conversion method of the present application. DETAILED DESCRIPTION

[0025] In order to make the purpose, technical scheme and advantages of the embodiments of the present application clearer, the technical scheme of the embodiments of the present application will be described clearly and completely below with reference to the drawings of the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, not all the embodiments of the present application. It should be understood that the specific embodiments described herein are only used to explain the present application, and are not used to limit the present application. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative labor fall within the scope of the present application.

[0026] In the description of the present application, the terms "first", "second", etc. are used only for descriptive purposes, and cannot be understood as indicating or implying relative importance or implicitly indicating the number of the indicated technical features. Therefore, the features limited by "first", "second" can explicitly or implicitly include one or more features. The content of the present application will be described in detail below in combination with the embodiments.

[0027] At present, there are many hairstyle conversion research works, most of which are based on Stylegan in the generative adversarial network, because Stylegan largely guarantees the semantic quality of the generated face. For example, MiChiGan refers to Stylegan, designs a conditional generation network, and uses a guide picture to control the attributes of the generated hairstyle, and LOHO is obtained by iteratively optimizing the hidden vector of Stylegan to obtain a relatively natural result, but they are only applicable to the case where the posture is relatively consistent, and the supported picture resolution is relatively low.

[0028] In addition, there are face swapping algorithms such as SimSwap, which can be considered as reverse hairstyle conversion compared with hairstyle conversion, but it has a relatively large change to the target face. However, due to the limitations of the prior art, hairstyle conversion is faced with the pain point of poor conversion effect.

[0029] In order to improve the hairstyle conversion effect and obtain a natural and high-definition hairstyle conversion result, the present application provides a hairstyle conversion method, device and storage medium. As shown in Figure 1 The hairstyle conversion method of the present application comprises the following steps: obtaining a to-be-converted source image and a target hairstyle image, performing alignment processing to obtain a target hairstyle alignment image; inputting the to-be-converted source image into a first encoder to obtain a first feature vector and a first feature map; performing first hair segmentation on the first feature map and inputting the first feature map into a neural network for image inpainting to obtain a third feature vector and a third feature map; inputting the target hairstyle alignment image into the first encoder to obtain a second feature vector and a second feature map; performing second hair segmentation on the second feature map, compressing the second hair segmentation result, and recording the compression result as M; based on the second feature map, the third feature map and M, performing hairstyle fusion (also referred to as first fusion) to obtain a first fusion image; and performing secondary fusion on the first fusion image to obtain a second fusion image.

[0030] The present application can modify the hidden space features of the generation network according to the posture of the to-be-transferred source image through alignment processing, automatically match the face angle of the to-be-transferred source image, and obtain a hairstyle with a suitable angle; through first fusion, the target hairstyle can be adjusted by referring to the hair color, environmental light and other parameters of the to-be-transferred source image; through secondary fusion, the fusion effect can be improved, and a natural and high-definition hairstyle conversion result can be obtained.

[0031] For the convenience of understanding, the present application will be described below in conjunction with Figure 2 (an embodiment schematic diagram of the hairstyle conversion method of the present application) of the present application. As shown in Figure 2As shown, a1 is a source image to be converted, which can be input by a user; b1 is a target hairstyle image, which can be input by a user, selected by a user or randomly selected by a system; a2 is a first feature map obtained according to the source image to be converted, which has a corresponding first feature vector; b2 is a target hairstyle alignment image obtained according to the target hairstyle image; c is a second feature map obtained according to the target hairstyle alignment image, which has a corresponding second feature vector; a3 is a third feature map obtained based on the first feature map; d1 is a first fusion image; d2 is a second fusion image; m1 is a source image hairstyle mask obtained based on first hair segmentation; m2 is a target hairstyle mask obtained based on second hair segmentation; and m3 is a fusion mask obtained based on the source image hairstyle mask m1 and the target hairstyle mask m2.

[0032] In the embodiment, the alignment network is divided into an encoder (i.e., a second encoder in the following) and a generator. The alignment process further includes: performing key point detection on the source image to be converted a1 and the target hairstyle image b1 respectively to obtain key points of the source image to be converted a1 and key points of the target hairstyle image b1; inputting the target hairstyle image b1 and the key points of the target hairstyle image b1 into the second encoder of the alignment network to perform feature extraction to obtain corresponding high-dimensional features; and inputting the high-dimensional features and the key points of the source image to be converted a1 into the generator of the alignment network to generate the target hairstyle alignment image b2.

[0033] Preferably, the alignment network is trained by a K-hairstyle dataset, and the alignment network adopts a u-net structure.

[0034] The alignment network is trained by a K-hairstyle dataset (a same-person same-hairstyle multi-angle dataset) in the application, so that the target output image has a pixel-level label, and a u-net structure is adopted to fully utilize a target face posture.

[0035] In the embodiment, the first encoder is obtained by pre-training, specifically, a plurality of face images are generated by a Stylegan generator, and a plurality of vectors corresponding to the plurality of face images are saved for training the first encoder.

[0036] The input of the Stylegan generator is a random vector, and the output is a face image. The application can ensure that the generated face image has high quality, which is very fundamental and important for a generated image. Preferably, in the embodiment, 20,000 face images are generated by the Stylegan generator, and 20,000 corresponding vectors are saved for training the first encoder; the encoder can reversely convert a picture into a corresponding high-dimensional vector, which can also be called a feature vector of the picture.

[0037] In the embodiment, the first feature map a2 is input into the neural network for image inpainting after the first hair segmentation, to obtain a third feature vector and a third feature map a3, which specifically includes the following steps: performing the first hair segmentation on the first feature map a2 to obtain a source image hairstyle mask m1, to determine the to-be-repaired region; obtaining a direction vector of edited hair length based on the face picture, and processing the first feature map a2 based on the direction vector to obtain a guide image for guiding image inpainting; inputting the first feature map a2 into the neural network, and the neural network performs image inpainting based on the source image hairstyle mask m1 and the guide image.

[0038] In the embodiment, the processing of the first feature map a2 based on the direction vector specifically includes: performing addition or subtraction on the first feature vector corresponding to the first feature map a2 through the direction vector, so that the hairstyle in the first feature map a2 is shortened.

[0039] It can be understood that the length of the hairstyle of the first feature map a2 obtained after the processing of the direction vector can be limited by setting a preset hair length threshold, so that no matter whether the hairstyle of the to-be-converted source image a1 is long hair or short hair, the hairstyle length meeting the application standard can be obtained through the processing of the direction vector.

[0040] In the embodiment, the direction vector of edited hair length is obtained based on the face picture, which specifically includes the following steps: obtaining a preset long hair threshold and a preset short hair threshold, and classifying the face picture through the face attribute discriminator to obtain a picture data set with a hair length label; training the corresponding feature vector based on the hair length label to obtain an svm classification hyperplane; obtaining the direction vector of edited hair length by taking the normal vector of the svm classification hyperplane.

[0041] Preferably, in the embodiment, 40,000 generated images of the Stylegan generator are randomly sampled, the two-class discriminator of long hair and short hair is used to obtain the labels and confidence scores of long hair and short hair, and the top 50% of long hair pictures and the top 50% of short hair pictures are obtained according to the specific confidence scores, and the corresponding threshold is set according to the specific confidence score, at this time, the picture data set with the hair length label is obtained, each of which is 10,000 (and the corresponding feature).

[0042] The face attribute discriminator can use an existing third-party tool, which can score the specified attributes of the input picture, such as: the confidence score of the hair being curly in the picture, the confidence score of the skin being yellow, etc., which is used to distinguish the length of the hair in the present application.

[0043] The application can make the hairstyle in the first feature map a2 shorter by editing the direction vector of the hair length, generate a guide image, guide the neural network to perform image inpainting through the guide image, and improve the image inpainting effect; and then combine the source image hairstyle mask m1 to perform image inpainting on the first feature map a2, generate non-hair in the source image hairstyle mask m1 area, make the inpainting result more real, and at the same time, do not change the area in the source image a1 except the hair, improve the image inpainting effect.

[0044] Preferably, the neural network used in the embodiment is pre-trained, and is trained using an e4e neural network. The e4e uses an encoder-based method to obtain the best perceptual quality and editable latents of the inversion image under the premise of controlling the distortion amount. It not only focuses on how to obtain a high-quality inversion image, but also focuses on how to obtain high-quality editable latents, so that the real image has stronger perceptual quality and evolution ability in the digital twin space.

[0045] In simple terms, the training method in the embodiment is as follows: a mask is randomly used to change the values of the corresponding parts of a picture to all 0, and then the mask and the picture are output to the e4e together. The output picture is supervised by the original picture before masking, and the e4e is trained in this way. When used, the neural network will perform image inpainting on the corresponding area according to the mask.

[0046] In the embodiment, the hair region (source image hairstyle mask m1) is obtained based on the first hair segmentation, the repair area can be determined based on the source image hairstyle mask m1, and the hairstyle in the first feature map a2 is changed to short hair / light head through the direction vector to generate a guide image for image inpainting; the neural network performs image inpainting on the corresponding area according to the source image hairstyle mask m1 and the guide image, and generates non-hair in the source image hairstyle mask m1 area for hairstyle conversion, so that no extra source image hair appears.

[0047] In the embodiment, the second hair segmentation is performed on the second feature map c, and the second hair segmentation result is compressed, and the compression result is denoted as M, which includes the following steps: a pre-trained hair segmentation toolkit is used to perform second hair segmentation on the second feature map c to obtain a target hairstyle mask m2; and the target hairstyle mask m2 is compressed to the scale of the feature space F, and the compression result is denoted as M.

[0048] In the embodiment, the hair segmentation toolkit is an open source model, which outputs a second hair segmentation result according to the input second feature map c, for example, a pixel position belonging to hair is represented by 1, and a pixel position not belonging to hair is represented by 0. It can be understood that the first hair segmentation principle is similar to the second hair segmentation, which is also realized by the above-mentioned hair segmentation toolkit, and details are not described herein.

[0049] In the embodiment, the F space is a subspace of the FS space. The FS space can be essentially regarded as a result of an intermediate layer of a Stylegan generator. F and S basically separate the structure information, F represents the structure, and S represents the color and other information. Once the value of FS is determined, the final picture obtained after the subsequent layer of the Stylegan generator is also determined.

[0050] W+ is a feature space of the Stylegan generator, and functionally, W+ can be edited, and FS cannot edit short hair because the value of F in FS is fixed, which has the structure information of the picture and cannot be changed.

[0051] The application compresses the target hairstyle mask m2 to the scale of the feature space F. Since the F space has a structure meaning, a fixed F has determined the structure of the generated graph, and therefore, compared with fusion at the pixel level of the picture, fusion at the F feature level will not appear unnatural transition edges after editing, and at the same time, the consistency of the irrelevant area and the original picture can be preserved, so that the hairstyle conversion effect is more natural. In the embodiment, based on the second feature map c, the third feature map a3 and M, the hairstyle fusion is performed to obtain the first fusion picture d1, and the specific formula is as follows:

[0052] F merge =M*F align +(1-M)*F short ;

[0053] Wherein, F merge represents the first fused high-dimensional feature, which has a spatial meaning, F align represents the F high-dimensional feature of the second feature map c, and F short represents the F high-dimensional feature of the third feature map a3.

[0054] The above hairstyle fusion is specifically based on the second feature vector of the second feature map c, the third feature vector of the third feature map a3 and M to perform hairstyle fusion to obtain the first fusion picture d1. The fusion is at the feature level rather than at the picture level, so the fusion result is more natural.

[0055] It should be noted that, in order to show the embodiment of the hairstyle conversion method of the application, Figure 2The second feature map is referred to as c, and the third feature map is referred to as a3. However, it can be understood that, after the first hair segmentation of the first feature map a2 is input into the neural network for image inpainting, the third feature vector (not shown in the figure) and the third feature map a3 are obtained; the target hairstyle alignment map b2 is input into the first encoder, and the second feature vector (not shown in the figure) and the second feature map are obtained. Therefore, c actually further includes the second feature corresponding to the second feature map, and a3 further includes the third feature vector corresponding to the third feature map, and a2 further includes the first feature vector corresponding to the first feature map.

[0056] The S feature used in the fusion is the source map to be transferred (that is, the second feature map c), and because the S space represents more non-structural attributes, the fusion result retains the user's light and hair color of the source map in terms of hair color and light, and can be adaptively matched with the source map to be transferred.

[0057] On the complete S space, some values of the S space can be modified so that the corresponding features change, for example, the hairstyle is changed to be curly or with bangs. Therefore, in the present application, when the hairstyle is converted, the corresponding values can also be modified to control the final hairstyle effect. That is, in the present application, the hairstyle conversion can not only convert into the appearance of the target hairstyle, but also convert into the style of the target hairstyle after editing, which is more flexible and diversified.

[0058] In the embodiment, the secondary fusion is based on the fusion mask m3 of the source hairstyle mask m1 and the target hairstyle mask m2, the first fusion image d1 and the source image to be converted a1. Specifically, the secondary fusion is based on the region of the source image to be converted a1 which is not fused by the fusion mask m3, the first fusion image d1 and the fusion mask m3. The fusion mask m3 is the union of the source hairstyle mask m1 and the target hairstyle mask m2.

[0059] Further, the secondary fusion is a partial replacement of the source image, so that the face of the second fusion image obtained after the secondary fusion is consistent with the face of the source image.

[0060] The present application can avoid changes in the face and body and other non-hair regions due to hairstyle conversion by performing secondary fusion based on the fusion mask m3 of the source hairstyle mask m1 and the target hairstyle mask m2, the region of the source image to be converted a1 which is not fused by the fusion mask m3, the first fusion image d1, thereby improving the hairstyle conversion effect.

[0061] The present application is realized based on the Stylegan generator, which can automatically match the face angle of the user input image, automatically adjust the hair color and light after the hairstyle conversion, and the target hairstyle conversion is more flexible and diversified, and the edited picture is high-definition, natural and realistic.

[0062] Corresponding to the hair style conversion method, the present application provides a hair style conversion device, comprising: an image acquisition module, used to acquire a source image a1 to be converted and a target hair style image b1; an alignment processing module, used to perform alignment processing on the source image a1 to be converted and the target hair style image b1, to obtain a target hair style alignment image; a first encoder, used to input the source image a1 to be converted into the first encoder, to obtain a first feature vector and a first feature map a2; and used to input the target hair style alignment image into the first encoder, to obtain a second feature vector and a second feature map c; an image inpainting module, used to input the first feature map a2 after first hair segmentation into a neural network to perform image inpainting, to obtain a third feature vector and a third feature map a3; a hair segmentation module, used to perform first hair segmentation on the first feature map a2, and perform second hair segmentation on the second feature map c; a compression module, used to compress the second hair segmentation result of the second feature map c, and record the compression result as M; a first fusion module, used to perform hair style fusion based on the second feature map c, the third feature map a3 and M, to obtain a first fusion image d1; and a second fusion module, used to perform secondary fusion on the first fusion image d1, to obtain a first fusion image d2.

[0063] In addition, in order to achieve the above-mentioned purpose, the present application also provides a computer readable storage medium, which can be a computer readable storage medium contained in a memory; or can exist independently without being assembled into a device. The computer readable storage medium stores at least one instruction, which is loaded and executed by a processor to realize the hair style conversion method shown in the present application. Figure 1 The computer readable storage medium can be a read-only memory, a disk or an optical disk, etc.

[0064] It should be noted that each embodiment in the present application is described in a progressive manner, and each embodiment focuses on the difference from other embodiments, and the same or similar parts of each embodiment can be referred to each other. For the device embodiment, the equipment embodiment and the storage medium embodiment, since they are basically similar to the method embodiment, the description is relatively simple, and the relevant parts are referred to the part of the method embodiment.

[0065] In addition, in this paper, the term "including", "containing" or any other variant thereof is intended to cover non-exclusive containing, so that the process, method, article or device including a series of elements not only includes those elements, but also includes other elements not explicitly listed, or includes elements inherent to such process, method, article or device.

[0066] The foregoing description has shown and described preferred embodiments of the application, but it will be understood that the application is not limited to the particular embodiments disclosed, and it will be understood that various other modifications, applications and embodiments can be made without departing from the spirit and scope of the application, and that the scope of the application will be measured by the claims that follow.

Claims

1. A method of hair style conversion, characterized by, The method comprises the following steps: Obtain a source image to be converted and a target hairstyle image, and perform alignment processing to obtain an aligned target hairstyle image; Input the source image to be converted into a first encoder to obtain a first feature vector and a first feature map; the first encoder is obtained by pre-training, specifically, a plurality of face images are generated by a Stylegan generator, and a plurality of vectors corresponding to the plurality of face images are saved for training the first encoder; After the first feature map is segmented for the first time, the neural network is input for image inpainting to obtain a third feature vector and a third feature map, specifically including the following steps: the first feature map is segmented for the first time to obtain a source image hairstyle mask to determine a region to be inpainted; a direction vector of edited hair length is obtained based on the face image, and the first feature map is processed based on the direction vector to obtain a guide image for guiding image inpainting; the first feature map is input into the neural network, and the neural network performs image inpainting based on the source image hairstyle mask and the guide image; Input the aligned target hairstyle image into the first encoder to obtain a second feature vector and a second feature map; the second feature map is segmented for the second time, and the second hair segmentation result is compressed, and the compression result is denoted as M; Based on the second feature map, the third feature map and M, hairstyle fusion is performed to obtain a first fusion image; The first fusion image is subjected to secondary fusion to obtain a second fusion image; the secondary fusion is performed based on a fusion mask of the source image hairstyle mask and the target hairstyle mask, the first fusion image and the source image to be converted; The fusion mask is the union set of the source image hairstyle mask and the target hairstyle mask.

2. A method of changing a hairstyle according to claim 1, characterized in that: The alignment processing further comprises: Respectively detecting key points of the source image to be converted and the target hairstyle image to obtain key points of the source image to be converted and key points of the target hairstyle image; Input the target hairstyle image and the key points of the target hairstyle image into a second encoder of an alignment network to extract features to obtain corresponding high-dimensional features; Input the high-dimensional features and the key points of the source image to be converted into a generator of the alignment network to generate the aligned target hairstyle image.

3. A method of changing a hairstyle according to claim 2, characterized in that: The alignment network is trained by a K-hairstyle dataset, and the alignment network adopts a u-net structure.

4. A method of changing a hairstyle according to claim 1, wherein: The direction vector of the edited hair length is obtained based on the face image, specifically including the following steps: Obtain a preset long hair threshold and a preset short hair threshold, and classify the face image by a face attribute discriminator to obtain a picture dataset with a hair length label; Train the corresponding feature vector based on the hair length label to obtain an svm classification hyperplane; Take the normal vector of the svm classification hyperplane to obtain the direction vector of the edited hair length.

5. A method of changing a hairstyle according to claim 1, wherein: The second feature map is segmented for the second time and compressed, and the compression result is denoted as M, specifically including the following steps: Segment the second feature map for the second time to obtain a target hairstyle mask; Then compress the target hairstyle mask to the scale of the feature space F, and denote the compression result as M.

6. A method of changing a hairstyle according to claim 1, wherein: Based on the second feature map, the third feature map and M, hairstyle fusion is performed to obtain a first fusion image, specifically as follows: wherein, represents the first fused high-dimensional feature with spatial meaning, represents the F high-dimensional feature of the second feature map, represents the F high-dimensional feature of the third feature map.

7. A hair conversion device characterized by, It comprises: An image acquisition module is configured to acquire a source image to be converted and a target hairstyle image; An alignment processing module is configured to perform alignment processing on the source image to be converted and the target hairstyle image to obtain a target hairstyle alignment image; A first encoder is configured to input the source image to be converted into the first encoder to obtain a first feature vector and a first feature map; The target hairstyle alignment image is input into the first encoder to obtain a second feature vector and a second feature map; the first encoder is obtained by pre-training, specifically, a plurality of face images are generated by a Stylegan generator, and a plurality of vectors corresponding to the plurality of face images are saved for training the first encoder; An image repairing module is configured to input the first feature map after first hair segmentation into a neural network for image repairing to obtain a third feature vector and a third feature map, specifically including the following steps: performing first hair segmentation on the first feature map to obtain a source image hairstyle mask to determine a region to be repaired; obtaining a direction vector of edited hair length based on the face image, and processing the first feature map based on the direction vector to obtain a guide image for guiding image repairing; inputting the first feature map into the neural network, and the neural network performs image repairing based on the source image hairstyle mask and the guide image; A hair segmentation module is configured to perform first hair segmentation on the first feature map and second hair segmentation on the second feature map; A compression module is configured to compress the second hair segmentation result of the second feature map, and record the compression result as M; A first fusion module is configured to perform hairstyle fusion based on the second feature map, the third feature map, and M to obtain a first fusion image; A second fusion module is configured to perform secondary fusion on the first fusion image to obtain a second fusion image; the secondary fusion is performed based on a fusion mask of the source image hairstyle mask and the target hairstyle mask, the first fusion image, and the source image to be converted; The fusion mask is a union set of the source image hairstyle mask and the target hairstyle mask.

8. A computer-readable storage medium, characterized in that, The computer readable storage medium stores an image processing program, and the image processing program is executed by the processor to implement the steps of the hairstyle conversion method according to any one of claims 1 to 6.

Images