Facial makeup migration method, device and computer-readable storage medium

Abstract

The present invention discloses a method, device and computer-readable storage medium for migrating facial makeup, wherein the method comprises: obtaining original segmentation features and original facial features corresponding to an original facial image, and obtaining makeup segmentation features and makeup facial features corresponding to a makeup facial image; determining a regional attention tensor according to the original segmentation features, the makeup segmentation features, the original facial features and the makeup facial features; determining a migrated makeup tensor according to the regional attention tensor and the original segmentation features; determining a migrated makeup facial feature according to the migrated makeup tensor and the original facial features, and inputting the migrated makeup facial feature into a decoder to determine a migrated makeup facial image. The present invention can obtain a regional attention tensor according to the original facial image and the makeup facial image, obtain a migrated makeup tensor according to the regional attention tensor, and then accurately obtain a migrated makeup facial image for migrating facial regional makeup according to the migrated makeup tensor.

Description

Technical Field

[0001] The present invention relates to the technical field of facial makeup migration, and in particular to a facial makeup migration method, device, and computer-readable storage medium. Background Art

[0002] Facial attribute editing, one of the hottest research areas in computer vision, has spawned numerous subtasks, such as makeup transfer, facial cartoonization, and expression editing, all of which have broad research and application value. The makeup transfer task aims to transfer makeup from a reference image to the corresponding area of the original image, while ensuring that the original image remains unchanged, without distortion or loss of identity.

[0003] Numerous methods have been proposed for makeup transfer in recent years, but most only work on frontal facial images. When both the reference and original images are profiles, the results can be disappointing. Furthermore, previous methods focus more on global makeup transfer and lack the flexibility of regional-level makeup transfer. This approach significantly limits the makeup transfer task.

[0004] The above content is only used to assist in understanding the technical solution of the present invention and does not constitute an admission that the above content is prior art. Summary of the Invention

[0005] The main purpose of the present invention is to provide a facial makeup migration method, device and computer-readable storage medium, aiming to solve the technical problem that the existing technology cannot achieve makeup migration in the facial area.

[0006] To achieve the above object, the present invention provides a method for transferring facial makeup, which comprises the following steps:

[0007] Obtaining original segmentation features and original facial features corresponding to the original face image, and obtaining makeup segmentation features and makeup facial features corresponding to the makeup face image;

[0008] Determining a regional attention tensor according to the original segmentation features, the makeup segmentation features, the original facial features, and the makeup facial features;

[0009] Determining a migration makeup tensor based on the regional attention tensor and the original segmentation features;

[0010] Determine the transferred makeup facial features according to the transferred makeup tensor and the original facial features, and input the transferred makeup facial features into a decoder to determine a transferred makeup face image.

[0011] Furthermore, the step of determining the regional attention tensor based on the original segmentation features, the makeup segmentation features, the original facial features, and the makeup facial features includes:

[0012] Determining a first facial region feature based on the original segmentation feature and the original facial feature;

[0013] determining a second facial region feature based on the makeup segmentation feature and the makeup facial feature;

[0014] determining a regional attention matrix according to the first face region feature and the second face region feature;

[0015] Determine a regional attention tensor based on the regional attention matrix.

[0016] Furthermore, the original segmentation features include a first eye shadow feature, a first lip feature, and a first skin feature, and the first facial region features include a first target eye shadow feature, a first target lip feature, and a first target skin feature. The step of determining the first facial region features based on the original segmentation features and the original facial features includes:

[0017] Performing a matrix dot multiplication calculation on the first eye shadow feature and the original face feature to obtain a first target eye shadow feature;

[0018] Performing a matrix dot multiplication calculation on the first lip feature and the original face feature to obtain a first target lip feature;

[0019] Perform a matrix dot multiplication calculation on the first skin feature and the original facial feature to obtain a first target skin feature.

[0020] Furthermore, the makeup segmentation feature includes a second eye shadow feature, a second lip feature, and a second skin feature, and the second facial region feature includes a second target eye shadow feature, a second target lip feature, and a second target skin feature. The step of determining the second facial region feature based on the makeup segmentation feature and the makeup facial feature includes:

[0021] Performing a matrix dot multiplication calculation on the second eye shadow feature and the makeup face feature to obtain a second target eye shadow feature;

[0022] Performing a matrix dot multiplication calculation on the second lip feature and the makeup face feature to obtain a second target lip feature;

[0023] Perform matrix dot multiplication on the second skin feature and the makeup face feature to obtain a second target skin feature.

[0024] Furthermore, the regional attention matrix includes an eye shadow attention matrix, a lip attention matrix, and a skin attention matrix. The step of determining the regional attention matrix according to the first face region feature and the second face region feature includes:

[0025] Performing a reshape operation and a transpose operation on the first target eye shadow feature to obtain a first matrix, performing a reshape operation on the second target eye shadow feature to obtain a second matrix, and performing a matrix multiplication operation on the first matrix and the second matrix to obtain an eye shadow attention matrix;

[0026] Performing a reshape operation and a transpose operation on the first target lip feature to obtain a third matrix, performing a reshape operation on the second target lip feature to obtain a fourth matrix, and performing a matrix multiplication operation on the third matrix and the fourth matrix to obtain a lip attention matrix;

[0027] The first target skin feature is reshaped and transposed respectively to obtain a fifth matrix, the second target skin feature is reshaped to obtain a sixth matrix, and the fifth matrix and the sixth matrix are multiplied to obtain a skin attention matrix.

[0028] Furthermore, the regional attention tensor includes an eye shadow attention tensor, a lip attention tensor, and a skin attention tensor. The step of determining the regional attention tensor according to the regional attention matrix includes:

[0029] Performing a flexible maximum transfer function operation on the eye shadow attention matrix to obtain a seventh matrix, and performing a matrix multiplication operation on the seventh matrix and the second matrix to obtain an eye shadow attention tensor;

[0030] Performing a flexible maximum transfer function operation on the lip attention matrix to obtain an eighth matrix, and performing a matrix multiplication operation on the eighth matrix and the fourth matrix to obtain a lip attention tensor;

[0031] A flexible maximum transfer function operation is performed on the skin attention matrix to obtain a ninth matrix, and a matrix multiplication operation is performed on the ninth matrix and the sixth matrix to obtain a skin attention tensor.

[0032] Furthermore, the step of obtaining a migration makeup tensor based on the regional attention tensor and the original segmentation feature includes:

[0033] Performing a convolution operation on the eye shadow attention tensor and the first convolution to obtain a first convolution operation result, and performing a matrix dot product calculation on the first convolution operation result and the first eye shadow feature to obtain an eye shadow makeup tensor;

[0034] Performing a convolution operation on the lip attention tensor and the second convolution to obtain a second convolution operation result, and performing a matrix dot product calculation on the second convolution operation result and the first lip feature to obtain a lip makeup tensor;

[0035] Performing a convolution operation on the skin attention tensor and the third convolution to obtain a third convolution operation result, and performing a matrix dot product calculation on the third convolution operation result and the first skin feature to obtain a skin makeup tensor;

[0036] The eye shadow makeup tensor, the lip makeup tensor, and the skin makeup tensor are concatenated to obtain a migrated makeup tensor.

[0037] Furthermore, the step of obtaining the migrated makeup facial features based on the migrated makeup tensor and the original facial features includes:

[0038] Performing a convolution operation on the migration makeup tensor and the fourth convolution to obtain a first region makeup tensor;

[0039] Performing a convolution operation on the migrated makeup tensor and the fifth convolution to obtain a second region makeup tensor;

[0040] Performing a matrix multiplication operation on the first region makeup tensor and the original facial features to obtain a tenth matrix;

[0041] A matrix addition operation is performed on the tenth matrix and the second region makeup tensor to obtain the migrated makeup facial features.

[0042] In addition, to achieve the above-mentioned purpose, the present invention also provides a facial makeup migration device, which includes: a memory, a processor, and a facial makeup migration program stored on the memory and runnable on the processor. When the facial makeup migration program is executed by the processor, the steps of the aforementioned facial makeup migration method are implemented.

[0043] In addition, to achieve the above-mentioned purpose, the present invention also provides a computer-readable storage medium, on which a facial makeup migration program is stored. When the facial makeup migration program is executed by a processor, the steps of the aforementioned facial makeup migration method are implemented.

[0044] The present invention obtains the original segmentation features and original facial features corresponding to the original face image, and obtains the makeup segmentation features and makeup facial features corresponding to the makeup face image, and then determines the regional attention tensor according to the original segmentation features, the makeup segmentation features, the original facial features and the makeup facial features, and then determines the migration makeup tensor according to the regional attention tensor and the original segmentation features, and then determines the migration makeup facial features according to the migration makeup tensor and the original facial features, and inputs the migration makeup facial features into the decoder to determine the migration makeup face image, and can obtain the regional attention tensor according to the original face image and the makeup face image, and obtain the migration makeup tensor according to the regional attention tensor, and then accurately obtain the migration makeup face image of the face area makeup migration according to the migration makeup tensor. BRIEF DESCRIPTION OF THE DRAWINGS

[0045] Figure 1 Schematic diagram of the structure of a facial makeup migration device in a hardware operating environment according to an embodiment of the present invention;

[0046] Figure 2 Schematic diagram of the process of the first embodiment of the facial makeup transfer method of the present invention.

[0047] The purpose, features and advantages of the present invention will be further described with reference to the accompanying drawings and in conjunction with the embodiments. DETAILED DESCRIPTION

[0048] It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention.

[0049] like Figure 1 As shown, Figure 1 It is a structural diagram of a facial makeup migration device in a hardware operating environment involved in an embodiment of the present invention.

[0050] The facial makeup migration device in the embodiment of the present invention can be a PC, or it can be a mobile terminal device with display function such as a smart phone, a tablet computer, an e-book reader, an MP3 (Moving Picture Experts Group Audio Layer III, Moving Picture Experts Compression Standard Audio Layer 3) player, an MP4 (Moving Picture Experts Group Audio Layer IV, Moving Picture Experts Compression Standard Audio Layer 4) player, or a portable computer.

[0051] like Figure 1As shown, the facial makeup migration device may include: a processor 1001, such as a CPU, a network interface 1004, a user interface 1003, a memory 1005, and a communication bus 1002. Among them, the communication bus 1002 is used to realize the connection and communication between these components. The user interface 1003 may include a display screen (Display), an input unit such as a keyboard (Keyboard), and the user interface 1003 may also include a standard wired interface and a wireless interface. The network interface 1004 may optionally include a standard wired interface and a wireless interface (such as a WI-FI interface). The memory 1005 may be a high-speed RAM memory, or a stable memory (non-volatile memory), such as a disk memory. The memory 1005 may also be a storage device independent of the aforementioned processor 1001.

[0052] Optionally, the facial makeup migration device may also include a camera, an RF (Radio Frequency) circuit, a sensor, an audio circuit, a WiFi module, and the like. Among them, sensors include light sensors, motion sensors, and other sensors. Specifically, the light sensor may include an ambient light sensor and a proximity sensor, wherein the ambient light sensor may adjust the brightness of the display screen according to the brightness of the ambient light, and the proximity sensor may turn off the display screen and / or backlight when the facial makeup migration device is moved to the ear. As a type of motion sensor, the gravity acceleration sensor can detect the magnitude of acceleration in all directions (generally three axes), and can detect the magnitude and direction of gravity when stationary. It can be used for applications that recognize the posture of the facial makeup migration device (such as horizontal and vertical screen switching, related games, magnetometer posture calibration), vibration recognition related functions (such as pedometer, tapping), etc.; of course, the facial makeup migration device can also be equipped with other sensors such as gyroscopes, barometers, hygrometers, thermometers, infrared sensors, etc., which will not be repeated here.

[0053] Those skilled in the art will understand that Figure 1 The terminal structure shown in the figure does not constitute a limitation to the terminal, and may include more or fewer components than shown in the figure, or combine certain components, or arrange the components differently.

[0054] like Figure 1 As shown, the memory 1005 as a computer storage medium may include an operating system, a network communication module, a user interface module, and a facial makeup migration program.

[0055] exist Figure 1In the terminal shown, the network interface 1004 is mainly used to connect to the background server and communicate data with the background server; the user interface 1003 is mainly used to connect to the client (user end) and communicate data with the client; and the processor 1001 can be used to call the facial makeup migration program stored in the memory 1005.

[0056] In this embodiment, the facial makeup migration device includes: a memory 1005, a processor 1001, and a facial makeup migration program stored on the memory 1005 and executable on the processor 1001, wherein the processor 1001 calls the facial makeup migration program stored in the memory 1005 and executes the steps of the facial makeup migration method in each of the following embodiments.

[0057] The present invention also provides a method for migrating facial makeup, referring to Figure 2 , Figure 2 Schematic diagram of the first embodiment of the method of the present invention.

[0058] In this embodiment, the facial makeup transfer method includes the following steps:

[0059] Step S101, obtaining original segmentation features and original facial features corresponding to the original face image, and obtaining makeup segmentation features and makeup facial features corresponding to the makeup face image;

[0060] In this embodiment, an original face image and a makeup face image are obtained, wherein the original face image refers to the image to be migrated with makeup, and the makeup face image refers to the image used for makeup migration. The original face image is input into the face semantic segmentation network to obtain original segmentation features, and the makeup face image is input into the face semantic segmentation network to obtain makeup segmentation features. Specifically, the face semantic segmentation network can use a pre-trained face segmentation network Bisenet. The original face image is input into the face segmentation network Bisenet to obtain original segmentation features, and the makeup face image is input into the face segmentation network Bisenet to obtain makeup segmentation features. The original face image is input into the feature extractor to obtain original face features, and the makeup face image is input into the feature extractor to obtain makeup face features. Specifically, the original face image is input into the feature extractor to obtain original face features. x , input the makeup face image into the feature extractor to obtain the makeup face feature f y .

[0061] Step S102, determining a regional attention tensor based on the original segmentation features, the makeup segmentation features, the original facial features, and the makeup facial features;

[0062] In this embodiment, the obtained original segmentation features, makeup segmentation features, original facial features, and makeup facial features are calculated to obtain a regional attention tensor.

[0063] Step S103, determining a transferred makeup tensor based on the regional attention tensor and the original segmentation features;

[0064] In this embodiment, based on the obtained regional attention tensor and the original segmentation features, the eye shadow makeup tensor, lip makeup tensor, and skin makeup tensor are obtained by calculation, and the eye shadow makeup tensor, lip makeup tensor, and skin makeup tensor are spliced to obtain the migrated makeup tensor.

[0065] Step S104 : determining a transferred makeup facial feature based on the transferred makeup tensor and the original facial feature, and inputting the transferred makeup facial feature into a decoder to determine a transferred makeup face image.

[0066] In this embodiment, the transferred makeup facial features can be determined based on the obtained transferred makeup tensor and the original facial features, and the transferred makeup facial features can be input into the decoder to determine the transferred makeup face image. The process of determining the transferred makeup face image is all implemented in a unified network framework, which not only can realize makeup transfer under the condition that the original image and the reference image are both side faces, but also can realize accurate regional makeup control.

[0067] Furthermore, in one embodiment, step S102 includes:

[0068] Step 201, determining a first facial region feature based on the original segmentation feature and the original facial feature;

[0069] Step 202: determining a second facial region feature based on the makeup segmentation feature and the makeup facial feature;

[0070] Step 203: determining a regional attention matrix based on the first face region feature and the second face region feature;

[0071] Step 204: Determine the regional attention tensor based on the regional attention matrix.

[0072] In this embodiment, the first facial region feature is first determined based on the calculation of the original segmentation feature and the original facial feature, then the second facial region feature is determined based on the makeup segmentation feature and the makeup facial feature, then the regional attention matrix is determined based on the first facial region feature and the second facial region feature, and finally the regional attention tensor is determined based on the regional attention matrix, wherein the original segmentation feature includes the first eye shadow feature, the first lip feature and the first skin feature, the first facial region feature includes the first target eye shadow feature, the first target lip feature and the first target skin feature, the makeup segmentation feature includes the second eye shadow feature, the second lip feature and the second skin feature, the second facial region feature includes the second target eye shadow feature, the second target lip feature and the second target skin feature, the regional attention matrix includes the eye shadow attention matrix, the lip attention matrix and the skin attention matrix, and the regional attention tensor includes the eye shadow attention tensor, the lip attention tensor and the skin attention tensor.

[0073] The face makeup migration method proposed in this embodiment obtains an original face image and a makeup face image, then inputs the original face image into a face semantic segmentation network to obtain original segmentation features, inputs the makeup face image into a face semantic segmentation network to obtain makeup segmentation features, and then inputs the original face image into a feature extractor to obtain original face features, inputs the makeup face image into a feature extractor to obtain makeup face features, and then according to the original segmentation features, the makeup segmentation features, the original face features and the makeup face image, a face makeup migration method is proposed. The method uses the facial features to determine the regional attention tensor, and then determines the migration makeup tensor according to the regional attention tensor and the original segmentation features. Finally, the migration makeup facial features are determined according to the migration makeup tensor and the original facial features, and the migration makeup facial features are input into the decoder to determine the migration makeup face map. The regional attention tensor can be obtained according to the original face map and the makeup face map, and the migration makeup tensor can be obtained according to the regional attention tensor. According to the migration makeup tensor, the migration makeup face map of the face region makeup migration can be accurately obtained.

[0074] Based on the first embodiment, a second embodiment of the facial makeup transfer method of the present invention is proposed. In this embodiment, step S201 includes:

[0075] Step S301, performing matrix dot multiplication calculation on the first eye shadow feature and the original face feature to obtain a first target eye shadow feature;

[0076] Step S302: performing a matrix dot multiplication on the first lip feature and the original face feature to obtain a first target lip feature;

[0077] Step S303: Perform matrix dot multiplication on the first skin feature and the original facial feature to obtain a first target skin feature.

[0078] It should be noted that the original segmentation features include the first eye shadow feature, the first lip feature and the first skin feature, and the first face region feature includes the first target eye shadow feature, the first target lip feature and the first target skin feature.

[0079] This embodiment proposes to perform matrix dot product calculation on the obtained first eye shadow feature and the original facial feature, and the calculation result is the first target eye shadow feature. Then, the obtained first lip feature and the original facial feature are subjected to matrix dot product calculation, and the calculation result is the first target lip feature. Finally, the first skin feature and the original facial feature are subjected to matrix dot product calculation, and the calculation result is the first target skin feature.

[0080] Furthermore, in one embodiment, step S202 includes:

[0081] Step b, performing matrix dot multiplication calculation on the second eye shadow feature and the makeup face feature to obtain a second target eye shadow feature;

[0082] Step c, performing matrix dot multiplication on the second lip feature and the makeup face feature to obtain a second target lip feature;

[0083] Step d: performing matrix dot multiplication on the second skin feature and the makeup face feature to obtain a second target skin feature.

[0084] It should be noted that the makeup segmentation features include the second eye shadow features, the second lip features and the second skin features, and the second facial region features include the second target eye shadow features, the second target lip features and the second target skin features.

[0085] In this embodiment, a matrix dot product is performed on the acquired second eye shadow feature and the makeup face feature, and the result is the second target eye shadow feature. The second target eye shadow feature is obtained. A matrix dot product is performed on the acquired second lip feature and the makeup face feature, and the result is used as the second target lip feature. A matrix dot product is performed on the acquired second skin feature and the makeup face feature, and the result is used as the second target skin feature.

[0086] Furthermore, in one embodiment, step S203 includes:

[0087] Step e: performing a reshaping operation and a transposition operation on the first target eye shadow feature to obtain a first matrix, performing a reshaping operation on the second target eye shadow feature to obtain a second matrix, and performing a matrix multiplication operation on the first matrix and the second matrix to obtain an eye shadow attention matrix;

[0088] Step f, performing a reshape operation and a transpose operation on the first target lip feature to obtain a third matrix, performing a reshape operation on the second target lip feature to obtain a fourth matrix, and performing a matrix multiplication operation on the third matrix and the fourth matrix to obtain a lip attention matrix;

[0089] Step g: reshape and transpose the first target skin feature to obtain a fifth matrix, reshape the second target skin feature to obtain a sixth matrix, and perform matrix multiplication on the fifth matrix and the sixth matrix to obtain a skin attention matrix.

[0090] It should be noted that the regional attention matrix includes the eye shadow attention matrix, the lip attention matrix and the skin attention matrix.

[0091] In this embodiment, a reshaping operation is first performed on the acquired first target eye shadow features to obtain a result, which is then transposed to obtain a calculation result, which is used as a first matrix and the first matrix is obtained. A reshaping operation is then performed on the acquired second target eye shadow features to obtain a calculation result, which is used as a second matrix and the second matrix is obtained. A matrix multiplication operation is performed on the first matrix and the second matrix to obtain a multiplication result, which is used as an eye shadow attention matrix and the eye shadow attention matrix is obtained.

[0092] A reshape operation is first performed on the acquired first target lip features to obtain a result. The result is then transposed to obtain a calculation result. The calculation result is used as a third matrix and the third matrix is obtained. A reshape operation is performed on the acquired second target lip features to obtain a calculation result. The calculation result is used as a fourth matrix and the fourth matrix is obtained. A matrix multiplication operation is performed on the third matrix and the fourth matrix to obtain a multiplication result. The multiplication result is used as a lip attention matrix and the lip attention matrix is obtained.

[0093] A reshaping operation is first performed on the acquired first target skin feature to obtain a result, the result is transposed to obtain a calculation result, the calculation result is used as a fifth matrix, and the fifth matrix is obtained. A reshaping operation is performed on the acquired second target skin feature to obtain a calculation result, the calculation result is used as a second matrix, and the second matrix is obtained. A matrix multiplication operation is performed on the first matrix and the second matrix to obtain a multiplication result, the multiplication result is used as a skin attention matrix, and the skin attention matrix is obtained.

[0094] Furthermore, in one embodiment, step S204 includes:

[0095] Step h, performing a flexible maximum transfer function operation on the eye shadow attention matrix to obtain a seventh matrix, and performing a matrix multiplication operation on the seventh matrix and the second matrix to obtain an eye shadow attention tensor;

[0096] Step i, performing a flexible maximum transfer function operation on the lip attention matrix to obtain an eighth matrix, and performing a matrix multiplication operation on the eighth matrix and the fourth matrix to obtain a lip attention tensor;

[0097] In step j, a flexible maximum transfer function operation is performed on the skin attention matrix to obtain a ninth matrix, and a matrix multiplication operation is performed on the ninth matrix and the sixth matrix to obtain a skin attention tensor.

[0098] It should be noted that the regional attention tensor includes the eye shadow attention tensor, the lip attention tensor, and the skin attention tensor.

[0099] In this embodiment, the eye shadow attention matrix is subjected to a flexible maximum transfer function operation. Specifically, the eye shadow attention matrix is input into the flexible maximum transfer function to obtain a calculation result. The above calculation result is used as the seventh matrix, and the seventh matrix is obtained. The seventh matrix is subjected to a matrix multiplication operation with the second matrix to obtain a calculation result. The above calculation result is used as the eye shadow attention tensor, and the eye shadow attention tensor is obtained.

[0100] The lip attention matrix is subjected to a flexible maximum transfer function operation. Specifically, the lip attention matrix is input into the flexible maximum transfer function to obtain a calculation result. The above calculation result is used as the eighth matrix, and the eighth matrix is obtained. The eighth matrix is subjected to a matrix multiplication operation with the fourth matrix to obtain a calculation result. The above calculation result is used as the lip attention tensor, and the lip attention tensor is obtained.

[0101] The skin attention matrix is subjected to a flexible maximum transfer function operation. Specifically, the skin attention matrix is input into the flexible maximum transfer function to obtain a calculation result. The above calculation result is used as the ninth matrix, and the ninth matrix is obtained. The ninth matrix is subjected to a matrix multiplication operation with the sixth matrix to obtain a calculation result. The above calculation result is used as the skin attention tensor to obtain the skin attention tensor.

[0102] The facial makeup migration method proposed in this embodiment determines the first facial region feature based on the original segmentation feature and the original facial feature, then determines the second facial region feature based on the makeup segmentation feature and the makeup facial feature, then determines the regional attention matrix based on the first facial region feature and the second facial region feature, and finally determines the regional attention tensor based on the regional attention matrix. The regional attention tensor can be obtained based on the original facial image and the makeup facial image, so that the migration makeup tensor can be obtained more accurately based on the regional attention tensor and the original segmentation feature, and then the migration makeup facial image of the facial region makeup migration can be obtained more accurately based on the migration makeup tensor and the original facial feature.

[0103] Based on the second embodiment, a third embodiment of the facial makeup transfer method of the present invention is proposed. In this embodiment, step S103 includes:

[0104] Step 401: performing a convolution operation on the eye shadow attention tensor and the first convolution to obtain a first convolution operation result, and performing a matrix dot product calculation on the first convolution operation result and the first eye shadow feature to obtain an eye shadow makeup tensor;

[0105] Step 402: Convolve the lip attention tensor with the second convolution to obtain a second convolution result, and perform a matrix dot product calculation on the second convolution result and the first lip feature to obtain a lip makeup tensor.

[0106] Step 403: performing a convolution operation on the skin attention tensor and the third convolution to obtain a third convolution operation result, and performing a matrix dot product calculation on the third convolution operation result and the first skin feature to obtain a skin makeup tensor;

[0107] Step 404: perform a splicing operation on the eyeshadow makeup tensor, the lip makeup tensor, and the skin makeup tensor to obtain a migrated makeup tensor.

[0108] In this embodiment, the acquired eye shadow attention tensor is convolved with the first convolution to obtain a first convolution result, where the parameters of the first convolution are adjusted based on the preset model training. Next, a matrix dot product is performed on the first convolution result and the acquired first eye shadow feature to obtain a matrix dot product result, which is used as the eye shadow makeup tensor, and the eye shadow makeup tensor is obtained.

[0109] The obtained lip attention tensor is convolved with the second convolution to obtain a second convolution result, where the parameters of the second convolution are adjusted based on the preset model training. Next, the second convolution result is matrix-dot multiplied with the obtained first lip feature to obtain a matrix dot product. This matrix product is used as a lip makeup tensor, and the lip makeup tensor is obtained.

[0110] The obtained skin attention tensor is convolved with the third convolution to obtain a third convolution result, where the parameters of the third convolution are adjusted based on the preset model training. Next, the third convolution result is matrix-dot multiplied with the obtained first skin feature to obtain a matrix dot product. The matrix dot product is used as a skin makeup tensor, and the skin makeup tensor is obtained.

[0111] Finally, the obtained eyeshadow makeup tensor, lip makeup tensor, and skin makeup tensor are concatenated to obtain the concatenated result, which is used as the transferred makeup tensor. The concatenation operation can be performed by the concatenation function.

[0112] The facial makeup migration method proposed in this embodiment performs a convolution operation on the eye shadow attention tensor and a first convolution operation to obtain a first convolution operation result, performs a matrix dot product operation on the first eye shadow feature, and obtains an eye shadow makeup tensor. Then, the lip attention tensor is convolved with a second convolution operation to obtain a second convolution operation result, and the second convolution operation result is matrix dot producted with the first lip feature to obtain a lip makeup tensor. Then, the skin attention tensor is convolved with a third convolution operation to obtain a third convolution operation result, and the third convolution operation result is matrix dot producted with the first skin feature to obtain a skin makeup tensor. Then, the eye shadow makeup tensor, the lip makeup tensor, and the skin makeup tensor are concatenated to obtain a migrated makeup tensor. This method can more accurately obtain a migrated makeup tensor based on the regional attention tensor and the original segmentation features, and thus more accurately obtain a migrated makeup face image for the facial regional makeup migration based on the migrated makeup tensor and the original facial features.

[0113] Based on the third embodiment, a fourth embodiment of the facial makeup transfer method of the present invention is proposed. In this embodiment, step S104 includes:

[0114] Step 501: performing a convolution operation on the migration makeup tensor and the fourth convolution to obtain a first region makeup tensor;

[0115] Step 502: performing a convolution operation on the migrated makeup tensor and the fifth convolution to obtain a second region makeup tensor;

[0116] Step 503: Perform a matrix multiplication operation on the first region makeup tensor and the original facial features to obtain a tenth matrix;

[0117] Step 504 : Perform a matrix addition operation on the tenth matrix and the second region makeup tensor to obtain the transferred makeup facial features.

[0118] In this embodiment, the migrated makeup tensor is first convolved with the fourth convolution to obtain the result of the convolution operation, and the above result is used as the first region makeup tensor, and the first region makeup tensor is obtained. Then, the migrated makeup tensor is convolved with the fifth convolution to obtain the result of the convolution operation, and the above result is used as the second region makeup tensor, and the second region makeup tensor is obtained. Then, the first region makeup tensor is matrix multiplied with the original facial feature to obtain the result of the matrix multiplication operation, and the above result is used as the tenth matrix, and the tenth matrix is obtained. Then, the tenth matrix is matrix added with the second region makeup tensor, and the result of the matrix addition operation is used as the migrated makeup facial feature, and the migrated makeup facial feature is obtained.

[0119] The facial makeup migration method proposed in this embodiment performs a convolution operation on the migrated makeup tensor and the fourth convolution to obtain a first-region makeup tensor, then performs a convolution operation on the migrated makeup tensor and the fifth convolution to obtain a second-region makeup tensor, and then performs a matrix multiplication operation on the first-region makeup tensor and the original facial features to obtain a tenth matrix, and then performs a matrix addition operation on the tenth matrix and the second-region makeup tensor to obtain migrated makeup facial features. The migrated makeup face image of the facial region makeup migration can be accurately obtained based on the migrated makeup tensor and the original facial features.

[0120] In addition, an embodiment of the present invention also proposes a facial makeup migration device, which includes: a memory, a processor, and a facial makeup migration program stored on the memory and runnable on the processor. When the facial makeup migration program is executed by the processor, the steps of the facial makeup migration method described above are implemented.

[0121] In addition, an embodiment of the present invention further proposes a computer-readable storage medium, on which a facial makeup migration program is stored. When the facial makeup migration program is executed by a processor, the steps of the facial makeup migration method described above are implemented.

[0122] It should be noted that, in this document, the terms "comprises," "includes," or any other variations thereof are intended to encompass non-exclusive inclusion, such that a process, method, article, or system comprising a series of elements includes not only those elements but also other elements not explicitly listed, or elements inherent to such process, method, article, or system. In the absence of further limitations, an element defined by the phrase "comprising a ..." does not exclude the presence of other identical elements in the process, method, article, or system comprising the element.

[0123] The serial numbers of the above embodiments of the present invention are for description only and do not represent the advantages or disadvantages of the embodiments.

[0124] Through the description of the above embodiments, those skilled in the art can clearly understand that the above embodiment methods can be implemented by means of software plus the necessary general hardware platform, and of course can also be implemented by hardware, but in many cases the former is a better embodiment. Based on this understanding, the technical solution of the present invention is essentially or the part that contributes to the prior art can be embodied in the form of a software product, which is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk) as described above, and includes a number of instructions for enabling a terminal device (which can be a mobile phone, computer, server, air conditioner, or network device, etc.) to execute the methods described in each embodiment of the present invention.

[0125] The above are only preferred embodiments of the present invention and are not intended to limit the patent scope of the present invention. Any equivalent structure or equivalent process transformation made using the contents of the present invention description and drawings, or directly or indirectly applied in other related technical fields, are also included in the patent protection scope of the present invention.

Claims

1. A facial makeup migration method, characterized in that: The facial makeup migration method comprises the following steps: Obtaining original segmentation features and original facial features corresponding to the original face image, and obtaining makeup segmentation features and makeup facial features corresponding to the makeup face image; Determining a regional attention tensor according to the original segmentation features, the makeup segmentation features, the original facial features, and the makeup facial features; Determining a migration makeup tensor based on the regional attention tensor and the original segmentation features; Determining a transferred makeup face feature according to the transferred makeup tensor and the original face feature, and inputting the transferred makeup face feature into a decoder to determine a transferred makeup face image; The step of determining the regional attention tensor according to the original segmentation features, the makeup segmentation features, the original facial features, and the makeup facial features comprises: Determine a first facial region feature based on the original segmentation feature and the original facial feature; the original segmentation feature includes a first eye shadow feature, a first lip feature, and a first skin feature, and the first facial region feature includes a first target eye shadow feature, a first target lip feature, and a first target skin feature; wherein, perform a matrix dot product calculation on the first eye shadow feature and the original facial feature to obtain a first target eye shadow feature; perform a matrix dot product calculation on the first lip feature and the original facial feature to obtain a first target lip feature; perform a matrix dot product calculation on the first skin feature and the original facial feature to obtain a first target skin feature; Determine a second facial region feature based on the makeup segmentation feature and the makeup facial feature; the makeup segmentation feature includes a second eye shadow feature, a second lip feature, and a second skin feature, and the second facial region feature includes a second target eye shadow feature, a second target lip feature, and a second target skin feature; wherein, perform a matrix dot multiplication on the second eye shadow feature and the makeup facial feature to obtain a second target eye shadow feature; perform a matrix dot multiplication on the second lip feature and the makeup facial feature to obtain a second target lip feature; perform a matrix dot multiplication on the second skin feature and the makeup facial feature to obtain a second target skin feature; Determine a regional attention matrix based on the first facial region feature and the second facial region feature; the regional attention matrix includes an eye shadow attention matrix, a lip attention matrix, and a skin attention matrix; wherein, the first target eye shadow feature is reshaped and transposed respectively to obtain a first matrix, the second target eye shadow feature is reshaped to obtain a second matrix, and the first matrix and the second matrix are matrix multiplied to obtain an eye shadow attention matrix; the first target lip feature is reshaped and transposed respectively to obtain a third matrix, the second target lip feature is reshaped to obtain a fourth matrix, and the third matrix and the fourth matrix are matrix multiplied to obtain a lip attention matrix; the first target skin feature is reshaped and transposed respectively to obtain a fifth matrix, the second target skin feature is reshaped to obtain a sixth matrix, and the fifth matrix and the sixth matrix are matrix multiplied to obtain a skin attention matrix; According to the regional attention matrix, a regional attention tensor is determined; the regional attention tensor includes an eye shadow attention tensor, a lip attention tensor and a skin attention tensor; wherein, a flexible maximum transfer function operation is performed on the eye shadow attention matrix to obtain a seventh matrix, and a matrix multiplication operation is performed on the seventh matrix and the second matrix to obtain an eye shadow attention tensor; a flexible maximum transfer function operation is performed on the lip attention matrix to obtain an eighth matrix, and a matrix multiplication operation is performed on the eighth matrix and the fourth matrix to obtain a lip attention tensor; a flexible maximum transfer function operation is performed on the skin attention matrix to obtain a ninth matrix, and a matrix multiplication operation is performed on the ninth matrix and the sixth matrix to obtain a skin attention tensor; The step of determining the migration makeup tensor according to the regional attention tensor and the original segmentation feature includes: Performing a convolution operation on the eye shadow attention tensor and the first convolution to obtain a first convolution operation result, and performing a matrix dot product calculation on the first convolution operation result and the first eye shadow feature to obtain an eye shadow makeup tensor; Performing a convolution operation on the lip attention tensor and the second convolution to obtain a second convolution operation result, and performing a matrix dot product calculation on the second convolution operation result and the first lip feature to obtain a lip makeup tensor; Performing a convolution operation on the skin attention tensor and the third convolution to obtain a third convolution operation result, and performing a matrix dot product calculation on the third convolution operation result and the first skin feature to obtain a skin makeup tensor; Performing a splicing operation on the eyeshadow makeup tensor, the lip makeup tensor, and the skin makeup tensor to obtain a migrated makeup tensor; The step of determining the transferred makeup facial features according to the transferred makeup tensor and the original facial features comprises: Performing a convolution operation on the migration makeup tensor and the fourth convolution to obtain a first region makeup tensor; Performing a convolution operation on the migrated makeup tensor and the fifth convolution to obtain a second region makeup tensor; Performing a matrix multiplication operation on the first region makeup tensor and the original facial features to obtain a tenth matrix; A matrix addition operation is performed on the tenth matrix and the second region makeup tensor to obtain the migrated makeup facial features.

2. A facial makeup transfer device, characterized in that: The facial makeup migration device includes: a memory, a processor, and a facial makeup migration program stored in the memory and runnable on the processor. When the facial makeup migration program is executed by the processor, the steps of the facial makeup migration method according to claim 1 are implemented.

3. A computer-readable storage medium, characterized in that The computer-readable storage medium stores a facial makeup migration program, which, when executed by a processor, implements the steps of the facial makeup migration method according to claim 1.

Images