Image processing method and device and electronic equipment

Abstract

The invention belongs to the field of computer vision, and provides an image processing method and device and electronic equipment. The method extracts a first feature of a first image; converting the first image into a second image according to the first feature; based on the second feature of the second image and the hairstyle feature corresponding to the third image with the target hairstyle, obtaining a fusion feature; and performing diffusion processing on the fusion feature to obtain a target image with the target hair style. The method can improve the generation efficiency and naturalness of the target image.

Description

Technical Field

[0001] The present application belongs to the field of computer vision, specifically to artificial intelligence technology, and in particular to an image processing method, device and electronic device. Background Art

[0002] At present, in the fields of short videos, movies, advertisements, e-commerce, etc., the hairstyle of the characters in the image can be changed through image processing technology to meet the user's requirements for the hairstyle of the characters in the image. For example, the user can use this technology to change the hairstyle, or obtain images of characters with different hairstyles, etc. However, in the related art, the user is usually required to manually operate the image processing software to transfer an existing hairstyle to the character in the image, resulting in low image processing efficiency. Summary of the invention

[0003] The present application provides an image processing method, device and electronic device to solve the technical problem of low image processing efficiency.

[0004] A first aspect of an embodiment of the present application provides an image processing method, the method comprising: extracting a first feature of a first image; converting the first image into a second image based on the first feature; obtaining a fusion feature based on a second feature of the second image and a hairstyle feature corresponding to a third image having a target hairstyle; and performing diffusion processing on the fusion feature to obtain a target image having the target hairstyle.

[0005] According to an embodiment of the present application, extracting the first feature of the first image includes: using a feature encoding network in a coding model to encode the first image to obtain image coding; using a feature mapping network in the coding model to map the image coding to obtain the first feature.

[0006] According to an embodiment of the present application, converting the first image into a second image based on the first feature includes: using a preset model to convert the first feature into a third feature; based on a first random noise, diffusing the third feature to obtain a fourth feature; and decoding the fourth feature to obtain the second image.

[0007] According to an embodiment of the present application, the training process of the preset model includes: constructing training samples, the training samples including facial images with hairstyles and facial images without hairstyles; based on the facial images with hairstyles, obtaining predicted images using a preset network; based on the facial images without hairstyles and the predicted images, determining a loss value of the preset network; based on the loss value, adjusting network parameters of the preset network to obtain the preset model.

[0008] According to an embodiment of the present application, the fusion feature is obtained based on the second feature of the second image and the hairstyle feature corresponding to the third image with the target hairstyle, including: performing self-attention analysis on the second feature to obtain a first attention feature; performing cross-attention analysis on the second feature and the hairstyle feature to obtain a second attention feature; and obtaining the fusion feature based on the first attention feature and the second attention feature.

[0009] According to an embodiment of the present application, the cross-attention analysis is performed on the second feature and the hairstyle feature to obtain the second attention feature, including: determining a query vector based on the second feature, and determining a key vector and a value vector based on the hairstyle feature; determining a weight coefficient of the value vector based on the query vector and the key vector; and performing a weighted operation on the value vector based on the weight coefficient to obtain the second attention feature.

[0010] According to an embodiment of the present application, the fusion feature is obtained based on the first attention feature and the second attention feature, including: performing weighted operation on the first attention feature and the second attention feature to obtain a third attention feature; performing nonlinear transformation processing on the third attention feature to obtain the fusion feature.

[0011] According to an embodiment of the present application, the diffusion processing of the fused feature to obtain a target image having the target hairstyle includes: denoising the fused feature based on a second random noise to obtain a fifth feature; upsampling the fifth feature to obtain a sixth feature; and decoding the sixth feature to obtain the target image.

[0012] According to a second aspect of an embodiment of the present application, there is provided an image processing device, comprising: an extraction unit for extracting a first feature of a first image; a conversion unit for converting the first image into a second image according to the first feature; a fusion unit for obtaining a fusion feature based on the second feature of the second image and a hairstyle feature corresponding to a third image having a target hairstyle; and a diffusion unit for performing diffusion processing on the fusion feature to obtain a target image having the target hairstyle.

[0013] A third aspect of an embodiment of the present application provides an electronic device, comprising: a memory storing computer-readable instructions; and a processor executing the computer-readable instructions stored in the memory to implement the image processing method.

[0014] A fourth aspect of an embodiment of the present application provides a computer-readable storage medium, in which computer-readable instructions are stored. The computer-readable instructions are executed by a processor in an electronic device to implement the image processing method.

[0015] In multiple embodiments of the present application, a target image with the target hairstyle can be generated by combining the first image and the third image with the target hairstyle. Since there is no need to manually process the image, the efficiency of generating the target image is improved. In addition, by performing feature fusion on the second feature of the second image and the hairstyle feature corresponding to the third image with the target hairstyle, the target hairstyle can be more naturally integrated with the person in the first image, thereby improving the naturalness of the target image. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] Figure 1 It is a schematic diagram of an application environment of an image processing method provided in an embodiment of the present application.

[0017] Figure 2 It is a flowchart of the image processing method provided in an embodiment of the present application.

[0018] Figure 3 It is a schematic diagram of the target image generation process provided in the embodiment of the present application.

[0019] Figure 4 It is a schematic diagram of obtaining a fusion feature based on the second feature and the hairstyle feature provided in an embodiment of the present application.

[0020] Figure 5 is a schematic diagram of a first image and a target image with a target hairstyle provided in an embodiment of the present application.

[0021] Figure 6 It is a flowchart of the training method of the preset model provided in the embodiment of the present application.

[0022] Figure 7 It is a functional module diagram of the image processing device provided in an embodiment of the present application.

[0023] Figure 8 It is a structural schematic diagram of an electronic device of an image processing method provided in an embodiment of the present application. DETAILED DESCRIPTION

[0024] In order to make the objectives, technical solutions and advantages of the present application clearer, the present application is described in detail below with reference to the accompanying drawings and specific embodiments.

[0025] It should be noted that in this application, "at least one" means one or more, and "more than one" means two or more than two. "And / or" describes the association relationship of associated objects, indicating that three relationships may exist. For example, A and / or B can mean: A exists alone, A and B exist at the same time, and B exists alone, where A and B can be singular or plural. The terms "first", "second", "third", "fourth", etc. (if any) in the specification, claims and drawings of this application are used to distinguish similar objects, rather than to describe a specific order or sequence.

[0026] In the embodiments of the present application, words such as "exemplary" or "for example" are used to indicate examples, illustrations or descriptions. Any embodiment or design described as "exemplary" or "for example" in the embodiments of the present application should not be interpreted as being more preferred or more advantageous than other embodiments or designs. Specifically, the use of words such as "exemplary" or "for example" is intended to present related concepts in a concrete way. The following embodiments and features in the embodiments may be combined with each other without conflict.

[0027] See also Figure 1 , Figure 1 A schematic diagram of an application environment of an image processing method provided in an embodiment of the present application. Figure 1 As shown, the user terminal 10 communicates with the server 20 through a network. The network can be a wired network communication or a wireless network communication. The wired network can be any one of a local area network, a metropolitan area network and a wide area network, a wireless fidelity (Wireless Fidelity, Wi-Fi), a self-organizing network wireless communication (ZigBee Wireless Networks, ZigBee) technology, an ultra-wideband (Ultra Wideband, UWB) technology, a wireless universal serial bus (Universal Serial Bus, USB) and the like.

[0028] The user terminal 10 may be an electronic device such as a mobile phone, a tablet computer, a multimedia player, a personal computer (PC), a wearable device, etc. The user terminal 10 may be a client with various applications installed, such as education applications, consulting applications, information broadcast applications, live broadcast applications, e-commerce applications, etc.

[0029] The server 20 is used to provide background services for the application in the user terminal 10. For example, the server 20 can be the background server of the above-mentioned e-commerce application. The server 20 can be a computer device. In one embodiment of the present application, it can be a server, a server cluster composed of multiple servers, or a cloud computing service center.

[0030] Exemplarily, an e-commerce application is used as an example for explanation. The user performs operations such as determining the first image and determining the target hairstyle in the client (for example, the user terminal 10) on which the e-commerce application is installed. The first image may indicate the user image whose hairstyle needs to be changed, and the target hairstyle may indicate the hairstyle after the change. The user terminal 10 sends the first image and the third image with the target hairstyle to the server 20; the server 20 changes the hairstyle of the first image based on the target hairstyle selected by the user to obtain an image with the target hairstyle. In this way, the server 20 can change the hairstyle of the first image according to the first image and the target hairstyle determined by the user, so that the target hairstyle is more naturally integrated with the character in the first image, thereby providing the user with a better image processing experience and facilitating the user to try out hairstyles online.

[0031] The following will be a computer program product running on an electronic device (such as Figure 8 The electronic device 100 shown in FIG. Figure 2 , is a flow chart of an image processing method provided by an embodiment of the present application. According to different requirements, the order of the steps in the flow chart can be changed, and some steps can be omitted. The image processing method provided by an embodiment of the present application includes the following steps.

[0032] S201, extracting a first feature of a first image.

[0033] In at least one embodiment of the present application, the first image may represent an image in which the user needs to change his hairstyle, and the object in the first image may be a full-body photo or head portrait of any person. Figure 3 As shown, the first image may be a head portrait of any person. The first image may be an image obtained by a user shooting an object with a shooting device, or the first image may be any frame of image captured from a pre-recorded video.

[0034] In at least one embodiment of the present application, the electronic device extracts a first feature of the first image, including: the electronic device encodes the first image using a feature encoding network in the encoding model to obtain an image code. The electronic device uses a feature mapping network in the encoding model to map the image code to obtain the first feature.

[0035] Among them, the prototype of the encoding model can also be an encoder in a variational auto-encoder (VAE), and the prototype of the encoding model can also be other types of encoders. The training method of the encoding model includes: obtaining training images and feature labels of the training images, processing the training images through a preset neural network to obtain prediction features, adjusting the network parameters of the preset neural network based on the prediction loss value of the prediction features and the feature labels until the prediction loss value meets the preset conditions, and using the adjusted preset neural network as the encoding model. The preset conditions include but are not limited to: the prediction loss value is within a preset range, and the preset range can be adjusted according to actual needs.

[0036] A coding table may be pre-configured in the feature coding network, and the coding table may include a mapping relationship between pixel values ​​and coding values, for example, the coding value corresponding to the pixel value [0, 127] is 0, and the coding value corresponding to the pixel value [128, 255] is 1; for another example, the coding value corresponding to the pixel value [0, 63] is 0, the coding value corresponding to the pixel value [64, 127] is 1, the coding value corresponding to the pixel value [128, 191] is 2, and the coding value corresponding to the pixel value [192, 255] is 3. When encoding the first image, the electronic device determines the pixel position and pixel value of each pixel in the first image, maps the pixel value of each pixel using the coding table in the feature coding network, obtains the coding value corresponding to each pixel, and obtains the image coding based on the pixel position of each pixel and the coding value corresponding to each pixel. For example, if the first image is a two-dimensional image, the image coding may be a two-dimensional matrix; if the first image is a three-dimensional image, the image coding may be a three-dimensional matrix.

[0037] The feature mapping network may include a multi-layer neural network. The feature mapping network may be a combination of a fully connected layer and a convolutional layer. The feature mapping network may also be a combination of a fully connected layer, a convolutional layer and other network layers. The feature mapping network may also include only a multi-layer fully connected layer, or the feature mapping network may also include only a multi-layer convolutional layer. When the image code is mapped, the image code may be feature extracted and mapped through the network layer in the feature mapping network to obtain the first feature.

[0038] The first feature may include key features of the object in the first image. For example, the first feature may include but is not limited to: the shape and color of the hairstyle of the object in the first image, the eye features and the nose features of the object in the first image. Figure 3 As shown, the first feature may be a feature output by the encoding model.

[0039] The embodiment of the present application can accurately encode the image code corresponding to the first image by utilizing the feature coding network in the coding model, and then use the feature mapping network to extract and map the features of the image code, thereby improving the extraction accuracy of the first feature.

[0040] S202: Convert the first image into a second image according to the first feature.

[0041] In at least one embodiment of the present application, the second image may be an image obtained by removing the hairstyle of the object in the first image, and the second image may be the first image without the hairstyle. Figure 3 As shown, the second image may be an image output from a pre-trained diffusion image.

[0042] In at least one embodiment of the present application, the electronic device converts the first image into the second image according to the first feature, including: the electronic device converts the first feature into a third feature using a preset model, diffuses the third feature based on the first random noise to obtain a fourth feature, and decodes the fourth feature to obtain the second image.

[0043] The prototype of the preset model can be a convolutional neural network model, and the prototype of the preset model can also be other neural network models. The training process of the preset model can be referred to below. Figure 6 The process of converting the first feature into the third feature by the preset model can also be referred to below Figure 6 Details in .

[0044] The first random noise may correspond to the image size of the first image, and the length of the first random noise may be proportional to the image size of the first image. For example, the larger the image size of the first image is, the longer the length of the first random noise is; and the smaller the image size of the first image is, the shorter the length of the first random noise is.

[0045] When the third feature is diffused, the first random noise is concatenated with the third feature to obtain a concatenated feature, and the concatenated feature is processed by a pre-trained diffusion model to obtain a fourth feature. The diffusion model may include but is not limited to: a convolution layer, a pooling layer, an upsampling layer, and an activation function.

[0046] When the fourth feature is decoded, the fourth feature is decoded by calling the decoding model corresponding to the encoding model to obtain the second image. If the prototype of the encoding model is the encoder in the VAE, the decoding model is the decoder in the VAE.

[0047] The embodiment of the present application converts the first feature through a preset model to obtain a deeper third feature, and diffuses the third feature through the first random noise to improve the anti-interference of the fourth feature, thereby improving the robustness of the second image. By calling the decoding model corresponding to the encoding model and decoding the fourth feature, the second image corresponding to the first image can be accurately obtained.

[0048] S203: Obtain a fusion feature based on the second feature of the second image and the hairstyle feature corresponding to the third image having the target hairstyle.

[0049] In at least one embodiment of the present application, the target hairstyle may represent a hairstyle desired by the user, and the third image with the target hairstyle may be a full-body photo or headshot of the subject with the target hairstyle. Figure 3 As shown, the third image may be a character portrait input or selected by the user on the user terminal.

[0050] In at least one embodiment of the present application, the second feature of the second image can also be obtained by encoding the second image through a coding model, for example Figure 3 As shown, the second feature may be a feature obtained after the encoding model encodes the second image.

[0051] In at least one embodiment of the present application, the hairstyle features corresponding to the third image having the target hairstyle can be extracted from the third image by a hairstyle extractor, the network structure of the hairstyle extractor can be a Unet structure, and the hairstyle extractor can be trained based on randomly generated training data. The present application embodiment does not limit the training method of the hairstyle extractor. Figure 3 As shown, the hairstyle feature may be a feature output by a hairstyle extractor.

[0052] In at least one embodiment of the present application, the electronic device obtains a fusion feature based on the second feature of the second image and the hairstyle feature corresponding to the third image having the target hairstyle, including: the electronic device performs self-attention analysis on the second feature to obtain the first attention feature, and performs cross-attention analysis on the second feature and the hairstyle feature to obtain the second attention feature. The electronic device obtains the fusion feature based on the first attention feature and the second attention feature.

[0053] The process of obtaining the first attention feature may include: performing self-attention analysis on the second feature through a self-attention mechanism to obtain the first attention feature.

[0054] The process of obtaining the second attention feature may include: determining a query vector according to the second feature, determining a key vector and a value vector according to the hairstyle feature, determining a weight coefficient of the value vector according to the query vector and the key vector, and performing a weighted operation on the value vector based on the weight coefficient to obtain the second attention feature. The process of determining the weight coefficient may include: determining a dot product result of the query vector and the key vector, and obtaining the weight coefficient by normalizing the dot product result.

[0055] The process of obtaining the fusion feature may include: performing a weighted operation on the first attention feature and the second attention feature to obtain a third attention feature, and performing a nonlinear transformation on the third attention feature to obtain the fusion feature. The weights corresponding to the first attention feature and the second attention feature may be set and adjusted according to actual needs.

[0056] See also Figure 4 , Figure 4 A schematic diagram of obtaining a fused feature based on a second feature and a hairstyle feature provided in an embodiment of the present application. The electronic device performs self-attention analysis using the second feature as a query vector, a key vector, and a value vector to obtain a first attention feature. A cross-attention analysis is performed using the second feature as a query vector and the hairstyle feature as a key vector and a value vector to obtain a second attention feature. A weighted operation is performed on the first attention feature and the second attention feature to obtain a third attention feature. The third attention feature is subjected to nonlinear transformation processing through a forward feedback network to obtain a fused feature.

[0057] The embodiment of the present application analyzes the second feature through the self-attention mechanism, and can obtain the correlation between different features in the second feature. By performing cross-attention analysis on the second feature and the hairstyle feature, it can obtain the correlation between the second feature and the hairstyle feature. Furthermore, by combining the first attention feature and the second attention feature, it can ensure that not only the correlation between different second features is included in the fused feature, but also the correlation between the second feature and the hairstyle feature is included in the fused feature.

[0058] S204, performing diffusion processing on the fused features to obtain a target image with a target hairstyle.

[0059] In at least one embodiment of the present application, the target image with the target hairstyle may represent an image after the hairstyle of the object in the first image is changed to the target hairstyle, for example Figure 3 The target image is shown.

[0060] In at least one embodiment of the present application, the electronic device performs diffusion processing on the fused feature to obtain a target image with a target hairstyle, including: the electronic device performs denoising processing on the fused feature based on the second random noise to obtain a fifth feature, performs upsampling processing on the fifth feature to obtain a sixth feature, and performs decoding processing on the sixth feature to obtain a target image. The length of the second random noise may be proportional to the image size of the first image, and the length of the second random noise may be equal to the length of the first random noise. The target image may be obtained by decoding the sixth feature through a decoding model corresponding to the encoding model.

[0061] The embodiment of the present application denoises the fused feature through the second random noise, which can eliminate the interference information in the fused feature, thereby improving the accuracy of the fifth feature. By upsampling the fifth feature, the resolution of the sixth feature can be improved, thereby improving the image quality of the target image.

[0062] See also Figure 5 As shown, Figure 5 A schematic diagram of a first image and a target image with a target hairstyle provided in an embodiment of the present application. A user can input the first position in the display interface of the user terminal as follows Figure 5 The first position may indicate the position of the image where the hairstyle needs to be changed. The user may enter the position in the second position of the display interface. Figure 5 The third image shown in FIG. 1 has a target hairstyle, and the second position may have a position where the image of the target hairstyle is located. The electronic device processes the first image and the third image based on the image processing method to obtain the following Figure 5 The target image with the target hairstyle is shown.

[0063] In multiple embodiments of the present application, a target image with a target hairstyle can be generated by combining a first image and a third image with a target hairstyle. Since there is no need to manually process the image, the efficiency of generating the target image is improved. In addition, by performing feature fusion on the second feature of the second image and the hairstyle feature corresponding to the third image with the target hairstyle, the target hairstyle can be more naturally integrated with the person in the first image, thereby improving the naturalness of the target image.

[0064] like Figure 6 As shown in FIG. 1 , it is a flow chart of the training method of the preset model provided in the embodiment of the present application. Figure 5 As shown, it specifically includes steps S601-S604:

[0065] S601, construct training samples.

[0066] In at least one embodiment of the present application, the training samples include facial images with hairstyles and facial images without hairstyles. Figure 3 As shown, a face image with a hairstyle can be Figure 3 As shown in the first image or the third image of FIG. , a facial image without a hairstyle can be as follows Figure 3 The second image of the face with hairstyle in each training sample corresponds to the face without hairstyle. The face without hairstyle can be obtained by photographing a bald user without a wig, and the face with hairstyle can be obtained by photographing a bald user wearing different wigs.

[0067] S602, based on the facial image with the hairstyle, obtain a predicted image using a preset network.

[0068] In at least one embodiment of the present application, the preset network includes an input layer, a convolution layer, an activation function layer, a pooling layer, a fully connected layer, and an output layer. The electronic device can encode the facial image with the hairstyle through the input layer to obtain a coding vector, extract features of the coding vector based on the convolution layer to obtain a convolution vector, normalize the convolution vector through the activation function layer to obtain a normalized vector, process the normalized vector through the pooling layer to obtain a pooling vector, process the pooling vector through the fully connected layer to obtain a fully connected vector, and map the fully connected vector through the output layer to obtain a predicted image.

[0069] S603, determining a loss value of a preset network based on the facial image without a hairstyle and the predicted image.

[0070] In at least one embodiment of the present application, a facial image without a hairstyle is grayed to obtain a first gray image, and a predicted image is grayed to obtain a second gray image, and a loss value is calculated based on the pixel difference between each pixel in the second gray image and the first gray image. The loss value can be a weighted sum of the pixel differences of each pixel, and the weights corresponding to different pixels can be the same, and the weights corresponding to pixels in different regions can also be different. For example, the weight of the pixel corresponding to the top of the head is greater than the weight of the pixel corresponding to the mouth region.

[0071] The embodiment of the present application performs grayscale processing on the facial image without hairstyle and the predicted image respectively, so as to eliminate the influence of light, thereby accurately quantifying the loss value. In addition, by setting different weights for pixels in different areas, the loss value can be quantified more accurately.

[0072] S604: Based on the loss value, adjust the network parameters of the preset network to obtain a preset model.

[0073] In at least one embodiment of the present application, the network parameters include but are not limited to: parameters in the input layer, convolution layer, activation function layer, pooling layer, fully connected layer and output layer. The electronic device adjusts the network parameters of the preset network until the loss value meets the preset requirements, and determines the adjusted preset network as the preset model. The preset requirements can be set and adjusted according to actual needs.

[0074] The embodiment of the present application performs training by using facial images with hairstyles and corresponding facial images without hairstyles, and then adjusts network parameters by determining the obtained loss values, thereby improving the training effect of the preset model.

[0075] like Figure 7 , which is a functional module diagram of an image processing device provided by an embodiment of the present application. The image processing device 11 includes an extraction unit 110, a conversion unit 111, a fusion unit 112, a diffusion unit 113, and a training unit 114. The module / unit referred to in the present application refers to a series of computer-readable instruction segments that can be acquired by the processor 23 and can perform fixed functions, which are stored in the memory 22.

[0076] In one embodiment, the extraction unit 110 is used to extract the first feature of the first image; the conversion unit 111 is used to convert the first image into a second image according to the first feature; the fusion unit 112 is used to obtain a fusion feature based on the second feature of the second image and the hairstyle feature corresponding to the third image having the target hairstyle; the diffusion unit 113 is used to perform diffusion processing on the fusion feature to obtain a target image having the target hairstyle.

[0077] In one embodiment, the extraction unit 110 is specifically used to: use the feature encoding network in the encoding model to encode the first image to obtain the image code; use the feature mapping network in the encoding model to map the image code to obtain the first feature.

[0078] In one embodiment, the conversion unit 111 is specifically used to: convert the first feature into a third feature using a preset model; diffuse the third feature based on the first random noise to obtain a fourth feature; and decode the fourth feature to obtain a second image.

[0079] In one embodiment, the training unit 114 is used to construct training samples, which include facial images with hairstyles and facial images without hairstyles; based on the facial images with hairstyles, a predicted image is obtained using a preset network; based on the facial images without hairstyles and the predicted image, a loss value of the preset network is determined; based on the loss value, a network parameter of the preset network is adjusted to obtain a preset model.

[0080] In one embodiment, the fusion unit 112 is specifically used to: perform self-attention analysis on the second feature to obtain a first attention feature; perform cross-attention analysis on the second feature and the hairstyle feature to obtain a second attention feature; and obtain a fusion feature based on the first attention feature and the second attention feature.

[0081] In one embodiment, the fusion unit 112 is specifically used to: determine a query vector based on the second feature, and determine a key vector and a value vector based on the hairstyle feature; determine a weight coefficient of the value vector based on the query vector and the key vector; and perform a weighted operation on the value vector based on the weight coefficient to obtain a second attention feature.

[0082] In one embodiment, the fusion unit 112 is specifically used to: perform weighted operation on the first attention feature and the second attention feature to obtain a third attention feature; and perform nonlinear transformation processing on the third attention feature to obtain a fused feature.

[0083] In one embodiment, the diffusion unit 113 is specifically used to: perform denoising processing on the fused feature based on the second random noise to obtain a fifth feature; perform upsampling processing on the fifth feature to obtain a sixth feature; and perform decoding processing on the sixth feature to obtain a target image.

[0084] In multiple embodiments of the present application, a target image with a target hairstyle can be generated by combining a first image and a third image with a target hairstyle. Since there is no need to manually process the image, the efficiency of generating the target image is improved. In addition, by performing feature fusion on the second feature of the second image and the hairstyle feature corresponding to the third image with the target hairstyle, the target hairstyle can be more naturally integrated with the person in the first image, thereby improving the naturalness of the target image.

[0085] like Figure 8 , is a schematic diagram of the structure of an electronic device of an image processing method provided in an embodiment of the present application.

[0086] In an embodiment of the present application, the image processing method is applied to one or more electronic devices 100. The electronic device 100 is a device that can execute computer-readable instructions to automatically perform numerical calculations and / or information processing, and its hardware includes but is not limited to a microprocessor, an application specific integrated circuit (ASIC), a programmable gate array (FPGA), a digital signal processor (DSP), an embedded device, etc.

[0087] The electronic device 100 can be any electronic product that can interact with a user, such as a personal computer, a tablet computer, a smart phone, a personal digital assistant (PDA), a game console, an interactive network television (IPTV), a smart wearable device, etc.

[0088] The electronic device 100 may include a network device and / or a user device, wherein the network device includes but is not limited to a single network electronic device, an electronic device group consisting of multiple network electronic devices, or a cloud consisting of a large number of hosts or network electronic devices based on cloud computing.

[0089] The network where the electronic device 100 is located includes, but is not limited to: the Internet, a wide area network, a metropolitan area network, a local area network, a virtual private network (VPN), etc.

[0090] In the embodiment of the present application, the electronic device 100 includes, but is not limited to, a memory 22, a processor 23, and computer-readable instructions stored in the memory 22 and executable on the processor 23, such as an image processing program.

[0091] Those skilled in the art will understand that the schematic diagram is merely an example of the electronic device 100 and does not constitute a limitation of the electronic device 100. The electronic device 100 may include more or fewer components than shown in the figure, or a combination of certain components, or different components. For example, the electronic device 100 may also include input and output devices, network access devices, buses, etc.

[0092] The processor 23 may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or a processor or any conventional processor, etc. The processor 23 is the computing core and control center of the electronic device 100, and uses various interfaces and lines to connect various parts of the entire electronic device 100, and executes the operating system of the electronic device 100 and various installed applications, program codes, etc.

[0093] The memory 22 may be an external memory and / or an internal memory of the electronic device 100. Furthermore, the memory 22 may be a memory in a physical form, such as a memory stick, a TF card (Trans-flash Card), and the like.

[0094] Combination Figure 2 , Figure 6 The memory 22 in the electronic device 100 stores computer-readable instructions, and the processor 23 can execute the computer-readable instructions stored in the memory 22 to implement the following Figure 2 , Figure 6 The multiple processes shown are used to implement the image processing method.

[0095] If the module / unit integrated in the electronic device 100 is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the present application implements all or part of the processes in the above-mentioned embodiment method, and can also instruct the relevant hardware to complete it through computer-readable instructions. The computer-readable instructions can be stored in a computer-readable storage medium, and when the computer-readable instructions are executed by the processor, the steps of the above-mentioned various method embodiments can be implemented.

[0096] The computer-readable instructions include computer-readable instruction codes, which may be in source code form, object code form, executable files, or some intermediate form, etc. The computer-readable medium may include: any entity or device capable of carrying the computer-readable instruction codes, recording medium, USB flash drive, mobile hard disk, magnetic disk, optical disk, computer memory, read-only memory (ROM), random access memory (RAM).

[0097] Specifically, the specific implementation method of the processor 23 for the above-mentioned computer readable instructions can be referred to Figure 2 , Figure 6 The description of the relevant steps in the corresponding embodiments will not be repeated here.

[0098] In the several embodiments provided in this application, it should be understood that the disclosed systems, devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic, for example, the division of modules is only a logical function division, and there may be other division methods in actual implementation.

[0099] The modules described as separate components may or may not be physically separated, and the components shown as modules may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0100] In addition, each functional module in each embodiment of the present application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The above integrated unit can be implemented in the form of hardware or in the form of hardware plus software functional modules.

[0101] Therefore, no matter from which point of view, the embodiments should be regarded as illustrative and non-restrictive, and the scope of the present application is limited by the appended claims rather than the above description, so it is intended that all changes falling within the meaning and scope of the equivalent elements of the claims are included in the present application. Any attached figure mark in the claims should not be regarded as limiting the claims involved.

[0102] In addition, it is obvious that the word "comprising" does not exclude other units or steps, and the singular does not exclude the plural. Multiple units or devices can also be implemented by one unit or device through software or hardware. The words first, second, etc. are used to indicate names, and do not indicate any specific order.

[0103] Finally, it should be noted that the above embodiments are only used to illustrate the technical solution of the present application and are not intended to limit it. Although the present application has been described in detail with reference to the preferred embodiments, a person of ordinary skill in the art should understand that the technical solution of the present application may be modified or replaced by equivalents without departing from the spirit and scope of the technical solution of the present application.

Claims

1. An image processing method, characterized in that: The method comprises: extracting a first feature of the first image; converting the first image into a second image according to the first feature; Obtaining a fusion feature based on the second feature of the second image and the hairstyle feature corresponding to the third image having the target hairstyle; The fused features are diffused to obtain a target image having the target hairstyle.

2. The image processing method according to claim 1, characterized in that: The extracting the first feature of the first image includes: Encoding the first image using a feature encoding network in the encoding model to obtain an image code; The image coding is mapped using a feature mapping network in the coding model to obtain the first feature.

3. The image processing method according to claim 1, characterized in that: The converting the first image into a second image according to the first feature includes: Using a preset model, converting the first feature into a third feature; Based on the first random noise, the third feature is diffused to obtain a fourth feature; The fourth feature is decoded to obtain the second image.

4. The image processing method according to claim 3, characterized in that: The training process of the preset model includes: Constructing training samples, wherein the training samples include facial images with hairstyles and facial images without hairstyles; Based on the facial image with the hairstyle, obtaining a predicted image using a preset network; Determining a loss value of the preset network based on the facial image without the hairstyle and the predicted image; Based on the loss value, the network parameters of the preset network are adjusted to obtain the preset model.

5. The image processing method according to claim 1, characterized in that: The obtaining of the fusion feature based on the second feature of the second image and the hairstyle feature corresponding to the third image having the target hairstyle comprises: Performing self-attention analysis on the second feature to obtain a first attention feature; Performing cross-attention analysis on the second feature and the hairstyle feature to obtain a second attention feature; Based on the first attention feature and the second attention feature, the fusion feature is obtained.

6. The image processing method according to claim 5, characterized in that: The cross-attention analysis is performed on the second feature and the hairstyle feature to obtain the second attention feature, including: Determine a query vector according to the second feature, and determine a key vector and a value vector according to the hairstyle feature; Determining a weight coefficient of the value vector according to the query vector and the key vector; Based on the weight coefficient, a weighted operation is performed on the value vector to obtain the second attention feature.

7. The image processing method according to claim 5, characterized in that: The obtaining the fusion feature based on the first attention feature and the second attention feature includes: Performing a weighted operation on the first attention feature and the second attention feature to obtain a third attention feature; The third attention feature is subjected to nonlinear transformation processing to obtain the fusion feature.

8. The image processing method according to claim 1, characterized in that: The step of performing diffusion processing on the fused features to obtain a target image having the target hairstyle includes: Based on the second random noise, denoising the fused feature to obtain a fifth feature; Performing upsampling processing on the fifth feature to obtain a sixth feature; The sixth feature is decoded to obtain the target image.

9. An image processing device, characterized in that: The device comprises: An extraction unit, configured to extract a first feature of a first image; a conversion unit, configured to convert the first image into a second image according to the first feature; a fusion unit, configured to obtain a fusion feature based on the second feature of the second image and the hairstyle feature corresponding to the third image having the target hairstyle; The diffusion unit is used to perform diffusion processing on the fused features to obtain a target image with the target hairstyle.

10. An electronic device, characterized in that: include: A memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the image processing method according to any one of claims 1 to 8 when executing the computer program.

Images