Image processing method, device, electronic device and storage medium

Abstract

The present disclosure relates to an image processing method, apparatus, electronic device and storage medium, wherein the method comprises: acquiring an initial image; determining a target style intensity value of the initial image in response to an operation instruction of a target object, wherein the target style intensity value is the similarity between the image style corresponding to the processed image and a preset style; inputting the initial image and the target style intensity value into a target image processing model for analysis to obtain a target image corresponding to the target style intensity value, wherein the target image processing model is jointly determined based on a first image generation model and a second image generation model, wherein the first image generation model is an image generation model for generating an image with minimum style intensity, and the second image generation model is an image generation model for generating an image with maximum style intensity.

Description

Technical Field

[0001] The present disclosure relates to the field of machine learning, and in particular to an image processing method, device, electronic device, and storage medium. Background Art

[0002] In related technologies, generative neural network-based portrait stylization effects are widely used in scenarios such as short video shooting, live streaming, and image beautification. Typically, these stylization effects produce a fixed effect, but different users have varying degrees of acceptance for the intensity of the stylization. Due to the poor interpretability of deep neural networks, generative neural network-based portrait stylization effects cannot easily control the intensity of the stylization, unlike traditional filtering-based stylization effects, when faced with diverse needs.

[0003] While generative network technology for generating fixed effects is mature, allowing for straightforward training of multiple networks to support stylization of varying intensities, mobile devices, due to limited memory, cannot load too many models at once. This requires constant release and loading of models as users adjust the intensity, adding an additional burden. Furthermore, this solution can only preset a limited number of stylization intensities. Summary of the Invention

[0004] The present disclosure provides an image processing method, apparatus, electronic device, and storage medium to at least address the problem of generative neural network intelligence producing fixed effects in related technologies. The technical solutions of the present disclosure are as follows:

[0005] According to a first aspect of an embodiment of the present disclosure, there is provided an image processing method, comprising: acquiring an initial image; determining a target style intensity value of the initial image in response to an operation instruction of a target object, wherein the target style intensity value is a similarity between an image style corresponding to the processed image and a preset style; inputting the initial image and the target style intensity value into a target image processing model for analysis to obtain a target image corresponding to the target style intensity value, wherein the target image processing model is jointly determined based on a first image generation model and a second image generation model, the first image generation model being an image generation model for generating an image with minimum style intensity, and the second image generation model being an image generation model for generating an image with maximum style intensity.

[0006] Optionally, when the initial image includes a head portrait of the target object, the initial image and the target style intensity value are input into a target image processing model for analysis, and the step of obtaining a target image corresponding to the target style intensity value includes: determining multiple image features in the initial image, wherein the image features are used to characterize the head portrait attributes of the target object; based on the target style intensity value, adjusting each image feature of the multiple image features to obtain a target image feature corresponding to each image feature; and generating a target image based on the multiple target image features.

[0007] Optionally, based on the target style strength value, adjusting each image feature in a plurality of image features to obtain a target image feature corresponding to each image feature includes: determining the style codes corresponding to the first image generation model and the second image generation model respectively, wherein the style codes are the weights of the feature maps generated by each convolutional layer in the corresponding image generation model when splicing; determining the target style code corresponding to the target style strength value based on the style codes corresponding to the first image generation model and the second image generation model respectively, and adjusting each image feature in a plurality of image features based on the target style code to obtain a target image feature corresponding to each image feature.

[0008] Optionally, the target image processing model is implemented in the following manner: determining a training image set whose image style is a preset style, and training a first image generation model based on the training image set to obtain a second image generation model; based on the first image generation model and the second image generation model, determining a training image set, and training an image processing model based on the training image set to obtain a target image processing model; wherein the input of the target image processing model is an initial image and a target style intensity value, and the output is a target image corresponding to the target style intensity value, and the target style intensity value is the similarity between the image style corresponding to the target image and the preset style.

[0009] Optionally, a training image set is determined based on the first image generation model and the second image generation model, and the image processing model is trained based on the training image set to obtain a target image processing model. The steps include: determining a third image generation model based on the first image generation model and the second image generation model, wherein the image style intensity corresponding to the image generated by the third image generation model is a target style intensity value; and training the image processing model based on the first image generation model and the third image generation model to obtain a target image processing model.

[0010] Optionally, based on the first image generation model and the second image generation model, the step of determining the third image generation model includes: determining the first weight corresponding to each convolutional layer in the first image generation model and the second weight corresponding to each convolutional layer in the second image generation model based on the target style strength, wherein the first image generation model and the second image generation model have the same number of convolutional layers, and the sum of the first weight and the second weight corresponding to the same convolutional layer in the first image generation model and the second image generation model is 1; determining the style coding of the third image generation model based on the first weight, the second weight and the style coding corresponding to the first image generation model and the second image generation model, wherein the style coding is the convolutional layer parameters of each convolutional layer in the corresponding image generation model; and determining the third image generation model based on the style coding.

[0011] Optionally, the image processing model is trained based on the first image generation model and the third image generation model to obtain the target image processing model, including the steps of: inputting target noise data into the first image generation model to obtain a first image, wherein the target noise data is noise data obtained by random sampling; inputting the target noise data into the third image generation model to obtain a second image; and training the image processing model using the first image and the target style intensity value as input and the second image as the expected output result to obtain the target image processing model.

[0012] According to a second aspect of an embodiment of the present disclosure, an image processing device is provided, comprising: a first receiving unit, configured to acquire an initial image; a second receiving unit, configured to execute an operation instruction in response to a target object, and determine a target style intensity value of the initial image, wherein the target style intensity value is a similarity between an image style corresponding to the processed image and a preset style; and a processing unit, configured to input the initial image and the target style intensity value into a target image processing model for analysis, and obtain a target image corresponding to the target style intensity value, wherein the target image processing model is jointly determined based on a first image generation model and a second image generation model, the first image generation model being an image generation model for generating an image with minimum style intensity, and the second image generation model being an image generation model for generating an image with maximum style intensity.

[0013] Optionally, when the initial image includes a head portrait of the target object, the processing unit is further configured to perform: determining multiple image features in the initial image, wherein the image features are used to characterize the head portrait attributes of the target object; adjusting each of the multiple image features based on the target style strength value to obtain a target image feature corresponding to each image feature; and generating a target image based on the multiple target image features.

[0014] Optionally, the processing unit is further configured to perform: determining the style codes corresponding to the first image generation model and the second image generation model respectively, wherein the style codes are the weights of the feature maps generated by each convolutional layer in the corresponding image generation model when splicing; determining the target style code corresponding to the target style intensity value based on the style codes corresponding to the first image generation model and the second image generation model respectively, and adjusting each image feature of the multiple image features based on the target style code to obtain the target image feature corresponding to each image feature.

[0015] Optionally, the processing unit is further configured to perform: determining a training image set whose image style is a preset style, and training a first image generation model based on the training image set to obtain a second image generation model; determining a training image set based on the first image generation model and the second image generation model, and training an image processing model based on the training image set to obtain a target image processing model; wherein the input of the target image processing model is an initial image and a target style intensity value, and the output is a target image corresponding to the target style intensity value, and the target style intensity value is the similarity between the image style corresponding to the target image and the preset style.

[0016] Optionally, the processing unit is further configured to perform: determining a third image generation model based on the first image generation model and the second image generation model, wherein the image style intensity corresponding to the image generated by the third image generation model is a target style intensity value; and training the image processing model based on the first image generation model and the third image generation model to obtain a target image processing model.

[0017] Optionally, the processing unit is further configured to perform: based on the target style strength, determining the first weight corresponding to each convolutional layer in the first image generation model, and the second weight corresponding to each convolutional layer in the second image generation model, wherein the first image generation model and the second image generation model have the same number of convolutional layers, and the sum of the first weight and the second weight corresponding to the same convolutional layer in the first image generation model and the second image generation model is 1; based on the first weight, the second weight and the style codes corresponding to the first image generation model and the second image generation model, determining the style code of the third image generation model, wherein the style code is the convolutional layer parameters of each convolutional layer in the corresponding image generation model; and determining the third image generation model based on the style code.

[0018] Optionally, the processing unit is further configured to perform: inputting the target noise data into the first image generation model to obtain a first image, wherein the target noise data is noise data obtained by random sampling; inputting the target noise data into the third image generation model to obtain a second image; training the image processing model using the first image and the target style intensity value as input and the second image as the expected output result to obtain a target image processing model.

[0019] According to a third aspect of an embodiment of the present disclosure, there is provided an image processing electronic device, comprising: a processor; and a memory for storing processor-executable instructions; wherein the processor is configured to execute the instructions to implement an image processing method.

[0020] According to a fourth aspect of an embodiment of the present disclosure, a computer-readable storage medium is provided. When instructions in the computer-readable storage medium are executed by a processor of an image processing electronic device, the image processing electronic device is enabled to perform an image processing method.

[0021] According to a fifth aspect of an embodiment of the present disclosure, a computer program product is provided, comprising a computer program, which implements an image processing method when executed by a processor.

[0022] The technical solutions provided by the embodiments of the present disclosure bring at least the following beneficial effects:

[0023] In an embodiment of the present invention, an initial image is obtained; in response to an operation instruction of a target object, a target style intensity value of the initial image is determined, wherein the target style intensity value is the similarity between the image style corresponding to the processed image and the preset style; the initial image and the target style intensity value are input into a target image processing model for analysis to obtain a target image corresponding to the target style intensity value, wherein the target image processing model is determined based on a first image generation model and a second image generation model, wherein the first image generation model is an image generation model that generates an image with minimum style intensity, and the second image generation model is an image generation model that generates an image with maximum style intensity. The target image generation model is determined by two image generation models that generate images with different style intensities, thereby achieving the purpose of generating images with different style intensities through the target image generation model.

[0024] It is to be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the disclosure. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] The accompanying drawings herein are incorporated into and constitute a part of the specification, illustrate embodiments consistent with the present disclosure, and together with the description are used to explain the principles of the present disclosure, and do not constitute an improper limitation of the present disclosure.

[0026] Figure 1 The figure is a flowchart of an image processing method according to an exemplary embodiment.

[0027] Figure 2 The figure is a schematic diagram showing an image processing process according to an exemplary embodiment.

[0028] Figure 3 The figure is a flowchart of an image processing model training method according to an exemplary embodiment.

[0029] Figure 4 The figure is a schematic diagram of an image processing model training process according to an exemplary embodiment.

[0030] Figure 5 is a block diagram of an image processing apparatus according to an exemplary embodiment.

[0031] Figure 6 It is a block diagram of an electronic device according to an exemplary embodiment. DETAILED DESCRIPTION

[0032] In order to enable ordinary persons in the art to better understand the technical solutions of the present disclosure, the technical solutions in the embodiments of the present disclosure will be clearly and completely described below with reference to the accompanying drawings.

[0033] It should be noted that the terms "first," "second," and the like in the specification and claims of the present disclosure and the above-mentioned drawings are used to distinguish similar objects and are not necessarily used to describe a specific order or precedence. It should be understood that the numbers used in this manner are interchangeable where appropriate so that the embodiments of the present disclosure described herein can be implemented in an order other than those illustrated or described herein. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the present disclosure. Instead, they are merely examples of apparatus and methods consistent with certain aspects of the present disclosure as detailed in the appended claims.

[0034] Since the image processing method provided in this application only needs to use one image processing model when processing an image, it can process the original image to obtain a target image of different styles. Therefore, the image processing method provided in this application can be applied to common smart terminal devices (such as smart phones, tablets, etc.) and can also be applied to servers.

[0035] Figure 1 is a flowchart of an image processing method according to an exemplary embodiment. Figure 1 As shown, the image processing method is used in any terminal device or server and includes the following steps.

[0036] In step S11, an initial image is acquired;

[0037] In some examples of the present disclosure, a user may input a picture stored locally on a terminal device into a program that runs the above-mentioned image processing method, or the program itself may call a camera of the terminal device to acquire an image.

[0038] In some examples of the present disclosure, before the step of acquiring the initial image, the target image processing model needs to be trained based on the first image generation model and the second image generation model.

[0039] In some examples of the present disclosure, the first image generation model and the second image generation model may be StyleGAN models. The StyleGAN model can map the input noise into an image as required, and can obtain a StyleGAN model that generates images of different styles and types by selecting different training data sets. Specifically, each layer of convolution in the StyleGAN model has a certain semantic function, which can control the shape, position, color and other attributes of the facial features respectively, and the semantic function of each layer of convolution will not be destroyed during the fine-tuning process. Therefore, the weights of the two StyleGAN models can be mixed to obtain a natural and continuous interpolation effect.

[0040] In some examples of the present disclosure, before the step of training the target image processing model based on the first image generation model and the second image generation model, the first image generation model and the second image generation model need to be adjusted. Specifically, it is first necessary to select a preset data set, such as the FFHQ (Flickr-Faces-HQ Dataset) data set to train the first image generation model, thereby obtaining a first image generation model that can generate portraits. Then, based on the style corresponding to the image to be generated, a set of training images with the preset style can be determined; finally, based on the training image set, the first image generation model is trained, and the trained first image generation model is used as the second image generation model.

[0041] In some examples of the present disclosure, by selecting a cloud-linked image set of a specific style type to train a first image generation model, a second image generation model can be generated that generates images of the style of the specific style type and has the highest style strength. Furthermore, by setting different coefficients for the first and second image generation models, image generation models of any style strength can be generated.

[0042] In some examples of the present disclosure, the step of training the target image processing model based on the first image generation model and the second image generation model includes: determining a third image generation model based on the first image generation model and the second image generation model, wherein the image style intensity corresponding to the image generated by the third image generation model is a target style intensity value; and training the image processing model based on the first image generation model and the third image generation model.

[0043] In some examples of the present disclosure, since the images generated by the first image generation model are images without obvious style tendencies, and the images generated by the second image generation model are all images of a preset style, it can be considered that the style intensity corresponding to the images generated by the first image generation model is 0, and the style intensity corresponding to the images generated by the second image generation model is 1. Then, based on the first image generation model and the second image generation model, the third image generation model is obtained, wherein the style intensity corresponding to the images generated by the third image generation model can be any value between 0 and 1.

[0044] In some examples of the present disclosure, in the process of training the first image generation model with a preset style training image set to obtain the second image generation model, in order to make the second image generation model more stable, a small learning rate can be selected to train the generator and discriminator of the first image generation model, and random data enhancement can be performed. For example, the learning rate can be selected to be no more than 2×10 -4 Any value of .

[0045] Specifically, the step of determining the third image generation model based on the first image generation model and the second image generation model includes: determining the first weight corresponding to each convolutional layer in the first image generation model and the second weight corresponding to each convolutional layer in the second image generation model based on the target style strength, wherein the first image generation model and the second image generation model have the same number of convolutional layers, and the sum of the first weight and the second weight corresponding to the same convolutional layer in the first image generation model and the second image generation model is 1; determining the style coding of the third image generation model based on the first weight, the second weight and the style coding corresponding to the first image generation model and the second image generation model, wherein the style coding is the convolutional layer parameters of each convolutional layer in the corresponding image generation model.

[0046] In some examples of the present disclosure, different convolutional layers in the image generation model process different image features. By setting the convolutional layer parameters of each convolutional layer in the image generation model, accurate modification of different features in the image can be achieved, so that the above-mentioned image generation model can generate images that meet user needs.

[0047] After obtaining a third image generation model that can generate a target style intensity picture, in some examples of the present disclosure, the step of training the image processing model based on the first image generation model and the third image generation model includes: inputting target noise data into the first image generation model to obtain a first image, wherein the target noise data is noise data obtained by random sampling; inputting the target noise data into the third image generation model to obtain a second image; and training the image processing model using the first image and the target style intensity value as input and the second image as the expected output result.

[0048] In some examples of the present disclosure, when training the above-mentioned image processing model, you can choose to use any loss function related to supervised image conversion to train the image processing model, such as LPIPS loss, L1 loss, and conditional GAN ​​loss.

[0049] In some examples of the present disclosure, the target image processing model obtained by the above method can generate a target image of any style intensity, and since the target image processing model itself is a lightweight model of linear programming, the target image processing model will not occupy too many computing resources and can be directly run in various mobile devices, such as smartphones, tablets, etc.

[0050] In step S12, in response to the operation instruction of the target object, a target style strength value of the initial image is determined, wherein the target style strength value is the similarity between the image style corresponding to the processed image and the preset style;

[0051] In some examples of the present disclosure, Figure 2 As shown, when a user wants to obtain an image with a target style intensity, the user can input the initial image to be processed into the program in an electronic device running an executable image processing method, and select the desired style intensity value through an interactive interface, wherein the above program will execute the image processing method provided in this application when running.

[0052] In step S13, the initial image and the target style intensity value are input into a target image processing model for analysis to obtain a target image corresponding to the target style intensity value, wherein the target image processing model is jointly determined based on the first image generation model and the second image generation model, the first image generation model is an image generation model that generates a minimum style intensity image, and the second image generation model is an image generation model that generates a maximum style intensity image.

[0053] In some examples of the present disclosure, when a user inputs an initial image and a desired style strength value into a program on an electronic device for executing the image processing method provided herein, the program first determines a corresponding style code based on the style strength value input by the user, and then determines various image features in the initial image. The program then determines, based on the style code, how to adjust the various image features in the initial image and how to fuse the adjusted image features to obtain a target image.

[0054] Specifically, when the initial image includes a head portrait of the target object, the initial image and the target style strength value are input into a target image processing model for analysis, and the step of obtaining a target image corresponding to the target style strength value includes: determining a plurality of image features in the initial image, wherein the image features are used to characterize the head portrait attributes of the target object; based on the target style strength value, adjusting each of the plurality of image features to obtain a target image feature corresponding to each image feature; and generating the target image based on the adjusted plurality of target image features.

[0055] In some examples of the present disclosure, the target object may be an animal, a human, or others.

[0056] In some examples of the present disclosure, when the target object is a person, the above-mentioned multiple image features include at least one of the following: body shape, face shape, facial feature shape, facial feature position, facial feature color, hair density, hair color, and skin color of the person in the image.

[0057] In some examples disclosed herein, the aforementioned image processing methods based on image features can accurately adjust each image feature in an initial image. This feature-based image adjustment approach allows the resulting target image to better meet the user's actual needs. Furthermore, by determining the degree of adjustment for each image feature based on a user-selected target style strength value, target images with diverse style strengths can be generated.

[0058] In some examples of the present disclosure, the target image processing model determines the style coding corresponding to the style strength value input by the user as follows: determine the style coding corresponding to the first image generation model and the second image generation model respectively, wherein the style coding is the weight of the feature maps generated by each convolutional layer in the corresponding image generation model when splicing; based on the style coding corresponding to the first image generation model and the second image generation model respectively, determine the target style coding corresponding to the target style strength value, and adjust each of the multiple image features based on the target style coding.

[0059] In some examples of the present disclosure, the style coding of the image processing model is determined based on the style coding of the first image generation model and the second image generation model. Different coefficients can be set for the style coding of the first image generation model and the second image generation model to obtain style coding of different image processing models, thereby achieving the generation of target images with different style intensities through the image processing model.

[0060] In some examples of the present disclosure, in order to reduce the resources consumed by the electronic device when running the image processing method, the above-mentioned image processing model can be a lightweight model using feature space linear style coding.

[0061] Figure 3 is a flowchart of an image processing model training method according to an exemplary embodiment. Figure 3 As shown, the following steps are included:

[0062] In step S31, a training image set having an image style of a preset style is determined, and a first image generation model is trained based on the training image set to obtain a second image generation model;

[0063] In step S32, a training image set is determined based on the first image generation model and the second image generation model, and the image processing model is trained according to the training image set to obtain a target image processing model, wherein the input of the target image processing model is the initial image and the target style intensity value, and the output is a target image corresponding to the target style intensity value, and the target style intensity value is the similarity between the image style corresponding to the target image and the preset style.

[0064] In some examples of the present disclosure, Figure 4 As shown, the above-mentioned steps of determining a training image set based on the first image generation model and the second image generation model, and training a target image processing model based on the training image set include: determining a third image generation model based on the first image generation model and the second image generation model, wherein the image style intensity corresponding to the image generated by the third image generation model is the target style intensity value; and training the image processing model based on the first image generation model and the third image generation model.

[0065] In some examples of the present disclosure, Figure 4 As shown, each set of training data in the above-mentioned training image set includes an input image generated by the first image generation model, a target image generated by the third image generation model, and a style code corresponding to the third image generation model. When training the target image processing model, the input of the model is the input image and the style code, and the target image is the expected output.

[0066] Specifically, in some examples of the present disclosure, based on the first image generation model and the second image generation model, the step of determining the third image generation model includes: determining the first weight corresponding to each convolutional layer in the first image generation model and the second weight corresponding to each convolutional layer in the second image generation model based on the target style strength, wherein the first image generation model and the second image generation model have the same number of convolutional layers, and the sum of the first weight and the second weight corresponding to the same convolutional layer in the first image generation model and the second image generation model is 1; determining the style coding of the third image generation model based on the first weight, the second weight and the style coding corresponding to the first image generation model and the second image generation model, wherein the style coding is the convolutional layer parameters of each convolutional layer in the corresponding image generation model.

[0067] In order to facilitate understanding of the above process of generating the third image generation model, the above process is further explained below with a specific example: First, a style strength value between 0 and 1 is randomly selected as the target style strength value (0 is the weakest and 1 is the strongest), and then the weights of the first image generation model and the second image generation model are determined based on the target style strength value when they are fused. Finally, based on the weight, a third image generation model that can generate an image with the target style strength value is obtained, and the corresponding style code is generated. Finally, the third image generation model is used to generate a stylized image and, together with the input image and style code generated by the first image generation model, a batch of training data is formed to train the lightweight generation network. Specifically, let the style strength be s and the pre-trained first image generation model be G 1 , the fine-tuned second image generation model is G 2 , we can use any S-shaped function to get the weight of the convolutional layers of the same number of layers in the first image generation model and the second image generation model when they are fused. Taking the sigmoid function as an example, the weight w i =sigmoid(2(is*16)), where i is the number of convolutional layers. Using this set of weights, the first image generation model and the second image generation model are weighted and summed to obtain the fused third image generation model G. 3 , where G 3 i =G 1 i *w i +G 2 i *(1-w i ).

[0068] In some examples of the present disclosure, the style code corresponding to the first image generation model is C1, the style code corresponding to the second image generation model is C2, and the target style strength is S. Then the style code corresponding to the third image generation model is C s=C1*(1-s)+C2*s.

[0069] In some examples of the present disclosure, the steps of determining the training image set based on the first image generation model and the third image generation model, and training the image processing model based on the training image set include: inputting target noise data into the first image generation model to obtain a first image, wherein the target noise data is noise data obtained by random sampling; inputting the target noise data into the third image generation model to obtain a second image; and training the image processing model using the first image and the target style intensity value as input and the second image as the expected output result.

[0070] Specifically, in some examples of the present disclosure, when training the target image processing model based on the first image generation model and the third image generation model, a batch of noise data z can be randomly sampled, and the noise data z can be converted into the input first image I by the first image generation model and the third image generation model respectively. i =G 1 (z) and the stylized second image I s =G 3 (z). {Ii, Is, Cs} constitutes a batch of training data, and the target image processing model G light Training is performed, where Cs is spliced ​​to G light The decoder of , so that it can generate an image of a specific style strength I s '=G light (I i ,C s ).

[0071] In some examples of the present disclosure, after the above training, the similarity between the image output by the target image processing model and the image generated by the third image generation model can reach a preset similarity threshold.

[0072] In some examples of the present disclosure, the trained target image processing model can also be tested. In the testing phase, the style strength s can be obtained from the user interface and the corresponding style code C can be interpolated. s . The user input image I i with C s Feed into the generative network G light The stylized result G of intensity s can be generated light (I i ,C s ).

[0073] In an embodiment of the present invention, an initial image is obtained; in response to an operation instruction of a target object, a target style intensity value of the initial image is determined, wherein the target style intensity value represents the similarity between the image style corresponding to the processed image and a preset style; the initial image and the target style intensity value are input into a target image processing model for analysis to obtain a target image corresponding to the target style intensity value, wherein the target image processing model is determined based on a first image generation model and a second image generation model, wherein the first image generation model is an image generation model that generates an image with minimum style intensity, and the second image generation model is an image generation model that generates an image with maximum style intensity. The target image generation model is determined by using two image generation models that generate images with different style intensities. Since only one generation network is used, no additional memory is added when outputting stylized results of different intensities, and interpolation is performed in the feature space to allow a natural transition without ghosting between stylized results of different intensities.

[0074] Figure 5 FIG. 1 is a block diagram of an image processing apparatus according to an exemplary embodiment. Figure 5 The device includes a first receiving unit 151, a second receiving unit 152 and a processing unit 153.

[0075] The first receiving unit 151 is configured to acquire an initial image;

[0076] The second receiving unit 152 is configured to execute an operation instruction in response to the target object and determine a target style strength value of the initial image, wherein the target style strength value is a similarity between an image style corresponding to the processed image and a preset style;

[0077] The processing unit 153 is configured to input the initial image and the target style intensity value into a target image processing model for analysis to obtain a target image corresponding to the target style intensity value, wherein the target image processing model is determined based on a first image generation model and a second image generation model, the first image generation model is an image generation model that generates an image with minimum style intensity, and the second image generation model is an image generation model that generates an image with maximum style intensity.

[0078] In some examples of the present disclosure, when the initial image includes a head portrait of the target object, the processing unit 153 is further configured to perform: determining multiple image features in the initial image, wherein the image features are used to characterize the head portrait attributes of the target object; adjusting each of the multiple image features based on the target style strength value to obtain a target image feature corresponding to each image feature; and generating a target image based on the multiple target image features.

[0079] In some examples of the present disclosure, the processing unit 153 is further configured to perform: determining the style codes corresponding to the first image generation model and the second image generation model, wherein the style codes are the weights of the feature maps generated by the convolutional layers in the corresponding image generation models when they are spliced; determining the target style codes corresponding to the target style intensity values ​​based on the style codes corresponding to the first image generation model and the second image generation model, and adjusting each of the multiple image features based on the target style codes to obtain the target image features corresponding to each image feature.

[0080] In some examples of the present disclosure, the processing unit 153 is further configured to perform: determining a training image set whose image style is a preset style, and training the first image generation model based on the training image set to obtain a second image generation model; determining a training image set based on the first image generation model and the second image generation model, and training the image processing model based on the training image set to obtain a target image processing model; wherein the input of the target image processing model is an initial image and a target style intensity value, and the output is a target image corresponding to the target style intensity value, and the target style intensity value is the similarity between the image style corresponding to the target image and the preset style.

[0081] In some examples of the present disclosure, the processing unit 153 is further configured to perform: determining a third image generation model based on the first image generation model and the second image generation model, wherein the image style intensity corresponding to the image generated by the third image generation model is a target style intensity value; and training the image processing model based on the first image generation model and the third image generation model to obtain a target image processing model.

[0082] In some examples of the present disclosure, the processing unit 153 is further configured to perform: determining, based on the target style strength, a first weight corresponding to each convolutional layer in the first image generation model, and a second weight corresponding to each convolutional layer in the second image generation model, wherein the first image generation model and the second image generation model have the same number of convolutional layers, and the sum of the first weight and the second weight corresponding to the same convolutional layer in the first image generation model and the second image generation model is 1; determining the style coding of the third image generation model based on the first weight, the second weight and the style coding corresponding to the first image generation model and the second image generation model, wherein the style coding is the convolutional layer parameters of each convolutional layer in the corresponding image generation model; and determining the third image generation model based on the style coding.

[0083] In some examples of the present disclosure, the processing unit 153 is also configured to perform: inputting the target noise data into the first image generation model to obtain a first image, wherein the target noise data is noise data obtained by random sampling; inputting the target noise data into the third image generation model to obtain a second image; training the image processing model using the first image and the target style intensity value as input and the second image as the expected output result to obtain a target image processing model.

[0084] Regarding the apparatus in the above embodiment, the specific manner in which each module performs operations has been described in detail in the embodiment of the method, and will not be elaborated here.

[0085] Figure 6 FIG is a block diagram of an electronic device 800 for executing the above-mentioned image processing method according to an exemplary embodiment. Figure 6 As shown, the electronic device 800 includes a processor 820; a memory 804 for storing instructions executable by the processor; wherein the processor is configured to execute the instructions to implement the following image processing method: obtaining an initial image; determining a target style intensity value of the initial image in response to an operation instruction of the target object, wherein the target style intensity value is the similarity between the image style corresponding to the processed image and the preset style; inputting the initial image and the target style intensity value into a target image processing model for analysis to obtain a target image corresponding to the target style intensity value, wherein the target image processing model is determined based on a first image generation model and a second image generation model, the first image generation model is an image generation model for generating an image with minimum style intensity, and the second image generation model is an image generation model for generating an image with maximum style intensity.

[0086] In an exemplary embodiment, a computer-readable storage medium including instructions is further provided, such as a memory 804 including instructions, and the instructions can be executed by the processor 820 of the apparatus 800 to perform the above method. Alternatively, the storage medium can be a non-transitory computer-readable storage medium, such as a ROM, a random access memory (RAM), a CD-ROM, a magnetic tape, a floppy disk, an optical data storage device, and the like.

[0087] In an exemplary embodiment, a computer program product is also provided, including a computer program / instruction, characterized in that when the computer program / instruction is executed by a processor, the following image processing method is implemented: obtaining an initial image; determining a target style intensity value of the initial image in response to an operation instruction of the target object, wherein the target style intensity value is the similarity between the image style corresponding to the processed image and the preset style; inputting the initial image and the target style intensity value into a target image processing model for analysis to obtain a target image corresponding to the target style intensity value, wherein the target image processing model is determined based on a first image generation model and a second image generation model, the first image generation model is an image generation model for generating an image with minimum style intensity, and the second image generation model is an image generation model for generating an image with maximum style intensity.

[0088] Other embodiments of the present disclosure will readily occur to those skilled in the art after considering the specification and practicing the invention disclosed herein. This application is intended to cover any variations, uses, or adaptations of the present disclosure that follow the general principles of the present disclosure and include common knowledge or customary techniques in the art not disclosed herein. The description and examples are to be considered as exemplary only, with the true scope and spirit of the present disclosure being indicated by the following claims.

[0089] It should be understood that the present disclosure is not limited to the exact structures that have been described above and shown in the drawings, and that various modifications and changes can be made without departing from the scope thereof. The scope of the present disclosure is limited only by the appended claims.

Claims

1. An image processing method, characterized in that: include: Get the initial image; In response to an operation instruction of the target object, determining a target style strength value of the initial image, wherein the target style strength value is a similarity between an image style corresponding to the processed image and a preset style; The initial image and the target style intensity value are input into a target image processing model for analysis to obtain a target image corresponding to the target style intensity value, wherein the target image processing model is jointly determined based on a first image generation model and a second image generation model, including: determining the respective weights of the first image generation model and the second image generation model when fusing them based on the target style intensity value, and finally obtaining a third image generation model that can generate an image with the target style intensity value based on the weights, and training the image processing model based on the first image generation model and the third image generation model to obtain the target image processing model, wherein the first image generation model is an image generation model that generates an image with minimum style intensity, and the second image generation model is an image generation model that generates an image with maximum style intensity.

2. The image processing method according to claim 1, wherein: When the initial image includes a head portrait of a target object, the step of inputting the initial image and the target style intensity value into a target image processing model for analysis to obtain a target image corresponding to the target style intensity value includes: Determining a plurality of image features in the initial image, wherein the image features are used to characterize attributes of the avatar of the target object; Based on the target style strength value, adjusting each image feature of the plurality of image features to obtain a target image feature corresponding to each image feature; The target image is generated based on the plurality of target image features.

3. The image processing method according to claim 2, wherein: The step of adjusting each of the plurality of image features based on the target style strength value to obtain a target image feature corresponding to each image feature comprises: Determining style codes corresponding to the first image generation model and the second image generation model, respectively, wherein the style codes are weights of feature maps generated by each convolutional layer in the corresponding image generation model when they are concatenated; Based on the style codes corresponding to the first image generation model and the second image generation model respectively, the target style code corresponding to the target style intensity value is determined, and each of the multiple image features is adjusted based on the target style code to obtain the target image feature corresponding to each image feature.

4. The image processing method according to claim 1, wherein: The target image processing model is implemented in the following way: Determining a training image set whose image style is a preset style, and training a first image generation model based on the training image set to obtain a second image generation model; Based on the first image generation model and the second image generation model, a training image set is determined, and the image processing model is trained according to the training image set to obtain a target image processing model; wherein the input of the target image processing model is an initial image and a target style intensity value, and the output is a target image corresponding to the target style intensity value, and the target style intensity value is the similarity between the image style corresponding to the target image and a preset style.

5. The image processing method according to claim 4, characterized in that The step of determining the third image generation model based on the first image generation model and the second image generation model includes: Determining, based on the target style strength, a first weight corresponding to each convolutional layer in the first image generation model and a second weight corresponding to each convolutional layer in the second image generation model, wherein the first image generation model and the second image generation model have the same number of convolutional layers, and the sum of the first weight and the second weight corresponding to the same convolutional layer in the first image generation model and the second image generation model is 1; determining a style code of the third image generation model based on the first weight, the second weight, and style codes corresponding to the first image generation model and the second image generation model, wherein the style code is a convolutional layer parameter of each convolutional layer in the corresponding image generation model; The third image generation model is determined according to the style code.

6. An image processing device, characterized in that include: A first receiving unit is configured to acquire an initial image; a second receiving unit configured to execute an operation instruction in response to the target object and determine a target style strength value of the initial image, wherein the target style strength value is a similarity between an image style corresponding to the processed image and a preset style; A processing unit is configured to input the initial image and the target style intensity value into a target image processing model for analysis to obtain a target image corresponding to the target style intensity value, wherein the target image processing model is jointly determined based on a first image generation model and a second image generation model, including: determining the respective weights of the first image generation model and the second image generation model when fusing based on the target style intensity value, and finally obtaining a third image generation model that can generate an image with the target style intensity value based on the weights, training the image processing model based on the first image generation model and the third image generation model to obtain the target image processing model, the first image generation model is an image generation model that generates an image with minimum style intensity, and the second image generation model is an image generation model that generates an image with maximum style intensity.

7. An image processing electronic device, characterized in that: include: processor; a memory for storing instructions executable by the processor; The processor is configured to execute the instructions to implement the image processing method according to any one of claims 1 to 5. 8 . A computer-readable storage medium, wherein when instructions in the computer-readable storage medium are executed by a processor of an image processing electronic device, the image processing electronic device is enabled to perform the image processing method according to claim 1 .

9. A computer program product comprising a computer program / instructions, characterized in that When the computer program / instructions are executed by a processor, the image processing method according to any one of claims 1 to 5 is implemented.

Images