Image processing method and electronic equipment

Abstract

The invention provides an image processing method and electronic equipment, and relates to the field of image processing. The invention provides an image processing method, comprising: acquiring a first image and first prompt information input by a user, the first prompt information comprising text information of a target style; the first image and the first prompt information are input into a first model to obtain a reference image of the first image, the image style of the reference image is the same as the target style, and the image style comprises at least one of color features and texture features of the image; the first image and the reference image are input into the second model, a target image output by the second model is obtained, the image style of the target image is the same as the image style of the reference image, and the high-level semantic information of the target image is the same as the high-level semantic information of the first image. By adopting the method provided by the invention, the electronic equipment can change the image style of the image according to the prompt information input by the user, so that the flexibility of changing the image style is improved.

Description

Technical Field

[0001] The present application relates to the field of image processing, and in particular to an image processing method and electronic device. Background Art

[0002] With the popularity and development of mobile devices with cameras, people are increasingly acquiring images in a more convenient way. Images captured by electronic devices are affected by many external factors, resulting in unsatisfactory images. It is necessary to manually modify the images using software. For example, if the brightness and saturation of the captured image do not meet the needs of the user, the user can use the photo editing software to adjust the color temperature, brightness, saturation, hue and other parameters of the image. If the user wants to change the background environment in the image, the user needs to use professional photo editing software to manually edit the image. For example, image A includes a big tree with dense leaves; the user can remove the leaves from the big tree in image A through photo editing software (such as photoshop), and modify the big tree with dense leaves in image A into a big tree without leaves.

[0003] However, this method of photo editing requires users to have professional photo editing experience in order to modify the image, and this method of photo editing requires many manual steps, which results in users being unable to quickly modify images through electronic devices, affecting the user's image editing experience. Summary of the invention

[0004] In order to solve the above technical problems, the present application provides an image processing method and an electronic device, so that the electronic device can quickly and accurately convert the image style of an image into the image style indicated in the prompt information of style conversion input by the user. This method of converting the image style does not require manual adjustment of the image, reduces the steps of manual operation, and increases the flexibility of the electronic device in performing image style conversion on the image.

[0005] In a first aspect, the present application provides a method for image processing, comprising: obtaining a first image and first prompt information input by a user, the first prompt information including text information of a target style; inputting the first image and the first prompt information into a first model to obtain a reference image of the first image, the image style of the reference image being the same as the target style, the image style including at least one of the color features and texture features of the image; inputting the first image and the reference image into the second model to obtain a target image output by the second model, the image style of the target image being the same as the image style of the reference image and the target image having the same high-level semantic information as the first image.

[0006] In this way, the first model adjusts the image style of the first image according to the first prompt information, and obtains a reference image of the first image. The reference image of the first image has a target style, and the reference image of the first image has a high probability of image distortion (i.e., high-level semantic information changes) compared to the first image. The electronic device uses the reference image of the first image as a reference image for image style transfer of the first image, so that the second model can transfer the image style of the reference image to the first image to generate a target image, and the target image can maintain the high-level semantic information of the first image to ensure that there is no distortion problem in the target image. The electronic device can transform the image style of the first image into any image style through the first model and the second model, so that the electronic device can adjust the image style in a more flexible way, and there is no need to manually adjust the image style of the image, thereby increasing the application scenarios of the software for adjusting the image style.

[0007] According to the first aspect, before inputting the first image and the first prompt information into the first model to obtain the reference image of the first image, the method further includes: obtaining a first training sample set, the first training sample set including the first sample image, the style sample prompt information and the second sample image corresponding to the first sample image, the image style of the second sample image being the same as the image style indicated by the style sample prompt information; inputting the first sample image and the style sample prompt information into the first training model to obtain the image output by the first training model; using the second sample image to verify the image output by the first training model to update the parameters of the first training model until the first training model converges and generates the first model. In this way, the image style of the second sample image is the same as the image style indicated by the style sample prompt information, the first sample image and the second sample image in the first training sample set are paired training data, and the high-precision training data improves the success rate of the first model generated based on the first training model and the first training sample set to convert the image style of the first image, thereby improving the accuracy of the subsequent second model to convert the image style of the first image.

[0008] According to the first aspect, before the first image and the reference image are input into the second model, the method further includes: obtaining a second training sample set, the second training sample set includes a first sample image, a first sample reference image, and a second sample image, the first sample reference image being determined based on the second sample image; inputting the first sample image and the first sample reference image into the second training model to obtain an image output by the second training model; using the second sample image to verify the image output by the second training model to update the parameters of the second training model until the second training model converges and generates a second model. In this way, during the training of the second model, the data input into the second training model are the first sample image and the reference image of the first sample image, so that the second training model can transfer the image style of the reference image to the first sample image, rather than converting the image style of the first image into a fixed image style, thereby improving the flexibility of the second model in converting the image style of the first image.

[0009] According to the first aspect, before inputting the first image and the reference image into the second model, the method further includes: detecting whether the adjustment category of the first prompt information belongs to the first category, the adjustment category of the first prompt information being the category of the image style of the first image indicated by the first prompt information to the first model; if it is detected that the adjustment category belongs to the first category, obtaining the first candidate model as the second model, the first category being the category indicating that the first model adjusts the color features in the first image; if it is detected that the target style belongs to the second category, obtaining the second candidate model as the second model, the second category including the category indicating that the first model adjusts the color and texture features in the first image, and the category indicating that the first model adjusts the texture features in the first image. In this way, the electronic device selects different candidate models as the second model according to the adjustment category of the first prompt information, so that the processing of the first image by the selected candidate model is more in line with the requirements of the adjustment category, and the quality of the image processing by the second model can be improved.

[0010] According to the first aspect, detecting whether the adjustment category of the first prompt information belongs to the first category includes: if it is detected that there is information indicating the color of the image in the first prompt information and there is no information indicating the time, then determining that the adjustment category of the first prompt information belongs to the first category; if it is detected that there is information indicating the time in the first prompt information, then determining that the adjustment category of the first prompt information belongs to the second category. In this way, the adjustment of the color in the image will not affect the texture information in the image. By detecting whether there is color information in the first prompt information, the electronic device can quickly determine whether the adjustment category of the first prompt information is the first category; and objects usually change with the passage of time. When the adjustment of the image involves time conversion (such as season conversion, day and night conversion, etc.), it usually means that the structure of the object in the image will change (such as a big tree sprouts in spring and has no leaves in winter). The electronic device can determine whether the adjustment category of the first prompt information is the second category based on the time information. This judgment method is simple and fast.

[0011] According to the first aspect, before obtaining the category of image style adjustment indicated by the first prompt information, the method also includes: when the second training model adopts a high dynamic range HDR network model, obtaining a second training sample set, the second training sample set includes a first sample image, a first sample reference image, and a second sample image, the image style of the second sample image is the same as the image style indicated by the style sample prompt information, and the first sample reference image is determined based on the second sample image; the second training model performs low-resolution coefficient prediction processing on the first sample image and the second sample image to obtain a processed first sample image and a processed second sample image; the second training model performs pixel-level network fusion processing on the first sample image to generate a guide image of the first sample image; the second training model performs matrix transformation processing on the guide image, the processed first sample image, and the processed second sample image; the second training model combines the image processed by the matrix transformation with the first sample image to output a second predicted image; and the second predicted image is verified using the second sample image to update the parameters of the second training model until the second training model converges to generate a first candidate model. In this way, the second training model adopts HDRnet, and the first sample reference image is introduced in this training method, so that the output prediction image can obtain the image style in the first sample reference image, and the HDRnet has a good effect on adjusting the color of the image.

[0012] According to the first aspect, the second training model includes a weight prediction model and N basic three-dimensional lookup tables, where N is an integer greater than 1; before obtaining the category of image style adjustment indicated by the first prompt information, the method also includes: obtaining a second training sample set, the second training sample set includes a first sample image, a first sample reference image, and a second sample image, the image style of the second sample image is the same as the image style indicated by the style sample prompt information, and the first sample reference image is determined based on the second sample image; inputting the first sample image and the first sample reference image into the weight prediction model, and obtaining N weights corresponding to the N basic three-dimensional lookup tables output by the weight prediction model; fusing the N basic three-dimensional lookup tables according to the N weights corresponding to the N basic three-dimensional lookup tables to generate a three-dimensional lookup table of reference style, and the three-dimensional lookup table of reference style is a color gamut mapping of the target style; using the three-dimensional lookup table of reference style to transform the image style of the first sample image to obtain a second predicted image; using the second sample image to verify the second predicted image to iteratively update the parameters of the weight prediction model and the N basic three-dimensional lookup tables until the second training model converges to generate a first candidate model. In this way, during the second training model process, the electronic device continuously adjusts the N basic three-dimensional lookup tables and adjusts the weights corresponding to each of the N basic three-dimensional lookup tables by adjusting the parameters in the weight prediction model, so as to obtain a three-dimensional lookup table of reference style by fusing the N basic three-dimensional lookup tables. The electronic device can quickly and accurately convert the image style of the input image into the style of the first sample reference image through the three-dimensional lookup table of reference style.

[0013] According to the first aspect, before obtaining the category of image style adjustment indicated by the first prompt information, the method further includes: when the second training model is a restoration transformer model, obtaining a second training sample set, the second training sample set includes a first sample image, a first sample reference image, and a second sample image, the image style of the second sample image is the same as the image style indicated by the style sample prompt information, and the first sample reference image is determined based on the second sample image; inputting the first sample image and the first sample reference image into the second training model to obtain a second predicted image output by the second training model; using the second sample image to verify the second predicted image to update the parameters of the second training model until the second training model converges and generates a second candidate model. In this way, the restoration transformer model has a good effect on the texture adjustment of the image, and the texture features of the first image can be better adjusted by using this model.

[0014] According to the first aspect, the method further includes: compressing the second sample image according to a preset size to generate a first sample reference image. In this way, the first sample reference image has a small image size, which reduces the complexity of training the second training model, thereby reducing the magnitude of the trained second model, making it easier to deploy the second model in the mobile terminal.

[0015] According to the first aspect, the first image and the first prompt information are input into the first model to obtain a reference image of the first image, including: inputting the first image and the first prompt information into the first model to obtain a first predicted image output by the first model, wherein the image style of the first predicted image is the same as the target style; compressing the first predicted image according to a preset size to generate a reference image of the first image. In this way, the first predicted image output by the first model is compressed, and the compressed image is used as the reference image of the first image, which can reduce the speed of the second model processing the compressed image.

[0016] According to the first aspect, obtaining the first prompt information input by the user includes: displaying the first image; displaying an input box in response to the user inputting an operation to adjust the image style; obtaining the prompt information input by the user in the input box as the first prompt information. In this way, after the user inputs the operation to adjust the image style, the input box is displayed, so that the user can be prompted to input prompt information for style adjustment. This method allows the user to input any information for image style adjustment, and the electronic device can perform an image style adjustment operation on the first image according to any input style adjustment information.

[0017] According to the first aspect, obtaining the first prompt information input by the user includes: displaying a first image; in response to the user input operation of adjusting the image style, displaying at least two image style controls, each image style control having prompt information of the corresponding image style; in response to the user selecting an image style from the at least two image style controls, obtaining prompt information of the selected image style; and using the obtained prompt information of the image style as the first prompt information. In this way, the user does not need to manually input the prompt information of the image style, which reduces the user's operation on the electronic device.

[0018] In a second aspect, the present application provides an electronic device comprising: one or more processors; a memory; and one or more computer programs, wherein the one or more computer programs are stored in the memory, and when the computer programs are executed by the one or more processors, the electronic device performs the image processing method corresponding to the first aspect and any one of the implementation methods of the first aspect.

[0019] The second aspect and any implementation of the second aspect correspond to the first aspect and any implementation of the first aspect respectively. The technical effects corresponding to the second aspect and any implementation of the second aspect can refer to the technical effects corresponding to the above-mentioned first aspect and any implementation of the first aspect, which will not be repeated here.

[0020] In a third aspect, the present application provides a computer-readable medium for storing a computer program. When the computer program is executed on an electronic device, the electronic device executes the image processing method corresponding to the above-mentioned first aspect and any one of the implementations of the first aspect. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings required for use in the description of the embodiments of the present application will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative labor.

[0022] Figure 1 is a schematic diagram exemplarily showing the structure of an electronic device;

[0023] Figure 2 is a software structure block diagram of an electronic device exemplarily shown;

[0024] Figure 3 is a flowchart exemplarily showing a method for image processing;

[0025] Figure 4 is a schematic diagram exemplarily showing the adjustment of the image style of a first image;

[0026] Figure 5 are exemplary images of large trees in different seasons;

[0027] Figure 6 is an exemplary illustration of a first image, a target image, and a reference image;

[0028] Figure 7a is a schematic diagram showing an exemplary second model training;

[0029] Figure 7b is a schematic diagram showing another exemplary second model training;

[0030] Figure 8 is a schematic diagram showing an exemplary second candidate model training;

[0031] Fig. 9 is a flowchart exemplarily showing a method for image processing;

[0032] Fig.10 is a schematic diagram exemplarily showing an adjustment of the image style of an image;

[0033] Fig.11 FIG. 1 is a schematic diagram showing an exemplary method of adjusting the image style of an image. DETAILED DESCRIPTION

[0034] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0035] The term "and / or" in this article is merely a description of the association relationship of associated objects, indicating that three relationships may exist. For example, A and / or B can mean: A exists alone, A and B exist at the same time, and B exists alone.

[0036] The terms "first" and "second" in the description and claims of the embodiments of the present application are used to distinguish different objects rather than to describe a specific order of objects. For example, a first target object and a second target object are used to distinguish different target objects rather than to describe a specific order of target objects.

[0037] In the embodiments of the present application, words such as "exemplary" or "for example" are used to indicate examples, illustrations or descriptions. Any embodiment or design described as "exemplary" or "for example" in the embodiments of the present application should not be interpreted as being more preferred or more advantageous than other embodiments or designs. Specifically, the use of words such as "exemplary" or "for example" is intended to present related concepts in a specific way.

[0038] In the description of the embodiments of the present application, unless otherwise specified, the meaning of "multiple" refers to two or more than two. For example, multiple processing units refer to two or more processing units; multiple systems refer to two or more systems.

[0039] In some embodiments, the user may edit the acquired image through an image style editing module in the electronic device so that the edited image meets the user's needs.

[0040] The electronic device can be a mobile phone, a computer, a digital camera and other devices. In this example, the electronic device is explained by taking a mobile phone as an example. An image style editing module is installed in the mobile phone, and the image style editing module can adjust the color features of the image such as saturation and sharpness. The user can adjust the style in the image through an image style editing module, such as Photoshop. For example, the color of the image style of the original image is a cold tone (such as winter), and the user can use Photoshop to change the season in the image (such as changing it to spring). However, this method requires the user to be proficient in using Photoshop software, and is not suitable for every user, such as users who cannot use Photoshop. In other words, the electronic device cannot change the image style through simple operations, which makes the electronic device inflexible in adjusting the image style of the image and reduces the application scenarios of the electronic device changing the image style of the image.

[0041] In an embodiment of the present application, a method for image processing is provided, and the method is performed by an electronic device. The electronic device may be a mobile phone, a computer, a tablet computer, a PAD, etc. An image style editing module is installed in the electronic device. Optionally, the image style editing module may be embedded in a gallery application. For example, a user may click on a gallery in a mobile phone and click on a style adjustment icon in the gallery application. In response to the user clicking on the style adjustment operation, the mobile phone starts the image style editing module. Optionally, the image style editing module may also be a separate application software.

[0042] The image processing method includes an electronic device acquiring a first image and first prompt information input by a user, and text information of a target style of the first prompt information; inputting the first image and the first prompt information into a first model to obtain a reference image of the first image, and the image style of the reference image is the same as the target style; inputting the first image and the reference image into a second model to obtain a target image output by the second model, and the image style of the target image is the same as the image style of the reference image and the high-level semantic information of the target image is the same as that of the first image.

[0043] In this example, the electronic device can adjust the image style of the first image to any image style through the first model and the second model. At the same time, the high-level semantic information of the adjusted image can be the same as the high-level semantic information of the original image, that is, the image processing method of the present application can ensure that the image style of the first image is converted arbitrarily, and can also ensure the quality of the converted image, such as no pixel reduction or distortion of objects in the image.

[0044] Figure 1 This is a schematic diagram of the structure of an electronic device 100 shown in an embodiment of the present application. It should be understood that: Figure 1The electronic device 100 shown is only one example of an electronic device, and the electronic device 100 may have more or fewer components than shown in the drawings, may combine two or more components, or may have a different configuration of components. Figure 1 The various components shown in the EMBODIMENTS 2000 may be implemented in hardware, software, or a combination of hardware and software, including one or more signal processing and / or application specific integrated circuits.

[0045] The electronic device 100 may include: a processor 110, an external memory interface 120, an internal memory 121, a universal serial bus (USB) interface 130, a charging management module 140, a power management module 141, a battery 142, an antenna 1, an antenna 2, a mobile communication module 150, a wireless communication module 160, an audio module 170, a speaker 170A, a receiver 170B, a microphone 170C, an earphone interface 170D, a sensor module 180, a button 190, a motor 191, an indicator 192, a camera 193, a display screen 194, and a subscriber identification module (SIM) card interface 195, etc. The sensor module 180 may include a pressure sensor, a gyroscope sensor, an air pressure sensor, a magnetic sensor, an acceleration sensor, a distance sensor, a proximity light sensor, a fingerprint sensor, a temperature sensor, a touch sensor, an ambient light sensor, a bone conduction sensor, etc.

[0046] Figure 2 It is a software structure block diagram of the electronic device 100 according to an embodiment of the present application.

[0047] The layered architecture of the electronic device 100 divides the software into several layers, each with a clear role and division of labor. The layers communicate with each other through software interfaces. In some embodiments, the Android system is divided into three layers, from top to bottom, namely, the application layer, the application framework layer, and the kernel layer. It is understandable that Figure 2 The layers in the software structure and the components contained in each layer do not constitute a specific limitation on the electronic device 100. In other embodiments of the present application, the electronic device 100 may include more or fewer layers than shown in the figure, and each layer may include more or fewer components, which is not limited in the present application.

[0048] like Figure 2As shown, the application layer may include a series of application packages. The application package may include applications such as gallery, social application, camera, calendar, short message, etc. The application framework layer provides an application programming interface (API) and a programming framework for the applications of the application layer. The application framework layer includes some predefined functions. The image style editing module in the present application can be embedded in the gallery, social application, and camera application. For example, before the display interface of the social application displays an image, the electronic device responds to the user's operation of starting the image style editing module in the social application, starts the image style editing module, and the electronic device uses the image style editing module to arbitrarily convert the image style of the image to be displayed, and displays the image with the converted image style on the display interface.

[0049] like Figure 2 As shown, the application framework layer may include a window manager, a resource manager, a content provider, a view system, a phone manager, a notification manager, and the like.

[0050] The window manager is used to manage window programs. The window manager can obtain the display screen size, determine whether there is a status bar, lock the screen, capture the screen, etc.

[0051] The resource manager provides various resources for applications, such as localized strings, icons, images, layout files, video files, and so on.

[0052] Content providers are used to store and retrieve data and make it accessible to applications. The data may include videos, images, audio, calls made and received, browsing history and bookmarks, phone books, etc.

[0053] The view system includes visual controls, such as controls for displaying text, controls for displaying images, etc. The view system can be used to build applications. A display interface can be composed of one or more views. For example, a display interface including a text notification icon can include a view for displaying text and a view for displaying images.

[0054] The phone manager is used to provide communication functions of the electronic device 100, such as management of call status (including connecting, hanging up, etc.).

[0055] The notification manager enables applications to display notification information in the status bar. It can be used to convey notification-type messages and can disappear automatically after a short stay without user interaction. For example, the notification manager is used to notify download completion, message reminders, etc.

[0056] The kernel layer is the layer between hardware and software. The kernel layer contains at least display driver, camera driver, audio driver, and sensor driver.

[0057] Before explaining the image processing method in the present application, some technical terms involved in the present application are explained.

[0058] (1) Semantic information of an image: The content of an image is the semantic information of the image. The speech information of an image is divided into low-level semantic information and high-level semantic information. The low-level semantic information of an image includes: image style. The high-level semantic information of an image refers to the things expressed by the image that are closest to human understanding. The high-level semantic information of an image is what people can see. For example, when extracting features from a face image, the low-level semantic information extracted includes: the contour of the face, nose, eyes, etc., and the high-level semantic information extracted is the face.

[0059] (2) Image style: The image style may include the texture features of the image and the artistic expression of the image. The image style may include daytime style, nighttime style, spring style, summer style, autumn style, winter style, sunny style, rainy style, shooting star style, Hideaki Hamada style, etc. The image style may also include the style of famous paintings, such as Van Gogh's "Sunflowers" and "Starry Night", etc., and may also include the type of image creation, such as Chinese style, cubism, modernism, surrealism, cartoon, comic, oil painting, watercolor, ink painting, etc. The present application embodiment does not limit the image style of the image.

[0060] Figure 3 : is a flowchart of an exemplary method of image processing. In this example, a mobile phone is used as an example of an electronic device for illustration. The image processing process specifically includes:

[0061] Step 301: The electronic device obtains a first image and first prompt information input by a user.

[0062] Exemplarily, a mobile phone (such as mobile phone A) obtains an image whose image style is to be adjusted as a first image. The first image may be an image sent by another device (such as mobile phone B) or an image taken by mobile phone A.

[0063] The process of the mobile phone acquiring the first image and the first prompt information input by the user may include: responding to the user's operation of adjusting the image style to be edited (such as Fig.1010b, click the "style adjustment" icon), the image style editing module determines that the current image to be edited is the first image, and at the same time, an input box can be displayed in the display interface. The input box is used to obtain information about the target style entered by the user. Optionally, before the user enters text information into the input box, the input box may display text input prompt information, for example, the text input prompt information may be "Please enter the information about the image style to be converted!". When the user clicks the input box, the displayed text input prompt information is eliminated, and a soft keyboard pops up in the interface. When the image style editing module detects that the user has completed the information entered into the input box, the information in the input box is obtained as the first prompt information. For example, the user enters in the input box: Convert to spring. The first prompt information is "Convert to spring".

[0064] Step 302: The electronic device determines a reference image of the first image according to the first image, the first prompt information and the first model.

[0065] Exemplarily, the image style editing module in the mobile phone includes a trained first model and a second model. The first model is used to output a first predicted image according to the original image and the style conversion prompt information input by the user, and the image style of the first predicted image is consistent with the target style in the style conversion prompt information.

[0066] The following specific Figure 5 and Figure 6 The following describes the process of training the first model.

[0067] A first training model is preset, and the network structure of the first training model can adopt the network structure of a conditional diffusion model (instructPix2Pix). The first training sample set is a data set used to train the first training model. The first training sample set includes: a first sample image, style sample prompt information, and a second sample image corresponding to the first sample image, and the image style of the second sample image is the same as the image style indicated by the style sample prompt information. The first sample image and the second sample image are a pair of images.

[0068] In this example, the electronic device can collect a large amount of style sample prompt information and a first sample image, and can manually synthesize a second sample image using Photoshop according to the target style to ensure that the high-level semantic information of the second sample image and the first sample image remains unchanged.

[0069] For example, the first sample image is Figure 5The big tree shown in 5a. The style sample prompt information is "Convert to Summer". The big tree image shown in 5a can be manually edited according to the style sample prompt information to generate an image as shown in 5b. The electronic device can use the big tree image shown in 5a as the second sample image. Similarly, the big tree image shown in 5c and the big tree image shown in 5d can be manually synthesized according to "Convert to Autumn" and "Convert to Winter". The electronic device uses the big tree image shown in 5c and the big tree image shown in 5d as the second sample image. Figure 5 In the tree images shown in 5a to 5d, the main body in the four images is the tree, the number and shape of the branches of the tree have not changed, and the shape and number of the leaves of the tree are different in different seasons, that is, the high-level semantic information of the tree images shown in 5a to 5d is the same. Paired sample images can improve the accuracy of the predicted image output by the first model.

[0070] The style sample prompt information in the first training sample set may be text information with a target style, for example: the background is a meteor, the tree sprouts, the tree sheds leaves, the photo is transformed into an oil painting style, and the photo looks brighter.

[0071] The first training model includes a text encoder and an image encoder, wherein the text encoder can extract text features in the style sample prompt information, and the image encoder can extract image features in the first sample image. During the training process, the first training model fuses the text features and the image features, and outputs a predicted image having the image style indicated by the style prompt information. The electronic device verifies the image output by the first training model using the second sample image to update the parameters of the first training model until the first training model converges to generate the first model.

[0072] In this example, the image style editing module installs the trained first model. After the image style editing module obtains the first image and the first prompt information, the first image and the first prompt information are input into the first model to obtain a reference image of the first image. The first model outputs a first predicted image, the image style of which is the same as the target style, and the subject in the first predicted image may be distorted, incomplete, and other problems.

[0073] For example, if Figure 6 The building image shown in 6a is the first image, and the building image shown in 6b is the target image. The first prompt information is "the background is a meteor". The image style editing module inputs the first image and the first prompt information into the first model to obtain the first predicted image output by the first model. The first predicted image is as follows: Figure 6 As shown in 6c, the building in the first predicted image is distorted, and the image quality of the first predicted image is lower than that of the images shown in 6a and 6c.

[0074] The image quality of the first predicted image output by the first model is poor, but the image style of the first predicted image is consistent with the target style, so the image style editing module can use the first predicted image as a reference image for the first image.

[0075] Optionally, the image style editing module may compress the first predicted image (resolution of 1920*1080 pixels) according to a preset size to obtain a thumbnail image of the first predicted image (such as a resolution of 200*200 pixels). The image style editing module uses the thumbnail image of the first predicted image as a reference image of the first image.

[0076] Step 303: The electronic device inputs the first image and the reference image into the second model to obtain a target image output by the second model, wherein the image style of the target image is the same as that of the reference image and the high-level semantic information of the target image is the same as that of the first image.

[0077] Exemplarily, the image style editing module is pre-installed with a second model. The second model is used to output a target image based on an input original image and a reference image of the original image, wherein the image style of the target image is the same as the image style of the reference image and the target image has the same high-level semantic information as the first image.

[0078] Combine the following Figure 7a or Figure 7b The training process of the second model is described in detail.

[0079] The electronic device obtains a second training set, where the second training sample set includes a first sample image, a first sample reference image, and a second sample image, where the first sample reference image is determined based on the second sample image. Figure 7a As shown, Figure 7a The original image in is the first sample image, the image after style transformation is the second sample image, the first sample image and the second sample image are paired images. The second sample image is the same image as the second sample image used to train the first model.

[0080] The electronic device presets a second training model, and the network structure of the second training model can adopt a high dynamic range network structure (High Dynamic Range Imaging Network, “HDRnet”). Figure 7a is a schematic diagram of training the second training model, such as Figure 7a As shown, the electronic device inputs the first sample image and the second sample image into the second training model, and the second training model converts the first sample image (i.e. Figure 7a The original image in the figure) and the second sample image (i.e. Figure 7aThe first processing is performed on the image after the style transformation shown in the figure to obtain low-resolution images 1 and 2. The second training model performs low-resolution coefficient prediction processing on images 1 and 2 to obtain the processed first sample image (recorded as Feature_Ori_1) and the processed second sample image (recorded as Feature_Tar_2). The low-resolution coefficient prediction processing is used to perform bilateral filtering on the image to obtain the image features of the input image.

[0081] The second training model simultaneously performs pixel-wise-network processing on the first sample image to generate a guidance map of the first sample image. The guidance map is used to obtain the interaction information of the first sample image. The second training model filters the guidance map of the first sample image, the first sample image (Feature_Tar_1) after low-resolution coefficient prediction processing, and the second sample image (Feature_Tar_2) in a preset manner (i.e., matrix transformation) to obtain the image after matrix transformation. The second training model combines the image after matrix transformation with the first sample image to generate a second predicted sample image. The electronic device obtains the difference between the second predicted sample image and the second sample image to update the parameters in the second training model until the second training model converges to generate a second model.

[0082] In some embodiments, the second training model may be based on 3D Look-Up Tables (“3DLUTs”). Figure 7b Schematic diagram of a second training model based on 3D LUTs training is shown as an example. The second training set for training the second training model includes a first sample image, a first sample reference image, and a second sample image. The first sample image is Figure 7b The second sample image is the image after the style transformation corresponding to the original image, and the first sample reference image is the thumbnail of the image after the style transformation (i.e. Figure 7b The image style of the second sample image is the same as the image style of the first sample reference image. In this example, the image style of the first sample reference image is referred to as the reference style.

[0083] like Figure 7bAs shown, the preset second training model may include a preset weight prediction model and N basic 3DLUTs (i.e., Basis 3D LUTs). The N Basis 3D LUTs are different N Basis 3D LUTs, where N is an integer greater than 1. For example, N may be 3, 4, 5, etc. In this example, N is 3 as an example. When the electronic device starts training the second model, 3 Basis 3D LUTs may be pre-set. The weight prediction model may be a convolutional neural network (CNN), in which the input data of the weight prediction model is a first sample image and a first sample reference image, and the output is weights corresponding to the N Basis 3D LUTs.

[0084] The electronic device inputs the first sample image and the first sample reference image into the weight prediction model, and obtains three weights output by the weight prediction model, the three weights being w1, w2 and w3, wherein w1 is the weight corresponding to the first Basis 3D LUTs, w2 is the weight corresponding to the second Basis 3D LUTs, and w3 is the weight corresponding to the third Basis 3D LUTs. Figure 7b The weights of the three Basis 3D LUTs are fused to obtain a 3D LUT based on the reference style (hereinafter referred to as the 3D LUTs of the reference style). The electronic device uses the 3D LUTs of the reference style to transform the original image to generate an image after the style transformation (i.e., the second predicted sample image). The electronic device obtains the difference between the second predicted sample image and the second sample image to iteratively update the parameters in the weight prediction model and the three Basis3D LUTs until the second training model converges to generate a second model.

[0085] The image style editing module installs the trained second model. After the electronic device obtains the first image and the reference image of the first image, the first image and the reference image of the first image are input into the second model to obtain the target image output by the second model. The image style of the target image is the same as the image style of the reference image of the first image, and the high-level semantic information of the target image and the first image is the same.

[0086] Figure 4 FIG. 1 is a schematic diagram of the image style editing module processing the first image and the first prompt information. Figure 4 As shown, the image style editing module includes a trained first model and a second model. The image style editing module converts the first image (such as Figure 4The image style editing module inputs the reference image and the original image into the second model, and the second model outputs a target image. The image style editing module inputs the reference image and the original image into the second model, and the second model outputs a target image. The image style of the target image is consistent with the image style of the reference image, and the character of the target image is consistent with the character in the original image without distortion. The image style editing module achieves the purpose of converting the image style of the first image into any style through the first model and the second model.

[0087] In this example, the electronic device can obtain a reference image of the first image through the first model. The image style of the reference image of the first image is the same as the target style in the first prompt information. When the first model adjusts the image style of the first model according to the first prompt information, the adjusted image may be distorted, and the image quality of the reference image of the first image is poor. By using the reference image of the first image as the reference image for adjusting the image style of the first image by the second model, the image style of the reference image of the first image can be transferred to the first image, while keeping the high-level semantic information in the first image unchanged, so that the image quality of the target image output by the second model is high. This method allows the electronic device to change the image style of the first image through the first prompt information input by the user, simplifies the steps of manually editing the image, and improves the efficiency of the electronic device in processing the first image.

[0088] In one embodiment, the electronic device may pre-store a first candidate model and a second candidate model. The electronic device may select one of the first candidate model and the second candidate model as the second model according to the first prompt information input by the user. The image processing process is as follows: Fig. 9 As shown, including:

[0089] Step 901: The electronic device obtains a first image and first prompt information input by a user.

[0090] This step is similar to the related description in step 301 and will not be repeated here.

[0091] Step 902: The electronic device detects whether the adjustment category is the first category. If it is, step 903 is executed; if it is not, step 904 is executed.

[0092] Exemplarily, the adjustment category of the first prompt information is the category of the image style of the first image that the first prompt information indicates the first model to adjust, and the adjustment category includes the first category and the second category. The first category is the category that indicates the first model to adjust the color characteristics of the image. For example, the first prompt information is "adjust the color to bright", and the first prompt information indicates that the first model adjusts the color characteristics of the first image, and the adjustment category of the first prompt information is the first category. For another example, "adjust from fresh blue to Akita", fresh blue indicates that the image's hue is blue, and Akita indicates that the image's hue is orange. That is to say, the adjustment category of the first prompt information is the first category.

[0093] The second category includes a category indicating that the first model adjusts the texture features of the image and a category for adjusting the texture and color features. For example, the first prompt information is "turning to winter", and the current image is taken in spring. The scenery will change with time, which will cause the texture of the image to change. The adjustment category of the first prompt information is the second category.

[0094] In one example, since the first category is the category of the color feature of the image adjusted by the first model, the electronic device can detect whether the first prompt information carries information indicating the color and does not contain information indicating the time. If it is detected that the first prompt information carries information indicating the color and does not contain information indicating the time, it is determined that the adjustment category of the image style of the first image indicated by the first prompt information is the first category. For example, the first prompt information is "adjust the color to bright", and the electronic device detects "bright", which is information indicating the color, and determines that the adjustment indicated by the first prompt information is the first category.

[0095] The electronic device can detect whether there is information indicating time conversion in the first prompt information, and if so, determine that the adjustment category of the image style of the first image indicated by the first model in the first prompt information is the second category. For example, "convert to night", the electronic device detects that there is night in the first prompt information, and determines that the adjustment category indicated by the first prompt information is the second category.

[0096] When the electronic device determines that the adjustment category of the first prompt information is the first category, step 903 is executed. If it is detected that the adjustment category of the first prompt information is the second category, step 904 is executed.

[0097] Step 903: the electronic device obtains the first candidate model as the second model. After this step, step 905 is executed.

[0098] Exemplarily, the image style editing module can store the trained first candidate model and the second candidate model. The training model of the first candidate model can adopt the structure of HDRnet. The training process of the second candidate model can refer to Figure 7a or Figure 7b The relevant description in will not be repeated here.

[0099] Step 904: the electronic device obtains a second candidate model as the second model. After this step, step 905 is executed.

[0100] Exemplarily, the training model of the second candidate model may adopt a Restoration Transformer (“Restormer”) model. The electronic device obtains a second training sample set, which includes: a first sample image, a first sample reference image, and a second sample image, wherein the first sample reference image is determined based on the second sample image. In this example, the first sample reference image is a thumbnail image of the second sample image, and the resolution of the thumbnail image is a preset resolution, such as 360*360 pixels.

[0101] Figure 8 The process of training the second candidate model is as follows: Figure 8 As shown, the first sample image (i.e. Figure 8 The original image in ) and the first sample reference image (i.e. Figure 8 The electronic device obtains the difference between the second predicted sample image and the second sample image, and iteratively trains the second training model according to the obtained difference until the second training model converges to obtain a second candidate model.

[0102] When the image style editing module detects that the adjustment category of the first prompt information is the second category, the second candidate model is obtained as the second model. When the target style adjusted by the second model involves time transformation, it is necessary to adjust the texture features in the image, or adjust the texture features and color features in the image, and use the Restormer model so that the predicted image output by the second model is more in line with the actual needs of the user, and the output image quality is better.

[0103] Step 905: The electronic device determines a reference image of the first image according to the first image, the first prompt information and the first model.

[0104] This step is substantially the same as step 302 , and the specific process may refer to the relevant description in step 302 .

[0105] Step 906: The electronic device inputs the first image and the reference image into the second model to obtain the target image output by the second model.

[0106] This step is substantially the same as step 303 , and the specific process may refer to the relevant description in step 302 .

[0107] Fig.10 This is a schematic diagram showing an example of a user using an image style editing module to adjust the image style of an image. In this example, the image style editing module stores a first model and a second model. The second model is a fixed model.

[0108] In this example, user A starts the gallery application on the phone and displays the gallery interface. The gallery application has an embedded image style editing module. The user opens the gallery interface ( Fig.10 (not shown) select an image (such as clicking a thumbnail image in the gallery), such as Fig.10 As shown in 10a of FIG. 10 , the gallery displays the image 1002 in the interface 1001 in response to the user's selection operation. The interface 1001 includes an edit icon 1003, a delete icon, etc. In response to the user clicking the edit icon 1003, the gallery jumps to the image editing interface 1004, as shown in FIG. Fig.10 10b in the figure. The image editing interface 1004 displays the image 1002 to be edited. The image editing interface 1004 displays a style adjustment icon 1005 and a filter icon. The image editing interface may also include icons for functions such as one-click beautification and image cropping. In response to the user clicking the style adjustment icon 1005, the gallery may jump to the style adjustment interface 1006. The style adjustment interface 1006 displays the image 1002 (i.e., the first image) to be adjusted in style. The style adjustment interface 1006 also displays an input box 1007 for the first prompt information. Before the user enters information, the input box 1007 may display input prompt information such as, "Please enter the text information of the target style of the image!" to prompt the user to enter the first prompt information.

[0109] like Fig.10 As shown in 10c in the figure, the user inputs the first prompt information in the input box 1007, and the content of the first prompt information is "Convert to winter". In response to the user clicking the "OK" control 1008, the image style editing module obtains the image 1002 and the first prompt information. The image style editing module inputs the image 1002 and the first prompt information into the first model, and obtains the first predicted image output by the first model. The image style of the first predicted image is winter (i.e., the big trees have no leaves). The image style editing module compresses the first predicted image (the resolution before compression is 1920*1080 pixels) according to a preset size, obtains the compressed first predicted image and uses it as a reference image of the first image. The resolution of the reference image of the first image is 200*200 pixels. The image style editing module inputs the first image and the reference image of the first image into the second model, obtains the target image output by the second model, and instructs the display interface to display the target image. As shown Fig.10As shown in FIG. 10d , the target image 1010 is displayed in the interface 1009 . The big tree in the target image 1010 has no leaves, and the shooting environment of the image is winter.

[0110] In this example, the first model and the second model are used to change the texture of the tree in the image (i.e., the tree with leaves is transformed into a tree without leaves). This image style adjustment method adjusts the image texture and ensures that the adjusted image is not distorted, and the pixels remain the same as the original image, thereby improving the display effect of the image after transformation.

[0111] In other examples, the electronic device may pre-store a plurality of prompt information for converting image styles, and in response to an operation of the user selecting prompt information for converting image styles, the electronic device obtains the prompt information for converting image styles selected by the user as the first prompt information.

[0112] For example, the image style editing module stores the first model, the first candidate model, and the second candidate model. User B starts the gallery application on the mobile phone and displays the gallery interface. The image style editing module is embedded in the gallery application. User B opens the gallery interface ( Fig.11 (not shown) selects an image (such as clicking a thumbnail image in the gallery), and the gallery responds to the user's selection operation on a display interface (the display interface can refer to Fig.10 The image is displayed in the interface 1001 in 10a. The display interface includes an edit icon, a delete icon, etc. In response to the user clicking the edit icon, the gallery jumps to the image editing interface 1101, such as Fig.11 The image editing interface 1101 includes: an image 1102 to be edited and a start icon of each image editing function. The start icons of the image editing functions include: a style adjustment icon, a filter icon, a cropping icon, an icon for adding mosaics, etc. In this example, Fig.11 As shown in 11a of FIG. 11 , the start icons of the image editing function include: a style adjustment icon 1103 and a filter icon. Other start icons are not shown in 11a. As shown in 11a, the user clicks the style adjustment icon 1103. In response to the user's click operation, the interface jumps to the style adjustment interface 1104. Fig.11 As shown in FIG. 11b, the style adjustment interface 1104 displays an image 1102 (i.e., the first image) to be subjected to image style adjustment. Fig.11As shown in 11b, multiple hint messages for converting the image style are displayed in the hint message display area 1105 of the style adjustment interface. The hint messages are, for example, meteor, cloudy, bright, night, etc. The user clicks on the "meteor" control 11051, obtains the image 1102, and obtains the hint message corresponding to the "meteor" control 11051 as the first hint message. In this example, the hint message corresponding to the "meteor" control 11051 is "The sky is converted into meteors".

[0113] The image style editing module inputs the image 1102 and the first hint message into the first model, and obtains the first predicted image output by the first model. The image style of the first predicted image is: the sky is meteors. The image style editing module compresses the first predicted image (the resolution before compression is 1920*1080 pixels) according to the preset size, obtains the compressed first predicted image and uses it as the reference image of the first image. The resolution of the reference image of the first image is 200*200 pixels. The image style editing module inputs the first image and the reference image of the first image into the second model, obtains the target image output by the second model, and instructs to display the target image on the display interface. The image 1102 is the first image, the time in the first image is daytime, and the corresponding brightness value is d1. The first hint message is "The sky is converted into meteors". The conversion conditions indicated in the first hint message include: the time of the image is night, and there are meteors in the night sky. As Fig.11 As shown in 11c, the target image 1107 is displayed in the interface 1106. The brightness corresponding to the converted image 1107 of the image 1102 is e1, and e1 < d1. It can be seen that the brightness of the image 1107 is lower than the brightness in the image 1102, and meteors are included in the sky of the image 1107.

[0114] In one example, the user clicks on the "bright" control 11052. The image style editing module responds to the user's click operation, obtains the image 1102 as the first image, and obtains the hint message corresponding to the "bright" control 11052 as the first hint message. In this example, the hint message corresponding to the "bright" control is "Increase the brightness value of the image to the preset value". The image style editing module detects whether there is information indicating color transformation in the first hint message and detects whether there is information indicating time transformation in the first hint message. In this example, the image style editing module detects that there is information indicating color (such as brightness value) in the first hint message and there is no information indicating time, and obtains the first candidate model (such as the model trained based on HDRnet or the model trained based on 3D LUTs) as the second model.

[0115] The image style editing module inputs the image 1102 and the first prompt message into the first model to obtain the first predicted image output by the first model. The image style of the first predicted image is: increasing the brightness value. The image editing software compresses the first predicted image (with a resolution of 1920*1080 pixels before compression) according to a preset size to obtain the compressed first predicted image, which is used as the reference image of the first image. The resolution of the reference image of the first image is 200*200 pixels. The image style editing module inputs the first image and the reference image of the first image into the second model to obtain the target image output by the second model, and instructs to display the target image 1110 on the display interface. The image 1102 is the first image, and the brightness value corresponding to the image in the first image is d1. The first prompt message is "Increase the brightness value of the image to a preset value". The conversion condition indicated in the first prompt message includes: increasing the brightness value to a preset value. As Fig.11 shown in 11d in, the target image 1110 is displayed in the interface 1109. The brightness corresponding to the converted image 1110 of the image 1102 is c1, and d1 < c1. It can be seen that the brightness of the image 1110 is higher than that of the image 1102. Other information in the image remains unchanged.

[0116] It can be understood that in order for the electronic device to implement the above functions, it includes the corresponding hardware and / or software modules for executing each function. Combining the algorithm steps of each example described in the embodiments disclosed in this article, the present application can be implemented in the form of hardware or a combination of hardware and computer software. Whether a certain function is executed in the form of hardware or computer software driving hardware depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application in combination with the embodiments, but such implementation should not be considered to exceed the scope of the present application.

[0117] This embodiment also provides a computer storage medium. Computer instructions are stored in the computer storage medium. When the computer instructions run on the electronic device, the electronic device is caused to execute the above-related method steps to implement the image processing method in the above embodiment. The storage medium includes: various media such as USB flash drives, mobile hard disks, read only memory (ROM), random access memory (RAM), magnetic disks, or optical discs that can store program codes.

[0118] This embodiment also provides a computer program product. When the computer program product runs on a computer, the computer is caused to execute the above-related steps to implement the image processing method in the above embodiment.

[0119] Among them, the electronic device, computer storage medium, computer program product or chip provided in this embodiment is used to execute the corresponding method provided above. Therefore, the beneficial effects that can be achieved can refer to the beneficial effects in the corresponding method provided above and will not be repeated here.

[0120] Any content of each embodiment of the present application, as well as any content of the same embodiment, can be freely combined. Any combination of the above content is within the scope of the present application.

[0121] The embodiments of the present application are described above in conjunction with the accompanying drawings, but the present application is not limited to the above-mentioned specific implementation methods. The above-mentioned specific implementation methods are merely illustrative and not restrictive. Under the guidance of the present application, ordinary technicians in this field can also make many forms without departing from the purpose of the present application and the scope of protection of the claims, all of which are within the protection of the present application.

Claims

1. A method for image processing, characterized in that: include: Acquire a first image and first prompt information input by a user, wherein the first prompt information includes text information of a target style; Inputting the first image and the first prompt information into a first model to obtain a reference image of the first image, wherein the image style of the reference image is the same as the target style, and the image style includes at least one of a color feature and a texture feature of the image; The first image and the reference image are input into a second model to obtain a target image output by the second model, wherein the image style of the target image is the same as that of the reference image and the high-level semantic information of the target image is the same as that of the first image.

2. The method according to claim 1, characterized in that: Before inputting the first image and the first prompt information into a first model to obtain a reference image of the first image, the method further includes: Acquire a first training sample set, the first training sample set comprising a first sample image, style sample prompt information, and a second sample image corresponding to the first sample image, wherein the image style of the second sample image is the same as the image style indicated by the style sample prompt information; Inputting the first sample image and the style sample prompt information into the first training model to obtain an image output by the first training model; The image output by the first training model is verified using the second sample image to update the parameters of the first training model until the first training model converges to generate the first model.

3. The method according to claim 2, characterized in that Before inputting the first image and the reference image into the second model, the method further comprises: Acquire a second training sample set, where the second training sample set includes the first sample image, a first sample reference image, and the second sample image, where the first sample reference image is determined based on the second sample image; Inputting the first sample image and the first sample reference image into a second training model to obtain an image output by the second training model; The image output by the second training model is verified using the second sample image to update the parameters of the second training model until the second training model converges to generate the second model.

4. The method according to claim 2, characterized in that: Before inputting the first image and the reference image into the second model, the method further comprises: detecting whether an adjustment category of the first prompt information belongs to a first category, where the adjustment category of the first prompt information is a category of an image style of the first image indicated by the first prompt information to be adjusted by the first model; If it is detected that the adjustment category belongs to a first category, obtaining a first candidate model as the second model, the first category being a category indicating that the first model adjusts color features in the first image; If it is detected that the target style belongs to a second category, a second candidate model is obtained as the second model, and the second category includes a category indicating that the first model adjusts the color and texture features in the first image, and a category indicating that the first model adjusts the texture features in the first image.

5. The method according to claim 4, characterized in that Detecting whether the adjustment category of the first prompt information belongs to the first category includes: If it is detected that the first prompt information contains information indicating the image color and does not contain information indicating the time, determining that the adjustment category of the first prompt information belongs to the first category; If it is detected that the first prompt information contains information indicating time, it is determined that the adjustment category of the first prompt information belongs to the second category.

6. The method according to claim 4, characterized in that Before obtaining the category of image style adjustment indicated by the first prompt information, the method further includes: In the case where the second training model adopts a high dynamic range HDR network model, obtaining a second training sample set, where the second training sample set includes the first sample image, a first sample reference image, and the second sample image, where the image style of the second sample image is the same as the image style indicated by the style sample prompt information, and the first sample reference image is determined based on the second sample image; The second training model performs low-resolution coefficient prediction processing on the first sample image and the second sample image to obtain a processed first sample image and a processed second sample image; The second training model performs pixel-level network fusion processing on the first sample image to generate a guide image of the first sample image; The second training model performs matrix transformation processing on the guide image, the processed first sample image and the processed second sample image; The second training model combines the image processed by the matrix transformation with the first sample image to output a second predicted image; The second predicted image is verified using the second sample image to update the parameters of the second training model until the second training model converges to generate the first candidate model.

7. The method according to claim 4, characterized in that The second training model includes a weight prediction model and N basic three-dimensional lookup tables, where N is an integer greater than 1; Before obtaining the category of image style adjustment indicated by the first prompt information, the method further includes: Acquire a second training sample set, where the second training sample set includes the first sample image, a first sample reference image, and the second sample image, where the image style of the second sample image is the same as the image style indicated by the style sample prompt information, and the first sample reference image is determined based on the second sample image; Inputting the first sample image and the first sample reference image into the weight prediction model, and obtaining N weights corresponding to N basic three-dimensional lookup tables output by the weight prediction model; fusing the N basic three-dimensional lookup tables according to N weights corresponding to the N basic three-dimensional lookup tables to generate a three-dimensional lookup table of a reference style, where the three-dimensional lookup table of the reference style is a color gamut mapping of a target style; transforming the image style of the first sample image using the three-dimensional lookup table of the reference style to obtain a second predicted image; The second predicted image is verified using the second sample image to iteratively update the parameters of the weight prediction model and N basic three-dimensional lookup tables until the second training model converges to generate the first candidate model.

8. The method according to claim 4, characterized in that Before obtaining the category of image style adjustment indicated by the first prompt information, the method further includes: In the case where the second training model is a restoration transformer model, a second training sample set is obtained, where the second training sample set includes the first sample image, a first sample reference image, and a second sample image, where the image style of the second sample image is the same as the image style indicated by the style sample prompt information, and the first sample reference image is determined based on the second sample image; Inputting the first sample image and the first sample reference image into a second training model to obtain a second predicted image output by the second training model; The second predicted image is verified using the second sample image to update the parameters of the second training model until the second training model converges to generate the second candidate model.

9. The method according to any one of claims 6 to 8, characterized in that: The method further comprises: The second sample image is compressed according to a preset size to generate the first sample reference image.

10. The method according to claim 1, characterized in that The step of inputting the first image and the first prompt information into a first model to obtain a reference image of the first image includes: Inputting the first image and the first prompt information into the first model, obtaining a first predicted image output by the first model, wherein the image style of the first predicted image is the same as the target style; The first predicted image is compressed according to a preset size to generate a reference image of the first image.

11. The method according to claim 1, characterized in that: Get the first prompt information entered by the user, including: displaying the first image; In response to a user input operation for adjusting the image style, an input box is displayed; The prompt information input by the user in the input box is obtained as the first prompt information.

12. The method according to claim 1, characterized in that Get the first prompt information entered by the user, including: displaying the first image; In response to an operation of adjusting the image style input by the user, at least two image style controls are displayed, each of which has prompt information of the image style corresponding to the control; In response to a selection operation of a user selecting an image style from at least two image style controls, obtaining prompt information of the selected image style; The acquired prompt information of the image style is used as the first prompt information.

13. An electronic device, characterized in that: include: a memory and a processor, the memory being coupled to the processor; The memory stores program instructions, and when the program instructions are executed by the processor, the electronic device executes the image processing method according to any one of claims 1 to 12.

14. A computer-readable storage medium comprising a computer program, characterized in that: When the computer program is executed on an electronic device, the electronic device is enabled to execute the image processing method according to any one of claims 1 to 12.

Images