Style transfer method, medium, computer device and program product

Abstract

A style transfer method, medium, computer device and program product, the method comprising: in response to an image style transfer function being triggered, obtaining multiple input images; extracting image content of the multiple input images respectively; fusing the image content of the multiple input images to obtain fused image content; generating a fused image including the fused image content; and performing style transfer on the fused image to obtain an output image.

Description

Technical Field

[0001] The present specification relates to the field of image processing technology, and in particular to a style transfer method, a medium, a computer device, and a program product. Background Art

[0002] Style transfer can transfer the style of one image to another image, thus creating an image with a new style. When performing style transfer, the content of an image is usually completely retained, and only the style of the image is converted. However, this method cannot obtain diverse and personalized style transfer content. Summary of the invention

[0003] In a first aspect, an embodiment of the present specification provides a style transfer method, the method comprising: in response to an image style transfer function being triggered, obtaining multiple input images; extracting image content of the multiple input images respectively; fusing the image content of the multiple input images to obtain fused image content; generating a fused image including the fused image content; and performing style transfer on the fused image to obtain an output image.

[0004] In a second aspect, an embodiment of this specification provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the method described in any embodiment of this specification.

[0005] In a third aspect, an embodiment of this specification provides a computer device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the method described in any embodiment of this specification when executing the program.

[0006] In a fourth aspect, an embodiment of this specification provides a computer program product, including a computer program, which implements the method described in any embodiment of this specification when executed by a processor.

[0007] In the embodiments of the present specification, when the image style transfer function is triggered, the image contents of multiple input images are first fused to obtain fused image content, and then the style transfer is performed on the fused image including the fused image content. In this way, the image content in the obtained output image is not completely consistent with the content of a single input image, but is the result of the fusion of multiple input images, thereby improving the diversity and personalization of the output image obtained by style transfer.

[0008] It is to be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the present specification. BRIEF DESCRIPTION OF THE DRAWINGS

[0009] The drawings herein are incorporated into the specification and constitute a part of the specification. These drawings illustrate embodiments consistent with the specification and are used together with the specification to illustrate the technical solutions of the specification.

[0010] Figure 1 It is a comparison chart of images before and after style transfer in the related art.

[0011] Figure 2 is a flowchart of the style transfer method of an embodiment of this specification.

[0012] Figure 3A , Figure 3B and Figure 3C They are comparison diagrams of images before and after style transfer according to the embodiments of this specification.

[0013] Figure 4 It is a schematic diagram of a user interface of an embodiment of this specification.

[0014] Figure 5 is a schematic diagram of a computer device according to an embodiment of the present specification. DETAILED DESCRIPTION

[0015] Exemplary embodiments will be described in detail herein, examples of which are shown in the accompanying drawings. When the following description refers to the drawings, unless otherwise indicated, the same numbers in different drawings represent the same or similar elements. The implementations described in the following exemplary embodiments do not represent all implementations consistent with this specification. Instead, they are merely examples of devices and methods consistent with some aspects of this specification as detailed in the appended claims.

[0016] The terms used in this specification are only for the purpose of describing specific embodiments and are not intended to limit this specification. The singular forms of "a", "said" and "the" used in this specification and the appended claims are also intended to include plural forms, unless the context clearly indicates other meanings. It should also be understood that the term "and / or" used in this article refers to and includes any or all possible combinations of one or more associated listed items. In addition, the term "at least one" herein means any combination of at least two of any one or more of a plurality of.

[0017] It should be understood that although the terms first, second, third, etc. may be used in this specification to describe various information, this information should not be limited to these terms. These terms are only used to distinguish the same type of information from each other. For example, without departing from the scope of this specification, the first information may also be referred to as the second information, and similarly, the second information may also be referred to as the first information. Depending on the context, the word "if" as used herein may be interpreted as "at the time of" or "when" or "in response to determining".

[0018] In order to enable those skilled in the art to better understand the technical solutions in the embodiments of this specification and to make the above-mentioned purposes, features and advantages of the embodiments of this specification more obvious and understandable, the technical solutions in the embodiments of this specification are further described in detail below in conjunction with the accompanying drawings.

[0019] Style transfer can quickly transfer the style of a reference image to an input image, obtaining an output image that includes the image content of the input image and has the style of the reference image. Figure 1 As shown, the input image can be a rabbit image with a cartoon style. Assuming that the style of the reference image (not shown) is an embroidery style, the input image can be style-transferred to obtain a rabbit image with an embroidery style. This style transfer method will completely preserve the image content in the input image. However, users sometimes do not want to completely preserve the image content in the input image, but want to generate a more diverse and personalized output image. For example, the user may want the color of the rabbit in the output image to be different from the color of the rabbit in the input image, or want the output image to include other elements in addition to the rabbit. The above effects cannot be achieved through traditional style transfer.

[0020] Based on this, the embodiment of this specification provides a style transfer method, which fuses the image contents of multiple input images and then performs style transfer, so that the image content in the output image is not completely consistent with the content of a single input image, but the result of the fusion of multiple input images. In this way, users can add multiple different input images according to actual needs, thereby fusing different image contents, which improves the diversity and personalization of the output image obtained by style transfer.

[0021] The following is an example of the specific implementation details of the style transfer method of the embodiment of this specification with reference to the accompanying drawings. Figure 2 , the style transfer method of the embodiment of this specification includes:

[0022] Step S12: in response to the image style transfer function being triggered, obtaining a plurality of input images;

[0023] Step S14: extracting image contents of multiple input images respectively;

[0024] Step S16: fusing the image contents of multiple input images to obtain fused image contents;

[0025] Step S18: generating a fused image including fused image content;

[0026] Step S20: performing style transfer on the fused image to obtain an output image.

[0027] In step S12, when the image style transfer function is triggered, the user may upload multiple input images, or call an interface (API) providing image services to obtain multiple input images, or collect multiple input images through an image sensor, or generate multiple input images through an image generation algorithm. It is also possible to use a combination of at least two of the above methods, or to use other methods to obtain multiple input images, which is not limited in this specification. Among them, the triggering method of the image style transfer function may be that the user starts the image style transfer function in the application software, or the calling interface of the image style transfer function receives a call request, etc. The image content of at least two input images in the multiple input images is different. Among them, the image content may include but is not limited to at least one of the following: the pixel value of the image, the category and / or number of objects included in the image, the time information and / or weather information reflected in the image, etc. Among them, the time information may be specific hours, minutes, and seconds, such as 19:22:30, or it may be category information such as morning, noon, and evening. The weather information may be information representing the climate such as sunny, windy, rainy, and snowy.

[0028] In some embodiments, the multiple input images may include a first input image and at least one second input image. The first input image is the main image that needs to be style transferred, and each second input image is an image used to be fused with the first input image. The above-mentioned first input image can be uploaded by a user, and each second input image can be uploaded by a user or obtained in other ways, for example, received from an image acquisition interface, or crawled from the Internet, or obtained from an image data block. By fusing at least one second input image into the first input image, the image content of the first input image can be changed, so that the style transfer result is not just a complete copy of the image content in the first input image, thereby improving the diversification and personalization of the style transfer result. In other embodiments, it is also possible not to distinguish between the first input image and the second input image, but to use multiple input images as images that need to be fused and style transferred.

[0029] In an embodiment where the multiple input images include a first input image and at least one second input image, the first input image may be an image uploaded by a user or an image acquired in other ways. Each second input image may be an image uploaded by a user, or an image including specified image content, or an image generated or collected in a specified application scenario. In this way, the second input image including different image content may be fused into the first input image according to actual needs, so that the style transfer result is adapted to the actual needs.

[0030] In step S14, the image contents of multiple input images can be extracted respectively. In the embodiment where the pixel value of the image is used as the image content, the pixel value of each pixel point in each input image can be directly extracted, and the pixel value extracted from any input image is used as the image content of the input image. In the embodiment where the category and / or number of objects included in the image, the environmental information (such as time information and / or weather information) reflected in the image, etc. are used as the image content, the input image can be detected and recognized, and the image content of each input image can be determined based on the detection and recognition results. For example, the input image can be subjected to target detection to determine the number and category of objects included in the input image. For another example, the input image can be subjected to semantic recognition to determine the time information and weather information reflected in the input image. When the time information reflected in the input image is expressed as specific hours, minutes, and seconds, the timestamp of the input image can also be obtained, and the time information reflected in the input image can be determined based on the timestamp.

[0031] In step S16, the image contents of the multiple input images may be fused to obtain fused image contents. For example, the pixel values ​​in the multiple input images may be weighted, and the weighted pixel values ​​may be determined as the fused image contents. When weighting the pixel values ​​in the multiple input images, the weights of the input images may be the same or different. The specific method of setting the weights of the input images is described below by way of example.

[0032] In some embodiments, multiple input images may be divided into at least two clusters based on the image content included in the multiple input images. The input images in the same cluster include the same image content. In the example where the image content is a pixel value, the input images in the same cluster may include the same pixel value. It should be noted that two input images include the same pixel value, which may be that the difference between the pixel values ​​of the two input images is within a preset range, and the pixel values ​​of the two input images are not required to be strictly the same in value. From a visual perception point of view, two input images include the same pixel value, which may be that the colors expressed by the pixel values ​​of the two input images are the same, for example, both are red, or both are blue. Therefore, if the pixel values ​​of the two input images are both red, the two input images belong to the same cluster; and if the pixel value of one input image is red and the pixel value of the other input image is blue, the two input images belong to different clusters.

[0033] Furthermore, for two color input images, the pixel values ​​of the corresponding target image areas in the two input images can be compared. If the colors of the corresponding target image areas in the two input images are the same, the two input images belong to the same cluster, otherwise the two input images belong to different clusters. The corresponding target image areas may be image areas where the same object is located. For example, assuming that both input images include flowers, the image areas where the flowers are located in the two input images are the corresponding target image areas in the two input images. Alternatively, the relative positions of the corresponding target image areas in the corresponding input images may be the same. For example, an image area in one input image (denoted as image 1) is the central area of ​​the input image, and an image area in another input image (denoted as image 2) is also the central area of ​​the input image. Then the central area of ​​image 1 and the central area of ​​image 2 are the corresponding target image areas in image 1 and image 2. Alternatively, the corresponding target image areas may be image areas with the largest color proportion in the two input images. For example, the image area with the largest color ratio in one input image (recorded as image 3) is the image area where the red pixels are located, and the image area with the largest color ratio in another input image (recorded as image 4) is the image area where the blue pixels are located. Then, the image area where the red pixels are located in image 3 and the image area where the blue pixels are located in image 4 are the corresponding target image areas in image 3 and image 4. The corresponding target image areas in multiple input images can also be determined based on other methods, so as to determine whether the multiple input images belong to the same cluster, which will not be listed here one by one.

[0034] In the example where the image content is the object included in the image, the input images in the same cluster may include the same object. For example, if the objects in both input images are rabbits, the two input images belong to the same cluster; if the object in one input image is a rabbit and the object in the other input image is a frog, the two input images belong to different clusters. In the case where the image content is the environmental information reflected in the image, if the environmental information reflected in the two input images is the same, the two input images belong to the same cluster, otherwise the two input images belong to different clusters. In the example where the image content includes the pixel values ​​of the image and the objects included in the image, if the two input images include the same object and the pixel values ​​of the object in the two input images are the same, the two input images belong to the same cluster; if the objects included in the two input images are different, or the two input images include the same object but the pixel values ​​of the object in the two input images are different, the two input images belong to different clusters. Other methods can also be used to cluster multiple input images, which will not be listed here one by one.

[0035] After the cluster division is completed, the weights of the input images in the at least two clusters can be determined based on the number of input images respectively included in the at least two clusters, and the image contents of the multiple input images can be weighted based on the weights of the multiple input images to obtain the fused image content. The weight of the input image is positively correlated with the number of input images in the cluster to which the input image belongs. For example, assuming that one cluster (hereinafter referred to as cluster A) includes 3 input images and another cluster (hereinafter referred to as cluster B) includes two input images, the weight of the input image in cluster A is greater than the weight of the input image in cluster B. In this way, a larger weight can be set for the input image in the cluster including more input images, so that the image fusion result is more biased towards the input image in the cluster including more input images. In other words, the user can make the image fusion result more focused on the multiple similar input images by uploading multiple similar input images. Taking the image content including pixel values ​​as an example, assuming that the user wants the pixel values ​​in the fused image content to be more biased towards red, multiple images including red pixel values ​​can be uploaded. The above method enables users to upload different numbers of similar images to change the image fusion results, thereby improving the personalization and diversification of style transfer.

[0036] It is understood that the manner of setting the weights in the above embodiments is only an exemplary description. In other embodiments, the weights corresponding to the input images may also be randomly generated or set by the user.

[0037] In addition to obtaining the fused image content by weighting the pixel values ​​in the multiple input images, the objects in the multiple input images can also be detected separately, and the objects respectively included in the multiple input images can be fused to obtain the fused image content. For example, when one input image includes a first object and another input image includes a second object, the objects respectively included in the two input images can be fused, and the obtained fused image content includes the first object and the second object. For another example, when one input image includes M first objects and another input image includes N first objects, the objects respectively included in the two input images can be fused, and the obtained fused image content includes M+N first objects. In the above embodiment, the same weight can be set for each input image, so that the image content in each input image is fused equally. When fusing the image content, the image content in one input image (such as the second input image in the above embodiment) can also be used to replace the image content in another input image (such as the first input image in the above embodiment). Taking the case where the image content includes environmental information, when the image content in the first input image includes the first environmental information and the image content in the second input image includes the second environmental information, the first environmental information in the first input image can be replaced by the second environmental information. In this embodiment, a larger weight can be set for the second input image and a smaller weight can be set for the first input image, so that the image content in the two input images is preferentially adopted when the image content is fused to replace the image content in the first input image.

[0038] In some embodiments, the image contents of multiple input images may be fused with text information used to describe the specified image content to obtain fused image content. The text information may include but is not limited to randomly generated text information, user input text information, fixed text information, text information obtained by calling a specified interface, etc. For example, the text information is text information input by the user. Figure 4 As shown, a text input box 20 may be provided on a user interface 10, and a user may input text information in the text input box 20. The above text information is used to describe the content of a specified image, including but not limited to the category, quantity, angle, pixel value, position, material, size, pixel value of the object in the image, and / or environmental information reflected in the image.

[0039] In addition to the several fusion methods listed above, other methods can also be used to fuse the image contents of multiple input images, which will not be listed here one by one.

[0040] In step S18, a fused image including fused image content may be generated. In some embodiments, the fused image content is generated by weighting the pixel values ​​in the multiple input images, and the fused image is an image obtained by weighting the pixel values ​​of corresponding pixels in the multiple input images. Figure 3A As shown, one input image is a rabbit image including a black background area, and the other input image is a rabbit image including a white background area. After fusion, a rabbit image including a gray background area can be obtained.

[0041] In other embodiments, the fused image content is generated by fusing objects respectively included in multiple input images, so that the fused image includes the objects respectively included in the multiple input images. Figure 3B As shown, one input image includes a rabbit and another input image includes a flower, then the fused image is an image including a rabbit and a flower. In this case, the position of the object in each input image can also be obtained, and the position of the object in the fused image is determined based on the position of the object in each input image. The position of an object in the input image can be the same as the position of the object in the fused image. Still assuming that one input image includes a rabbit and the other input image includes a flower, then the fused image is an image including a rabbit and a flower, and the position of the rabbit in the fused image is the same as the position of the rabbit in the input image, and the position of the flower in the fused image is the same as the position of the flower in the input image. Alternatively, there can also be a preset offset between the position of an object in the input image and the position of the object in the fused image.

[0042] When generating a fused image, in order to make the fused image more natural and realistic, the fused image may also be post-processed, and the post-processing includes but is not limited to at least one of the following: white balance processing, color correction, tone matching, edge smoothing processing, brightness processing, contrast processing, and sharpening processing.

[0043] In some embodiments, the plurality of input images include a first input image and at least one second input image, and the first input image and each second input image include a first image area and a second image area. Optionally, the first image area may be a foreground area in the first input image and the second input image, that is, an image area where a main object in the first input image and the second input image is located. The second image area may be a background area in the first input image and the second input image, that is, an image area other than the main object in the first input image and the second input image. Alternatively, the first image area may be a background area in the first input image and the second input image, and the second image area may be a foreground area in the first input image and the second input image. Alternatively, the first image area may be a central area in the first input image and the second input image, and the second image area may be an edge area in the first input image and the second input image. Alternatively, the first image area may be an edge area in the first input image and the second input image, and the second image area may be the first image area may be a central area in the first input image and the second input image. Alternatively, the first image area and the second image area may also be image areas determined based on other methods, which are not listed here one by one.

[0044] In step S16, only the image content in the second image area of ​​the first input image and the image content in the second image area of ​​at least one second input image may be fused to obtain fused image content. In this step, a fused image including the image content in the first image area of ​​the first input image and the fused image content may be generated. In this way, the image content of the first image area of ​​the first input image may be retained, and the image content of the second image area of ​​the first input image may be personalized through fusion. For example, assuming that the first image area is the central area of ​​the first input image and the second input image, and the second image area is the edge area of ​​the first input image and the second input image, the image content of the edge area of ​​the first input image and the at least one second input image may be fused first to obtain the fused image content of the edge area. Then, the image content of the central area of ​​the first input image is fused with the fused image content of the edge area, and the obtained fused image also includes the central area and the edge area. Among them, the image content of the central area of ​​the fused image is consistent with the image content of the central area of ​​the first input image, and the image content of the edge area of ​​the fused image is the image content obtained after the edge area of ​​the first input image and the at least one second input image are fused.

[0045] like Figure 3CAs shown, the central area of ​​the first input image and the central area of ​​the second input image include rabbits of different colors, the edge area of ​​the first input image includes a red flower, and the edge area of ​​the second input image includes a yellow flower. The fused image includes the rabbit in the first input image, the red flower in the first input image, and the yellow flower in the second input image.

[0046] In step S20, the fused image may be style-transferred to obtain an output image. Specifically, the fused image may be transferred to an output image of a specified style. The specified style may be a target style selected by the user from a plurality of candidate styles. Figure 4 , multiple options 30 corresponding to candidate styles can be displayed on the user interface 10, and the multiple candidate styles include but are not limited to embroidery style, cartoon style, animation style, oil painting style, watercolor style, classical style, etc. The user selects a target option from the options corresponding to the multiple candidate styles, and determines the candidate style corresponding to the target option as the style of the output image. Figure 4 As shown, the user selects option 30 corresponding to the embroidery style, so that the embroidery style is the target style. Alternatively, the specified style may be the style of the target template image, and the target template image is selected by the user from a plurality of candidate template images. For example, a plurality of candidate template images may be displayed on the user interface 10, and different candidate template images have different styles. By displaying the candidate template images, it is convenient for the user to observe different styles intuitively. The user may select the target template image from a plurality of candidate template images on the user interface 10, so that the style of the target template image is determined as the style of the output image. Alternatively, the specified style may be the style of an image uploaded by the user. In the above manner, the user may freely select the style of the output image according to his or her own needs and preferences.

[0047] In some embodiments, the number of output images is greater than 1. Among them, different output images have the same style, and the image contents of different output images are at least partially different. Specifically, in step S16, multiple fusion methods can be used to fuse the image contents of multiple input images to obtain fused image contents corresponding to the multiple fusion methods, and multiple fused images including multiple fused image contents are generated in step S18. For example, different weights can be used to fuse the pixel values ​​of multiple input images to obtain multiple fused images, and different fused images include different pixel values. For another example, the pixel values ​​of multiple input images can be fused to obtain a fused image, and the objects in the multiple input images can be fused to obtain another fused image. In this step, style transfer can be performed on multiple fused images to obtain multiple output images.

[0048] In some embodiments, after obtaining multiple output images, the multiple output images may be post-processed. The post-processing may include selecting a target output image from the multiple output images. The selection operation may be performed manually by the user. Since different users may have different aesthetics and needs, by adopting the above method, the user can select the output image version that best suits their preferences, thereby improving the user experience. The post-processing may also include splicing the multiple output images. Through splicing, the image contents of the multiple output images may be integrated, which is convenient for users to view and compare.

[0049] In some embodiments, after obtaining multiple output images, the output images can also be input into a pre-trained super-resolution reconstruction model to obtain super-resolution output images. The resolution of the super-resolution output image is higher than the resolution of the output image. Through the above method, the clarity and detail of the first output image can be improved, making the first output image look more realistic and detailed, and it is also convenient to enlarge the image to a larger size while maintaining the image quality, so as to facilitate high-quality printing, display, advertising design and other processing of the image.

[0050] In some embodiments, a parameter adjustment instruction input by a user may also be obtained, and the image parameters of the output image may be adjusted in response to the parameter adjustment instruction to obtain the adjusted output image, and the adjusted output image may be displayed. The image parameters may include but are not limited to at least one of the following: resolution, quantity, aspect ratio, style intensity, color tone, etc. of the output image. A user interface 10 may be provided, and the user may select or input corresponding image parameters in the user interface 10. The user may preview the output image in real time on the user interface, and adjust the image parameters according to the preview result. In this way, the diversity of the output image may be improved, so that the output image may better meet the needs of the user.

[0051] The style transfer process of the embodiment of this specification can be implemented using a pre-trained style transfer model, and the above-mentioned style transfer model can be implemented based on a deep learning model, such as a convolutional neural network and a generative adversarial network. By performing style transfer, users can generate high-quality artistic images in a short time, greatly improving the creation efficiency and saving time and energy. In addition, even users without professional art skills can easily generate works of art, lowering the threshold for creation and allowing more people to participate in artistic creation. Users can convert their photos or images into various artistic styles, such as impressionism, abstraction, etc., enriching the source of visual creativity and meeting the needs of different scenes. The output image can be applied to multiple fields such as advertising design, social media, digital art, wallpaper, etc., and has a wide range of commercial and personal application value. In addition, users can freely choose and adjust style effects according to personal preferences and needs, realize personalized image creation, and enhance user experience.

[0052] An embodiment of the present specification also provides a computer device, which at least includes a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, the method described in any of the above embodiments is implemented.

[0053] Figure 5 A more specific hardware structure diagram of a computer device provided in an embodiment of this specification is shown, and the device may include: a processor 202, a memory 204, an input / output interface 206, a communication interface 208, and a bus 210. The processor 202, the memory 204, the input / output interface 206, and the communication interface 208 are connected to each other through the bus 210 in the device.

[0054] The processor 202 can be implemented by a general-purpose central processing unit (CPU), a microprocessor, an application-specific integrated circuit (ASIC), or one or more integrated circuits, etc., for executing relevant programs to implement the technical solutions provided in the embodiments of this specification. The processor 202 can also include a graphics card, which can be a Nvidia titan X graphics card or a 1080Ti graphics card, etc.

[0055] The memory 204 can be implemented in the form of a read-only memory (ROM), a random access memory (RAM), a static storage device, a dynamic storage device, etc. The memory 204 can store an operating system and other application programs. When the technical solutions provided in the embodiments of this specification are implemented by software or firmware, the relevant program codes are stored in the memory 204 and are called and executed by the processor 202.

[0056] The input / output interface 206 is used to connect the input / output module to realize information input and output. The input / output module can be configured in the device as a component (not shown in the figure), or it can be externally connected to the device to provide corresponding functions. The input device may include a keyboard, a mouse, a touch screen, a microphone, various sensors, etc., and the output device may include a display, a speaker, a vibrator, an indicator light, etc.

[0057] The communication interface 208 is used to connect a communication module (not shown) to realize communication interaction between the device and other devices. The communication module can realize communication through wired means (such as USB, network cable, etc.) or wireless means (such as mobile network, WIFI, Bluetooth, etc.).

[0058] The bus 210 comprises a pathway for transmitting information between the various components of the device, such as the processor 202 , the memory 204 , the input / output interface 206 , and the communication interface 208 .

[0059] It should be noted that, although the above device only shows the processor 202, the memory 204, the input / output interface 206, the communication interface 208 and the bus 210, in the specific implementation process, the device may also include other components necessary for normal operation. In addition, it can be understood by those skilled in the art that the above device may also only include the components necessary for implementing the embodiments of the present specification, and does not necessarily include all the components shown in the figure.

[0060] An embodiment of the present specification provides a computer program product, including a computer program. When the computer program is executed by a processor, the method described in any embodiment of the present specification is implemented.

[0061] The embodiments of the present specification also provide a computer-readable storage medium on which a computer program is stored. When the program is executed by a processor, the method described in any of the above embodiments is implemented.

[0062] Computer readable media include permanent and non-permanent, removable and non-removable media that can be implemented by any method or technology to store information. Information can be computer readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disk read-only memory (CD-ROM), digital versatile disk (DVD) or other optical storage, magnetic cassettes, magnetic tape magnetic disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computer device. As defined herein, computer readable media does not include temporary computer readable media (transitory media), such as modulated data signals and carrier waves.

[0063] Each embodiment in this specification is described in a progressive manner, and the same and similar parts between the embodiments can be referred to each other, and each embodiment focuses on the differences from other embodiments. In particular, for the device embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment. The device embodiment described above is merely schematic, wherein the modules described as separate components may or may not be physically separated, and the functions of each module can be implemented in the same one or more software and / or hardware when implementing the embodiment scheme of this specification. It is also possible to select some or all of the modules according to actual needs to achieve the purpose of the embodiment scheme. A person of ordinary skill in the art can understand and implement it without paying creative labor.

[0064] The above is only a specific implementation of the embodiments of this specification. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principles of the embodiments of this specification. These improvements and modifications should also be regarded as the protection scope of the embodiments of this specification.

Claims

1. A style transfer method, the method comprising: In response to the image style transfer function being triggered, obtaining a plurality of input images; respectively extracting image contents of the plurality of input images; Fusing the image contents of the multiple input images to obtain fused image content; generating a fused image including the fused image content; Performing style migration on the fused image to obtain an output image; The step of fusing the image contents of the plurality of input images to obtain fused image contents includes: Based on image contents included in the plurality of input images, the plurality of input images are divided into at least two clusters, wherein the input images in the same cluster include the same image content; Based on the number of input images respectively included in the at least two clusters, determining the weights of the input images in the at least two clusters; wherein the weight of the input image is positively correlated with the number of input images in the cluster to which the input image belongs; Based on the weights of the multiple input images, the image contents of the multiple input images are weighted to obtain fused image content.

2. According to the method of claim 1, the step of fusing the image contents of the plurality of input images to obtain fused image contents comprises: The image contents of the plurality of input images are fused with text information used to describe the specified image contents to obtain fused image contents.

3. According to the method of claim 1, performing style migration on the fused image to obtain an output image comprises: The fused image is transferred to an output image of a specified style, wherein the specified style is: A target style selected by the user from among multiple candidate styles; or the style of a target template image, where the target template image is selected by a user from a plurality of candidate template images; or The style of the image uploaded by the user.

4. The method according to claim 1, wherein the number of output images is greater than 1; wherein, The styles of the different output images are the same, and the image contents of the different output images are at least partially different.

5. The method according to claim 1, wherein the plurality of input images include a first input image and at least one second input image; the first input image is a main image for style transfer, and the at least one second input image is an image for fusion with the first input image; At least the first input image among the plurality of input images is uploaded by a user.

6. The method according to claim 1, further comprising: Inputting the output image into a pre-trained super-resolution reconstruction model to obtain a super-resolution output image; The resolution of the super-resolution output image is higher than the resolution of the output image.

7. The method according to claim 1, further comprising: Get the parameter adjustment instruction input by the user; In response to the parameter adjustment instruction, the image parameters of the output image are adjusted to obtain an adjusted output image; The adjusted output image is displayed.

8. The method according to claim 1, wherein the plurality of input images include a first input image and at least one second input image, and the first input image and the at least one second input image both include a first image region and a second image region; and fusing the image contents of the plurality of input images to obtain fused image contents, comprising: fusing image content in the second image region of the first input image with image content in the second image region of the at least one second input image to obtain fused image content; The generating of the fused image including the fused image content comprises: A fused image is generated that includes the image content in the first image region in the first input image and the fused image content.

9. A computer-readable storage medium having a computer program stored thereon, wherein the program, when executed by a processor, implements the method according to any one of claims 1 to 8.

10. A computer device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the method according to any one of claims 1 to 8 when executing the program.

11. A computer program product, comprising a computer program, wherein when the computer program is executed by a processor, the method according to any one of claims 1 to 8 is implemented.

Images