A method and device for image style conversion

Abstract

The present application provides a method and device for image style conversion, wherein the image style conversion method comprises: receiving an image to be converted and a style image; extracting first feature information of the image to be converted, extracting second feature information of the style image; inputting the first feature information and the second feature information into a pre-trained reverse network model for processing to generate an initial conversion image; adjusting the initial conversion image according to the image to be converted to obtain a target style conversion image. The method provided by the present application can transfer the image style of the image to be converted according to the style of the style image, which is simple and easy to implement, and saves a lot of manpower, material resources and art computing resources.

Description

Technical Field

[0001] The present application relates to the field of Internet technology, and in particular to a method and apparatus for image style conversion, a computing device, and a computer-readable storage medium. Background Art

[0002] Image style transfer refers to the transfer of the style of one image to another. The original image is also called the content image, which provides content features, and the style image provides style features. The new image is generated by combining the content features and style features. This is called style transfer.

[0003] In the existing image style transfer, a certain established image style is usually made into a template, and the image to be transferred is stylized by the template to obtain the image after style transfer. However, in this processing method, once the style of the template is determined, the processing of the original image is also determined accordingly. The image output after template processing follows the art style of the template, which is relatively rigid. If a new art style needs to be changed, the template must be redesigned, which consumes a lot of human resources as well as art and computing resources, is time-consuming and labor-intensive, and is very inconvenient.

[0004] Therefore, how to solve the above problems has become a problem that needs to be solved urgently by technical personnel. Summary of the invention

[0005] In view of this, the embodiments of the present application provide a method and apparatus for image style conversion, a computing device, and a computer-readable storage medium to address the technical defects in the prior art.

[0006] According to a first aspect of an embodiment of the present application, a method for image style conversion is provided, including:

[0007] Receive the image to be converted and the style image;

[0008] Extracting first feature information of the image to be converted, and extracting second feature information of the style image;

[0009] Inputting the first feature information and the second feature information into a pre-trained reverse network model for processing to generate an initial conversion image;

[0010] The initial conversion image is adjusted according to the image to be converted to obtain a target style conversion image.

[0011] Optionally, extracting first feature information of the image to be converted includes:

[0012] The first feature information of the image to be converted is extracted through a pre-trained convolutional neural network.

[0013] Optionally, extracting second feature information of the style image includes:

[0014] The second feature information of the style image is extracted through a pre-trained convolutional neural network.

[0015] Optionally, the reverse network model includes at least one convolutional layer;

[0016] Inputting the first feature information and the second feature information into a pre-trained reverse network model for processing to generate an initial conversion image, including:

[0017] Processing the first feature information through each of the convolutional layers to extract at least one first feature sub-information in the first feature information;

[0018] Processing the second feature information through each of the convolutional neural networks to extract at least one second feature sub-information in the second feature information;

[0019] Determine second target characteristic sub-information corresponding to each of the first characteristic sub-information;

[0020] The second target characteristic sub-information replaces the corresponding first characteristic sub-information to obtain overfitted initial first characteristic information;

[0021] The initial first feature information is decoded to obtain a corresponding initial conversion picture.

[0022] Optionally, adjusting the initial conversion image according to the image to be converted to obtain a target style conversion image includes:

[0023] Perform pixel interpolation on the image to be converted and the initial conversion image in three primary color channels to obtain a target style conversion image.

[0024] Optionally, performing pixel interpolation on the image to be converted and the initial conversion image in three primary color channels to obtain a target style conversion image includes:

[0025] Obtain a first channel value corresponding to each of the three primary color channels of the image to be converted;

[0026] Obtain a second channel value corresponding to each of the three primary color channels of the initial conversion image;

[0027] The channel value corresponding to each channel of the three primary color channels of the initial conversion image is adjusted according to each of the first channel values ​​and its weight and each of the second channel values ​​and its weight to obtain a target style conversion image.

[0028] Optionally, the reverse network model includes nine convolutional layers.

[0029] According to a second aspect of an embodiment of the present application, a picture style conversion device is provided, including:

[0030] A receiving module, configured to receive the image to be converted and the style image;

[0031] An extraction module, configured to extract first feature information of the image to be converted and second feature information of the style image;

[0032] A generation module is configured to input the first feature information and the second feature information into a pre-trained reverse network model for processing to generate an initial conversion image;

[0033] The adjustment module is configured to adjust the initial conversion image according to the image to be converted to obtain a target style conversion image.

[0034] Optionally, the extraction module is further configured to extract first feature information of the image to be converted through a pre-trained convolutional neural network.

[0035] Optionally, the extraction module is further configured to extract second feature information of the style image through a pre-trained convolutional neural network.

[0036] Optionally, the reverse network model includes at least one convolutional layer;

[0037] The generation module comprises:

[0038] a convolution subunit configured to process the first feature information through each of the convolution layers to extract at least one first feature sub-information in the first feature information, and process the second feature information through each of the convolution neural networks to extract at least one second feature sub-information in the second feature information;

[0039] a determination subunit, configured to determine second target characteristic sub-information corresponding to each of the first characteristic sub-information;

[0040] a replacement subunit, configured to replace the corresponding first feature sub-information with the second target feature sub-information to obtain overfitted initial first feature information;

[0041] The decoding subunit is configured to decode the initial first feature information to obtain a corresponding initial conversion picture.

[0042] Optionally, the adjustment module is further configured to perform pixel interpolation on the image to be converted and the initial conversion image in three primary color channels to obtain a target style conversion image.

[0043] Optionally, the adjustment module is further configured to obtain a first channel value corresponding to each channel of the three primary color channels of the image to be converted; obtain a second channel value corresponding to each channel of the three primary color channels of the initial conversion image; adjust the channel value corresponding to each channel of the three primary color channels of the initial conversion image according to each of the first channel values ​​and their weights and each of the second channel values ​​and their weights to obtain a target style conversion image.

[0044] Optionally, the reverse network model includes nine convolutional layers.

[0045] According to a third aspect of an embodiment of the present application, a computing device is provided, comprising a memory, a processor, and computer instructions stored in the memory and executable on the processor, wherein the processor implements the steps of the image style conversion method when executing the instructions.

[0046] According to a fourth aspect of an embodiment of the present application, a computer-readable storage medium is provided, which stores computer instructions, and when the instructions are executed by a processor, the steps of the image style conversion method are implemented.

[0047] In an embodiment of the present application, a picture to be converted and a style picture are received; first feature information of the picture to be converted is extracted, and second feature information of the style picture is extracted; the first feature information and the second feature information are input into a pre-trained reverse network model for processing to generate an initial conversion picture; the initial conversion picture is adjusted in real time according to the picture to be converted as needed to obtain a target style conversion picture. Through the method provided in the present application, any picture can be migrated according to any style picture. After obtaining the initial conversion picture, the initial conversion picture can be adjusted according to actual needs, making the picture style migration simpler and easier, saving a lot of manpower, material resources, as well as art and computing resources. BRIEF DESCRIPTION OF THE DRAWINGS

[0048] Figure 1 is a structural block diagram of a computing device provided in an embodiment of the present application;

[0049] Figure 2 is a flowchart of a method for image style conversion provided by an embodiment of the present application;

[0050] Figure 3 is a flowchart of a method for generating an initial conversion picture provided by an embodiment of the present application;

[0051] Figure 4 is a flowchart of a method for image style conversion provided by another embodiment of the present application;

[0052] Figure 5a is a picture to be converted provided by an embodiment of the present application;

[0053] Figure 5b It is a style picture provided in the embodiment of the present application;

[0054] Figure 5c is the initial conversion picture provided in the embodiment of the present application;

[0055] Figure 5d is the target style conversion picture provided by the embodiment of the present application;

[0056] Figure 6 It is a structural schematic diagram of the image style conversion device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0057] Many specific details are described in the following description to facilitate a full understanding of the present application. However, the present application can be implemented in many other ways than those described herein, and those skilled in the art can make similar generalizations without violating the connotation of the present application, so the present application is not limited by the specific implementation disclosed below.

[0058] The terms used in one or more embodiments of the present application are only for the purpose of describing specific embodiments, and are not intended to limit one or more embodiments of the present application. The singular forms of "a", "said" and "the" used in one or more embodiments of the present application and the appended claims are also intended to include plural forms, unless the context clearly indicates other meanings. It should also be understood that the term "and / or" used in one or more embodiments of the present application refers to and includes any or all possible combinations of one or more associated listed items.

[0059] It should be understood that, although the terms first, second, etc. may be used to describe various information in one or more embodiments of the present application, these information should not be limited to these terms. These terms are only used to distinguish the same type of information from each other. For example, without departing from the scope of one or more embodiments of the present application, the first may also be referred to as the second, and similarly, the second may also be referred to as the first. Depending on the context, the word "if" as used herein may be interpreted as "at the time of" or "when" or "in response to determining".

[0060] First, the terms involved in one or more embodiments of the present invention are explained.

[0061] Image style transfer: The style of one image is transferred to another image. The original image is also called the content image, which provides content features, and the style image provides style features. The new image is generated by combining the content features and style features. This is called style transfer.

[0062] Reverse network model: Decode and restore the new image based on the received feature information.

[0063] In the present application, a method and apparatus for image style conversion, a computing device, and a computer-readable storage medium are provided, which are described in detail one by one in the following embodiments.

[0064] Figure 1 The structure block diagram of a computing device 100 according to an embodiment of the present application is shown. The components of the computing device 100 include but are not limited to a memory 110 and a processor 120. The processor 120 is connected to the memory 110 via a bus 130, and the database 150 is used to store data.

[0065] The computing device 100 also includes an access device 140 that enables the computing device 100 to communicate via one or more networks 160. Examples of these networks include a public switched telephone network (PSTN), a local area network (LAN), a wide area network (WAN), a personal area network (PAN), or a combination of communication networks such as the Internet. The access device 140 may include one or more of any type of network interface (e.g., a network interface card (NIC)) whether wired or wireless, such as an IEEE 802.11 wireless local area network (WLAN) wireless interface, a World Wide Interoperability for Microwave Access (Wi-MAX) interface, an Ethernet interface, a universal serial bus (USB) interface, a cellular network interface, a Bluetooth interface, a near field communication (NFC) interface, and the like.

[0066] In one embodiment of the present application, the above components of the computing device 100 and Figure 1 Other components not shown in the figure may also be connected to each other, for example, via a bus. It should be understood that Figure 1 The computing device structure block diagram shown is only for the purpose of illustration, and is not intended to limit the scope of the present application. Those skilled in the art may add or replace other components as needed.

[0067] The computing device 100 may be any type of stationary or mobile computing device, including a mobile computer or mobile computing device (e.g., a tablet computer, a personal digital assistant, a laptop computer, a notebook computer, a netbook, etc.), a mobile phone (e.g., a smart phone), a wearable computing device (e.g., a smart watch, smart glasses, etc.), or other types of mobile devices, or a stationary computing device such as a desktop computer or PC. The computing device 100 may also be a mobile or stationary server.

[0068] The processor 120 may execute Figure 2 Steps in the image style transfer method shown. Figure 2 A flowchart of a method for image style conversion according to an embodiment of the present application is shown, including steps 202 to 208.

[0069] Step 202: Receive the image to be converted and the style image.

[0070] Image style transfer refers to transferring the style of one image to another. The original image is also called the content image, which provides content features, and the style image provides style features. The new image is generated by combining the content features and style features. This is called style transfer. For example, the content features of image C are extracted and a new image CS is generated according to the style features of image S, so that the CS image has the content of image C and the texture features of image S. This is called image style transfer of image C, where image C is the image to be converted and image S is the style image. The style of the image to be converted needs to have the texture features of the style image.

[0071] After the style picture is determined, multiple pictures to be converted can be converted into pictures with a unified style. For the same picture to be converted, it can be converted into pictures of different styles according to different style pictures.

[0072] In a specific implementation provided in the present application, a picture A to be converted and a style picture B input by a user are received.

[0073] Step 204: extracting first characteristic information of the image to be converted, and extracting second characteristic information of the style image.

[0074] Specifically, in practical applications, the first feature information of the image to be converted is extracted through a pre-trained convolutional neural network, and the second feature information of the style image is extracted through a pre-trained convolutional neural network.

[0075] Each image has three color value channels, RGB, representing red, green, and blue respectively. The pixels in each channel can be represented by a matrix, and the numerical value represents the pixel value between 0-255. The feature information of the image is extracted by performing convolution operation on the image. The convolution process is based on the convolution kernel. The convolution kernel is continuously scanned on each layer of the pixel matrix according to the step size. The scanned value is multiplied by the number of the corresponding position of the convolution kernel, and then the sum is taken. A value is obtained each time the scan is performed, and a new matrix is ​​generated after all the scans are completed. The convolution kernel is generally a 3*3 matrix. The step size of the convolution kernel refers to the number of grids that the convolution kernel moves each time, in two directions: horizontal and vertical. The convolution kernel is equivalent to a filter to extract the feature information we need.

[0076] The pre-trained convolutional neural network may be a VGG16 network or a VGG19 network. The convolutional neural network is used to extract feature information of an image. The specific convolutional neural network is not limited in the present application.

[0077] In this application, φ() is used to identify a pre-trained convolutional neural network (such as VGG19), then the first feature information of the image C to be converted is represented as φ(C), and the second feature information of the style image S is represented as φ(S).

[0078] In a specific implementation provided in the present application, following the above example, the image A to be converted and the style image B are input into the VGG19 network, and the first feature information φ(A) of the image A to be converted and the second feature information φ(B) of the style image B are respectively extracted through the VGG19 network.

[0079] Step 206: Input the first feature information and the second feature information into a pre-trained reverse network model for processing to generate an initial conversion image.

[0080] The reverse network model is a convolutional neural network, which is used to reversely restore the corresponding image based on the feature information of the image. The reverse network model can be regarded as a decoder of the feature information of the image, which decodes the input feature information and reversely restores the corresponding image. After training, the reverse network model has the ability to replace the partial features closest to the first feature information from the second feature information and directly generate the feature information of the new image. The reverse network model can quickly output the results for any first feature information and second feature information, which is fast, simple and has a high degree of freedom.

[0081] Optionally, the reverse network model includes at least one convolutional layer, and preferably the reverse network model includes nine convolutional layers. In practical applications, increasing the number of convolutional layers can make the final generated image produce an overfitting effect. The overfitted image is convenient for providing artists with a wider adjustment range in the subsequent adjustment process, so that artists can adjust the images to better meet the needs and be more aesthetically pleasing.

[0082] See also Figure 3 , Figure 3 A flowchart of a method for generating an initial conversion image provided by an embodiment of the present application is shown, including steps 302 to 310:

[0083] Step 302: Process the first feature information through each of the convolutional layers to extract at least one first feature sub-information in the first feature information.

[0084] In practical applications, the first feature information needs to be processed by convolution of each convolution layer to extract the first feature sub-information used for replacement. i (C) means, where i∈n c , n c Identifies the amount of first feature sub-information that can be extracted from φ(C).

[0085] Step 304: Process the second feature information through each of the convolutional neural networks to extract at least one second feature sub-information in the second feature information.

[0086] In practical applications, the second feature information is subjected to convolution processing of each of the convolutional layers to extract the second feature sub-information used for replacement from the second feature information. The second feature sub-information is represented by φ j (S) represents, where j∈n s , n s Identifies the amount of second feature sub-information that can be extracted from φ(S).

[0087] Step 306: Determine the second target characteristic sub-information corresponding to each of the first characteristic sub-information.

[0088] For each first feature sub-information φ i (C) It is necessary to determine the second characteristic sub-information φ that best matches it j (S), called For the specific determination method, please refer to the following formula 1:

[0089]

[0090] Among them, φ i (C) is the first characteristic sub-information, φ j (S) is the second feature sub-information, and argmax is a function that finds the parameter (set) of a function. i (C) To s φ j (S) to find the second characteristic sub-information that best matches it, and finally we can find n c indivual

[0091] Step 308: Replace the corresponding first feature sub-information with the second target feature sub-information to obtain overfitted initial first feature information.

[0092] Each Replace the corresponding first feature sub-information φ i (C), and then generate the corresponding initial first feature information. Due to the increase in the number of convolutional layers, an initial first feature information in an overfitting state will be generated.

[0093] Step 310: Decode the initial first feature information to obtain a corresponding initial conversion picture.

[0094] The initial first feature information is decoded accordingly to obtain the corresponding initial conversion image. In the present application, since the initial first feature information is in an overfitting state, the corresponding initial conversion image is also in an overfitting state. The overfitted initial conversion image finally output by the reverse model facilitates a wider adjustment range in the subsequent adjustment process.

[0095] Step 208: adjusting the initial conversion image according to the image to be converted to obtain a target style conversion image.

[0096] After obtaining an overfitted initial conversion picture, it is necessary to adjust the overfitted initial conversion picture according to the picture to be converted so that it better meets the actual needs and obtains a target style conversion picture with better effect.

[0097] Optionally, adjusting the initial conversion image according to the image to be converted to obtain a target style conversion image includes: performing pixel interpolation on the image to be converted and the initial conversion image in three primary color channels to obtain a target style conversion image.

[0098] Each image has three color value channels, RGB, representing red, green, and blue respectively. The pixels in each channel can be represented by a matrix, and the numerical value represents the pixel value between 0 and 255. The specific adjustment method is to adjust the initial conversion image by adjusting the interpolation weights in different channels by performing pixel interpolation between the initial conversion image and the image to be converted, and then generate the target style conversion image.

[0099] Specifically, obtain the first channel value corresponding to each channel of the three primary color channels of the image to be converted; obtain the second channel value corresponding to each channel of the three primary color channels of the initial conversion image; adjust the channel value corresponding to each channel of the three primary color channels of the initial conversion image according to each of the first channel values ​​and their weights and each of the second channel values ​​and their weights to obtain the target style conversion image.

[0100] For the sake of convenience, the image to be converted is represented as 1, the initial conversion image is represented as 2, and the method of adjusting according to the weight in the R channel is R 3 =R 1 *a+R 2 *(1-a), where R 1 is the color value of the image to be converted in the R channel, R 2 is the color value of the initial conversion image in the R channel, a is the weight in the R channel; the method for adjusting according to the weight in the G channel is G 3 =G 1 *b+G 2 *(1-b), where G 1 is the color value of the image to be converted in the G channel, G2 is the color value of the initial conversion image in the G channel, b is the weight in the G channel; the method for adjusting according to the weight in the B channel is B 3 =B 1 *c+B 2 *(1-c), where B 1 is the color value of the image to be converted in the B channel, B 2 is the color value of the initial conversion image in the B channel, and c is the weight in the B channel.

[0101] In practical applications, when the weights corresponding to the three channels R, G, and B are a=b=c, the three channels can be adjusted together. When a, b, and c are different, the results of each channel are adjusted separately. By adjusting the weights and linear interpolation in each channel, a mixed result is obtained, which is finally reflected on the image to obtain the target style conversion image.

[0102] The picture style conversion method provided in the embodiment of the present application receives a picture to be converted and a style picture; extracts first feature information of the picture to be converted, and extracts second feature information of the style picture; inputs the first feature information and the second feature information into a pre-trained reverse network model for processing to generate an initial conversion picture; adjusts the initial conversion picture in real time according to the picture to be converted as needed to obtain a target style conversion picture, adjusts the picture to be converted to the initial conversion picture corresponding to the style picture according to actual needs, and then adjusts the initial conversion picture in real time according to the picture to be converted. Through the method provided in the present application, any picture can be migrated according to any style picture. After obtaining the initial conversion picture, the initial conversion picture can be adjusted according to actual needs, so that the picture style migration is simpler and easier, saving a lot of manpower, material resources, and art and computing resources.

[0103] Secondly, the reverse network model generates an overfitted initial conversion image, which provides a wider adjustment range for subsequent adjustments and obtaining the target style transfer image, and is more conducive to optimizing the image style transfer results.

[0104] Figure 4 A flowchart of a method for image style conversion according to an embodiment of the present application is shown, including steps 402 to 414.

[0105] Step 402: Receive the image to be converted and the style image.

[0106] In the examples provided in this application, see Figure 5a and Figure 5b , Figure 5a shows a picture to be converted provided by an embodiment of the present application, Figure 5b A style picture provided by an embodiment of the present application is shown.

[0107] Step 404: extracting first characteristic information of the image to be converted, and extracting second characteristic information of the style image.

[0108] In the embodiment provided in the present application, the first feature information M of the image to be converted is extracted through the VGG19 network, and the second feature information N of the style image is extracted.

[0109] Step 406: extract at least one first characteristic sub-information from the first characteristic information, and extract at least one second characteristic sub-information from the second characteristic information.

[0110] In the embodiment provided in the present application, the first feature information M and the second feature information N are input into the reverse network model for processing. The reverse neural network includes 9 convolution layers. The first feature information M is processed by the convolution of the 9 convolution layers to obtain the corresponding M. i The first feature sub-information and the second feature information N are processed by convolution of 9 convolution layers to obtain the corresponding N j The second characteristic sub-information.

[0111] Step 408: Determine the second target characteristic sub-information corresponding to each of the first characteristic sub-information.

[0112] In the embodiment provided in the present application, corresponding second target characteristic sub-information is determined for each first characteristic sub-information.

[0113] Step 410: Replace the corresponding first feature sub-information with the second target feature sub-information to obtain overfitted initial first feature information.

[0114] In the embodiment provided in the present application, each second target characteristic sub-information is used to replace the corresponding first characteristic sub-information to obtain overfitted initial first characteristic information T.

[0115] Step 412: Decode the initial first feature information to obtain a corresponding initial conversion picture.

[0116] In the examples provided in this application, see Figure 5c , Figure 5c The initial conversion picture provided by the embodiment of the present application is shown. The initial conversion picture has the style of the style picture based on the picture to be converted, and the picture color of the initial conversion picture is A.

[0117] Step 414: performing pixel interpolation on the image to be converted and the initial conversion image in three primary color channels to obtain a target style conversion image.

[0118] In the embodiment provided in the present application, pixel interpolation is performed in the three primary color channels of the initial converted image and the image to be converted according to a unified weight, and linear interpolation is performed in each channel by adjusting the unified weight to obtain a mixed result corresponding to each channel, and finally a new result is obtained by adjusting the color values ​​of each of the three primary color channels, see Figure 5d , Figure 5d The target style conversion image provided by the embodiment of the present application is shown, and the image color of the target style conversion image is B. The image color of the final target conversion image can be controlled by adjusting the weights, such as the change of the four seasons can be achieved by adjusting the weights of the corresponding color values ​​of each channel.

[0119] The picture style conversion method provided in the embodiment of the present application receives a picture to be converted and a style picture; extracts first feature information of the picture to be converted, and extracts second feature information of the style picture; inputs the first feature information and the second feature information into a pre-trained reverse network model for processing to generate an initial conversion picture; adjusts the initial conversion picture in real time according to the picture to be converted as needed to obtain a target style conversion picture, adjusts the picture to be converted to the initial conversion picture corresponding to the style picture according to actual needs, and then adjusts the initial conversion picture in real time according to the picture to be converted. Through the method provided in the present application, any picture can be migrated according to any style picture. After obtaining the initial conversion picture, the initial conversion picture can be adjusted according to actual needs, so that the picture style migration is simpler and easier, saving a lot of manpower, material resources, and art and computing resources.

[0120] Secondly, the reverse network model generates an overfitted initial conversion image, which provides a wider adjustment range for subsequent adjustments and obtaining the target style transfer image, and is more conducive to optimizing the image style transfer results.

[0121] Corresponding to the above method embodiment, the present application also provides an embodiment of a picture style conversion device, Figure 6 FIG. 1 is a schematic diagram showing the structure of a picture style conversion device according to an embodiment of the present application. Figure 6 As shown, the device comprises:

[0122] The receiving module 602 is configured to receive the image to be converted and the style image.

[0123] The extraction module 604 is configured to extract the first feature information of the image to be converted and extract the second feature information of the style image.

[0124] The generation module 606 is configured to input the first feature information and the second feature information into a pre-trained reverse network model for processing to generate an initial conversion image.

[0125] The adjustment module 608 is configured to adjust the initial conversion image according to the image to be converted to obtain a target style conversion image.

[0126] Optionally, the extraction module 604 is further configured to extract the first feature information of the image to be converted through a pre-trained convolutional neural network.

[0127] Optionally, the extraction module 604 is further configured to extract second feature information of the style image through a pre-trained convolutional neural network.

[0128] Optionally, the reverse network model includes at least one convolutional layer;

[0129] Optionally, the reverse network model includes nine convolutional layers.

[0130] The generating module 606 includes:

[0131] a convolution subunit configured to process the first feature information through each of the convolution layers to extract at least one first feature sub-information in the first feature information, and process the second feature information through each of the convolution neural networks to extract at least one second feature sub-information in the second feature information;

[0132] a determination subunit, configured to determine second target characteristic sub-information corresponding to each of the first characteristic sub-information;

[0133] a replacement subunit, configured to replace the corresponding first feature sub-information with the second target feature sub-information to obtain overfitted initial first feature information;

[0134] The decoding subunit is configured to decode the initial first feature information to obtain a corresponding initial conversion picture.

[0135] Optionally, the adjustment module 608 is further configured to perform pixel interpolation on the to-be-converted image and the initial conversion image in three primary color channels to obtain a target style conversion image.

[0136] Optionally, the adjustment module 608 is further configured to obtain a first channel value corresponding to each channel of the three primary color channels of the image to be converted; obtain a second channel value corresponding to each channel of the three primary color channels of the initial conversion image; adjust the channel value corresponding to each channel of the three primary color channels of the initial conversion image according to each of the first channel values ​​and their weights and each of the second channel values ​​and their weights to obtain a target style conversion image.

[0137] The image style conversion device provided in the embodiment of the present application receives a picture to be converted and a style picture; extracts first feature information of the picture to be converted, and extracts second feature information of the style picture; inputs the first feature information and the second feature information into a pre-trained reverse network model for processing to generate an initial conversion picture; adjusts the initial conversion picture in real time according to the picture to be converted as needed to obtain a target style conversion picture, adjusts the picture to be converted to the initial conversion picture corresponding to the style picture according to actual needs, and then adjusts the initial conversion picture in real time according to the picture to be converted. Through the method provided in the present application, any picture can be migrated according to any style picture. After obtaining the initial conversion picture, the initial conversion picture can be adjusted according to actual needs, so that the picture style migration is simpler and easier, saving a lot of manpower, material resources, and art and computing resources.

[0138] Secondly, the reverse network model generates an overfitted initial conversion image, which provides a wider adjustment range for subsequent adjustments and obtaining the target style transfer image, and is more conducive to optimizing the image style transfer results.

[0139] In one embodiment of the present application, a computing device is also provided, including a memory, a processor, and computer instructions stored in the memory and executable on the processor, wherein the processor implements the steps of the image style conversion method when executing the instructions.

[0140] An embodiment of the present application further provides a computer-readable storage medium storing computer instructions, which, when executed by a processor, implement the steps of the image style conversion method as described above.

[0141] The above is a schematic scheme of a computer-readable storage medium of this embodiment. It should be noted that the technical scheme of the storage medium and the technical scheme of the above-mentioned picture style conversion method belong to the same concept, and the details not described in detail in the technical scheme of the storage medium can be referred to the description of the technical scheme of the above-mentioned picture style conversion method.

[0142] The above describes specific embodiments of the present application. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recorded in the claims can be performed in an order different from that in the embodiments and still achieve the desired results. In addition, the processes depicted in the accompanying drawings do not necessarily require the specific order or continuous order shown to achieve the desired results. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0143] The computer instructions include computer program codes, which may be in source code form, object code form, executable files or some intermediate forms, etc. The computer-readable medium may include: any entity or device capable of carrying the computer program code, recording medium, USB flash drive, mobile hard disk, magnetic disk, optical disk, computer memory, read-only memory (ROM), random access memory (RAM), electric carrier signal, telecommunication signal and software distribution medium, etc. It should be noted that the content contained in the computer-readable medium may be appropriately increased or decreased according to the requirements of legislation and patent practice in the jurisdiction. For example, in some jurisdictions, according to legislation and patent practice, computer-readable media do not include electric carrier signals and telecommunication signals.

[0144] It should be noted that, for the above-mentioned method embodiments, for the sake of simplicity of description, they are all expressed as a series of action combinations, but those skilled in the art should be aware that the present application is not limited by the described action sequence, because according to the present application, certain steps can be performed in other sequences or simultaneously. Secondly, those skilled in the art should also be aware that the embodiments described in the specification are all preferred embodiments, and the actions and modules involved are not necessarily required by the present application.

[0145] In the above embodiments, the description of each embodiment has its own emphasis. For parts that are not described in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0146] The preferred embodiments of the present application disclosed above are only used to help explain the present application. The optional embodiments do not describe all the details in detail, nor do they limit the invention to the specific implementation methods described. Obviously, many modifications and changes can be made according to the content of the present application. The present application selects and specifically describes these embodiments in order to better explain the principles and practical applications of the present application, so that those skilled in the art can understand and use the present application well. The present application is only limited by the claims and their full scope and equivalents.

Claims

1. A method for image style transfer, characterized in that: include: Receive the image to be converted and the style image; Extracting first feature information of the image to be converted, and extracting second feature information of the style image, wherein the first feature information includes overfitting feature information; Inputting the first feature information and the second feature information into a pre-trained reverse network model for processing to generate an initial conversion image in an overfitting state, wherein the reverse network model includes at least one convolutional layer; The step of inputting the first feature information and the second feature information into a pre-trained reverse network model for processing to generate an initial conversion picture in an overfitting state includes: processing the first feature information through each of the convolutional layers to extract at least one first feature sub-information in the first feature information; processing the second feature information through each of the convolutional neural networks to extract at least one second feature sub-information in the second feature information; determining the second target feature sub-information corresponding to each of the first feature sub-information; replacing the corresponding first feature sub-information with the second target feature sub-information to obtain the overfitted initial first feature information; and decoding the initial first feature information to obtain the corresponding initial conversion picture; The initial conversion image is adjusted according to the image to be converted to obtain a target style conversion image.

2. The image style conversion method according to claim 1, characterized in that: Extracting first feature information of the image to be converted includes: The first feature information of the image to be converted is extracted through a pre-trained convolutional neural network.

3. The image style conversion method according to claim 1, characterized in that: Extracting second feature information of the style image includes: The second feature information of the style image is extracted through a pre-trained convolutional neural network.

4. The image style conversion method according to claim 1, characterized in that: Adjusting the initial conversion image according to the image to be converted to obtain a target style conversion image includes: Perform pixel interpolation on the image to be converted and the initial conversion image in three primary color channels to obtain a target style conversion image.

5. The image style conversion method according to claim 4, characterized in that: Performing pixel interpolation on the image to be converted and the initial conversion image in three primary color channels to obtain a target style conversion image, including: Obtain a first channel value corresponding to each of the three primary color channels of the image to be converted; Obtain a second channel value corresponding to each of the three primary color channels of the initial conversion image; The channel value corresponding to each channel of the three primary color channels of the initial conversion image is adjusted according to each of the first channel values ​​and its weight and each of the second channel values ​​and its weight to obtain a target style conversion image.

6. The image style conversion method according to any one of claims 1 to 5, characterized in that: The reverse network model includes nine convolutional layers.

7. A picture style conversion device, characterized in that: include: A receiving module, configured to receive the image to be converted and the style image; An extraction module, configured to extract first feature information of the image to be converted and second feature information of the style image, wherein the first feature information includes overfitting feature information; A generation module is configured to input the first feature information and the second feature information into a pre-trained reverse network model for processing to generate an initial conversion image in an overfitting state, wherein the reverse network model includes at least one convolutional layer; The generating module further includes: a convolution subunit configured to process the first feature information through each of the convolution layers to extract at least one first feature sub-information in the first feature information, and process the second feature information through each of the convolution neural networks to extract at least one second feature sub-information in the second feature information; a determination subunit configured to determine the second target feature sub-information corresponding to each of the first feature sub-information; a replacement subunit configured to replace the corresponding first feature sub-information with the second target feature sub-information to obtain the overfitted initial first feature information; and a decoding subunit configured to decode the initial first feature information to obtain the corresponding initial conversion picture; The adjustment module is configured to adjust the initial conversion image according to the image to be converted to obtain a target style conversion image.

8. The image style conversion device according to claim 7, characterized in that: The extraction module is further configured to extract the first feature information of the image to be converted through a pre-trained convolutional neural network.

9. The image style conversion device according to claim 7, characterized in that: The extraction module is further configured to extract second feature information of the style image through a pre-trained convolutional neural network.

10. The image style conversion device according to claim 7, characterized in that: The adjustment module is further configured to perform pixel interpolation on the image to be converted and the initial conversion image in three primary color channels to obtain a target style conversion image.

11. The image style conversion device according to claim 10, characterized in that: The adjustment module is further configured to obtain a first channel value corresponding to each channel of the three primary color channels of the image to be converted; and obtain a second channel value corresponding to each channel of the three primary color channels of the initial conversion image; The channel value corresponding to each channel of the three primary color channels of the initial conversion image is adjusted according to each of the first channel values ​​and its weight and each of the second channel values ​​and its weight to obtain a target style conversion image.

12. The image style conversion device according to any one of claims 7 to 11, characterized in that: The reverse network model includes nine convolutional layers.

13. A computing device comprising a memory, a processor, and computer instructions stored in the memory and executable on the processor, characterized in that: When the processor executes the instructions, the steps of the method according to any one of claims 1 to 6 are implemented.

14. A computer-readable storage medium storing computer instructions, characterized in that: When the instruction is executed by a processor, the steps of the method described in any one of claims 1 to 6 are implemented.

Images