A method and system for neural image style transfer

Abstract

The application discloses a kind of neural image style migration methods, comprising the following steps: obtaining content image data and style image data;Content image data and style image data are analyzed semantically, and content image data features and style image data features are extracted;Output image data is generated by combining content image data features and style image data features using neural style migration method;Whether output image data meets the standard is judged using first rule;When output image data meets the standard, output image data is exported, and output image data is transmitted to a preset terminal;When neural style migration is carried out, the object in content image and the background in style image are distinguished, and the content object in content image is given higher weight, which is more helpful to retain the content of meaningful image elements in content image, and can effectively improve the quality of output image.

Description

Technical Field

[0001] This invention relates to the field of image processing, and more specifically to a method for neural image style transfer. Background Technology

[0002] Existing neural style transfer algorithms obtain satisfactory generated images after neural style transfer by optimizing the total loss function, which is a weighted sum of the loss of the content image and the loss of the style image. These methods treat the content image and the style image as a whole, without paying attention to the semantics of the content image. In other words, they assign the same importance to objects in the content image and backgrounds in the style image without making distinctions, resulting in a low degree of preservation of meaningful image elements in the output image.

[0003] To further improve the preservation of the original content of images, a neural image style transfer method is proposed. Summary of the Invention

[0004] The technical problem to be solved by this invention is: how to solve the problem that the existing technology does not treat objects in the content image and background in the style image differently, resulting in a low degree of retention of meaningful image elements in the output image, and provides a neural image style transfer method.

[0005] The present invention solves the above-mentioned technical problems through the following technical solution, and the present invention includes the following steps:

[0006] S1. Obtain content image data and style image data;

[0007] S2. Perform semantic analysis on the content image data and style image data, and extract the features of the content image data and style image data;

[0008] S3. Combine the features of content image data and style image data using the neural style transfer method to generate output image data;

[0009] S4. Use the first rule to determine whether the output image data meets the standard;

[0010] S5. When the output image data meets the requirements, export the output image data and transmit the output image data to the preset terminal.

[0011] Preferably, S2 specifically includes:

[0012] First, a convolutional neural network is used to perform semantic analysis on the content image data and style image data;

[0013] The content image data is then decomposed into P1, P2, ..., Pn and Q, where P1, P2, ..., Pn are the extracted content object data and Q is the background image data;

[0014] Finally, content image data features and style image data features are extracted based on the content object data and background image data.

[0015] Preferably, S4 specifically includes:

[0016] First, assign different weights to the content image data and the style image data, and require that the sum of the weights of the content image data and the style image data be 1, and that the weight of the content image data be greater than the weight of the style image data.

[0017] First, calculate the image data loss of the output image according to the first rule, which is:

[0018] L total (G)=α1L P1 (C,G)+α2L P2 (C,G)+…+α n L Pn (C,G)+α q L Q (C,G)+βL style (S,G)

[0019] Where Ltotal(G) is the loss function, α is the weight of the content image data, β is the weight of the style image data, C is the content image data, and S is the style image data;

[0020] Next, determine whether the output image data meets the standards. The specific determination process is as follows:

[0021] When Ltotal(G) ≤ preset threshold Q, it means that the output image data meets the standard;

[0022] When Ltotal(G) > the preset threshold Q, it means that the output image data does not meet the standard.

[0023] Preferably, when Ltotal(G) > preset threshold Q, the content image data and style image data are returned to step S2, the number of content image data decompositions is increased, and steps S2 to S4 are repeated.

[0024] A neural image style transfer system, comprising:

[0025] The image acquisition module is used to acquire content image data and style image data;

[0026] The image analysis module is used to perform semantic analysis on content image data and style image data, and extract features from content image data and style image data.

[0027] The image processing module is used to combine content image data features and style image data features using the neural style transfer method and generate output image data.

[0028] The image detection module is used to determine the compliance level of the output image data using a first rule;

[0029] The image output module is used to export the output image data when the output image data meets the requirements, and to transmit the output image data to a preset terminal.

[0030] Preferably, the specific processing procedure of the image analysis module is as follows:

[0031] First, a convolutional neural network is used to perform semantic analysis on the content image data and style image data;

[0032] The content image data is then decomposed into P1, P2, ..., Pn and Q, where P1, P2, ..., Pn are the extracted content object data and Q is the background image data;

[0033] Finally, content image data features and style image data features are extracted based on the content object data and background image data.

[0034] Preferably, the specific processing procedure of the image detection module is as follows:

[0035] First, assign different weights to the content image data and the style image data, and require that the sum of the weights of the content image data and the style image data be 1, and that the weight of the content image data be greater than the weight of the style image data.

[0036] First, calculate the image data loss of the output image according to the first rule, which is:

[0037] L total (G)=α1L P1 (C,G)+α2L P2 (C,G)+…+α n L Pn (C,G)+α q L Q (C,G)+βL style (S,G)

[0038] Where Ltotal(G) is the loss function, α is the weight of the content image data, β is the weight of the style image data, C is the content image data, and S is the style image data;

[0039] Next, determine whether the output image data meets the standards. The specific determination process is as follows:

[0040] When Ltotal(G) ≤ preset threshold Q, it means that the output image data meets the standard;

[0041] When Ltotal(G) > the preset threshold Q, it means that the output image data does not meet the standard.

[0042] Preferably, when Ltotal(G) > preset threshold Q, the content image data and style image data are returned to the image analysis module, the number of content image data decompositions is increased, and steps S2 to S4 are repeated.

[0043] Compared with the prior art, the present invention has the following advantages: When performing neural image style transfer, the method distinguishes between objects in the content image and background in the style image, giving higher weight to content objects in the content image. This helps to retain the content of meaningful image elements in the content image and can effectively improve the quality of the output image. Attached Figure Description

[0044] Figure 1 This is an overall structural diagram of the present invention. Detailed Implementation

[0045] The embodiments of the present invention are described in detail below. These embodiments are implemented based on the technical solution of the present invention, and provide detailed implementation methods and specific operation processes. However, the scope of protection of the present invention is not limited to the following embodiments.

[0046] like Figure 1 As shown, this embodiment provides a technical solution: a method for neural image style transfer, comprising the following steps:

[0047] S1. Obtain content image data and style image data;

[0048] S2. Perform semantic analysis on the content image data and style image data, and extract the features of the content image data and style image data;

[0049] S3. Combine the features of content image data and style image data using the neural style transfer method to generate output image data;

[0050] S4. Use the first rule to determine whether the output image data meets the standard;

[0051] S5. When the output image data meets the requirements, export the output image data and transmit the output image data to the preset terminal.

[0052] This neural image style transfer method treats objects in the content image and background in the style image differently during neural style transfer, giving higher weight to content objects in the content image. This helps to preserve the content of meaningful image elements in the content image and can effectively improve the quality of the output image.

[0053] Specifically, S2 is:

[0054] First, a convolutional neural network is used to perform semantic analysis on the content image data and style image data;

[0055] The content image data is then decomposed into P1, P2, ..., Pn and Q, where P1, P2, ..., Pn are the extracted content object data and Q is the background image data;

[0056] Finally, content image data features and style image data features are extracted based on the content object data and background image data.

[0057] First, semantic segmentation is performed on the content image data to distinguish between content object features and background image features, so that different weights can be assigned to the two. When performing image style transfer, the content object can retain the original content as much as possible.

[0058] Specifically, S4 is:

[0059] First, assign different weights to the content image data and the style image data, and require that the sum of the weights of the content image data and the style image data be 1, and that the weight of the content image data be greater than the weight of the style image data.

[0060] First, calculate the image data loss of the output image according to the first rule, which is:

[0061] L total (G)=α1L P1 (C,G)+α2L P2 (C,G)+…+α n L Pn (C,G)+α q L Q (C,G)+βL style (S,G)

[0062] Where Ltotal(G) is the loss function, α is the weight of the content image data, β is the weight of the style image data, C is the content image data, and S is the style image data;

[0063] Next, determine whether the output image data meets the standards. The specific determination process is as follows:

[0064] When Ltotal(G) ≤ preset threshold Q, it means that the output image data meets the standard;

[0065] When Ltotal(G) > the preset threshold Q, it means that the output image data does not meet the standard.

[0066] Furthermore, when Ltotal(G) > preset threshold Q, the content image data and style image data are returned to step S2, the number of content image data decompositions is increased, and steps S2 to S4 are repeated again.

[0067] Repeating the process on the data makes the transformed style image clearer and retains more of the original content.

[0068] A neural image style transfer system, comprising:

[0069] The image acquisition module is used to acquire content image data and style image data;

[0070] The image analysis module is used to perform semantic analysis on content image data and style image data, and extract features from content image data and style image data.

[0071] The image processing module is used to combine content image data features and style image data features using the neural style transfer method and generate output image data.

[0072] The image detection module is used to determine the compliance level of the output image data using a first rule;

[0073] The image output module is used to export the output image data when the output image data meets the requirements, and to transmit the output image data to a preset terminal.

[0074] The specific processing steps of the image analysis module are as follows:

[0075] First, a convolutional neural network is used to perform semantic analysis on the content image data and style image data;

[0076] The content image data is then decomposed into P1, P2, ..., Pn and Q, where P1, P2, ..., Pn are the extracted content object data and Q is the background image data;

[0077] Finally, content image data features and style image data features are extracted based on the content object data and background image data.

[0078] The specific processing procedure of the image detection module is as follows:

[0079] First, assign different weights to the content image data and the style image data, and require that the sum of the weights of the content image data and the style image data be 1, and that the weight of the content image data be greater than the weight of the style image data.

[0080] First, calculate the image data loss of the output image according to the first rule, which is:

[0081] L total (G)=α1LP1 (C,G)+α2L P2 (C,G)+…+α n L Pn (C,G)+α q L Q (C,G)+βL style (S,G)

[0082] Where Ltotal(G) is the loss function, α is the weight of the content image data, β is the weight of the style image data, C is the content image data, and S is the style image data;

[0083] Next, determine whether the output image data meets the standards. The specific determination process is as follows:

[0084] When Ltotal(G) ≤ preset threshold Q, it means that the output image data meets the standard;

[0085] When Ltotal(G) > the preset threshold Q, it means that the output image data does not meet the standard.

[0086] Furthermore, when Ltotal(G) > the preset threshold Q, the content image data and style image data are returned to the image analysis module, the number of content image data decompositions is increased, and steps S2 to S4 are repeated again.

[0087] Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include at least one of that feature. In the description of this invention, "a plurality of" means at least two, such as two, three, etc., unless otherwise explicitly specified.

[0088] In the description of this specification, the references to terms such as "one embodiment," "some embodiments," "example," "specific example," or "some examples," etc., indicate that a specific feature, structure, material, or characteristic described in connection with that embodiment or example is included in at least one embodiment or example of the present invention. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples. Moreover, without contradiction, those skilled in the art can combine and integrate the different embodiments or examples described in this specification, as well as the features of different embodiments or examples.

[0089] Although embodiments of the present invention have been shown and described above, it is understood that the above embodiments are exemplary and should not be construed as limiting the present invention. Those skilled in the art can make changes, modifications, substitutions and variations to the above embodiments within the scope of the present invention.

Claims

1. A method for neural image style transfer, characterized in that, Includes the following steps: S1. Obtain content image data and style image data; S2. Perform semantic analysis on the content image data and style image data, and extract the features of the content image data and style image data; S3. Combine the features of content image data and style image data using the neural style transfer method to generate output image data; S4. Use the first rule to determine whether the output image data meets the standard; S5. When the output image data meets the standard, export the output image data and transmit the output image data to the preset terminal; Specifically, S2 is: First, a convolutional neural network is used to perform semantic analysis on the content image data and style image data; The content image data is then decomposed into P1, P2, ..., Pn and Q, where P1, P2, ..., Pn are the extracted content object data, and Q is the background image data; Finally, content image data features and style image data features are extracted based on content object data and background image data; Specifically, S4 is: First, assign different weights to the content image data and the style image data, and require that the sum of the weights of the content image data and the style image data be 1, and that the weight of the content image data be greater than the weight of the style image data. First, calculate the image data loss of the output image according to the first rule, which is: L total (G)= α1L P1 (C, G) + α2L P2 (C,G) + ... + α n L Pn (C,G) +α q L Q (C,G) +βL style (S,G) Where Ltotal(G) is the loss function, α is the weight of the content image data, β is the weight of the style image data, C is the content image data, and S is the style image data; Next, determine whether the output image data meets the standards. The specific determination process is as follows: When Ltotal(G) ≤ preset threshold Q, it means that the output image data meets the standard; When Ltotal(G) > the preset threshold Q, it means that the output image data does not meet the standard; When Ltotal(G) > preset threshold Q, return the content image data and style image data to step S2, increase the number of content image data decompositions, and repeat steps S2 to S4.

2. A neural image style transfer system, characterized in that, include: The image acquisition module is used to acquire content image data and style image data; The image analysis module is used to perform semantic analysis on content image data and style image data, and extract features from content image data and style image data. The image processing module is used to combine content image data features and style image data features using the neural style transfer method and generate output image data. The image detection module is used to determine the compliance level of the output image data using a first rule; The image output module is used to export the output image data when the output image data meets the requirements, and to transmit the output image data to the preset terminal. The specific processing procedure of the image analysis module is as follows: First, a convolutional neural network is used to perform semantic analysis on the content image data and style image data; The content image data is then decomposed into P1, P2, ..., Pn and Q, where P1, P2, ..., Pn are the extracted content object data, and Q is the background image data; Finally, content image data features and style image data features are extracted based on content object data and background image data; The specific processing procedure of the image detection module is as follows: First, assign different weights to the content image data and the style image data, and require that the sum of the weights of the content image data and the style image data be 1, and that the weight of the content image data be greater than the weight of the style image data. First, calculate the image data loss of the output image according to the first rule, which is: L total (G)= α1L P1 (C, G) + α2L P2 (C,G) + ... + α n L Pn (C,G) +α q L Q (C,G) +βL style (S,G) Where Ltotal(G) is the loss function, α is the weight of the content image data, β is the weight of the style image data, C is the content image data, and S is the style image data; Next, determine whether the output image data meets the standards. The specific determination process is as follows: When Ltotal(G) ≤ preset threshold Q, it means that the output image data meets the standard; When Ltotal(G) > the preset threshold Q, it means that the output image data does not meet the standard; When Ltotal(G) > preset threshold Q, the content image data and style image data are returned to the image analysis module, the number of content image data decompositions is increased, and steps S2 to S4 are repeated.

Images