A High-Quality Realistic Image Style Transfer Method and System Based on Composite Filtering

Abstract

The present invention relates to the technical field of computer image processing, and in particular to a method for high-quality realistic image style transfer based on composite filtering, including semantic segmentation of images; performing per-channel color transfer and boundary feature repair on the input image and the reference image to obtain a feature-repaired image, performing per-channel feature loss estimation on the ILS structure layer of the feature-repaired image, converting it into texture layer feature loss, and texture loss compensation to obtain the final ILS texture layer; fusing the structure layer of the feature-repaired image with the final ILS texture layer to obtain an image with high-quality realistic image style transfer, solving the contradiction between maintaining the texture features of the complete image and removing the color style information of the original image, and moreover, processing input images or reference style images with bright colors of high quality, and can also transfer the reference style containing complex styles to the input image with high quality. The present invention also discloses a high-quality realistic image style transfer system based on composite filtering.

Description

Technical Field

[0001] The present invention relates to the technical field of computer image processing, and in particular to a high-quality realistic image style transfer method and system based on composite filtering. Background Art

[0002] Realistic image style transfer technology refers to the technology that uses various algorithms to transfer the style (such as color, etc.) of a reference image to an input image while keeping the features of the original image unchanged, and regenerates a realistic image style transfer result image. The essence of realistic image style transfer technology is to transfer the color of a reference image to the input image.

[0003] There are two mainstream technologies in the existing realistic image color transfer technology: one is the image style transfer method based on the loss function of reference style loss and input content feature loss. This type of method often needs to make a trade-off between the saliency of style transfer and feature preservation, resulting in the two being incompatible. The second is to whiten (de-style) the input image, which can also be regarded as a feature extraction process. Then the reference image style is transferred on this basis. This type of method adds a reference style on the basis of feature extraction, which is also prone to feature loss. Therefore, both methods are prone to cause the image style transfer and content feature preservation modules to merge and interfere with each other, resulting in color overflow, feature blur and other problems in the final generated image style transfer results.

[0004] Another method is to decouple the image based on the WLS image filter, perform color migration on the structure layer, and process the feature layer separately. This method can keep the image features unaffected by style migration. However, when the WLS filter extracts image features, the feature layer extracted by the WLS filter usually contains the color information of the image, resulting in incomplete style migration in the final output. The greater the correlation between the input image color and the reference image color, the more serious the phenomenon of incomplete style migration will be. Figure 9 (c) Figure 10 (d) and Figure 11 As shown in (e) in . Summary of the invention

[0005] The purpose of the present invention is to provide a high-quality realistic image style transfer method and system based on composite filtering to solve the contradiction between maintaining complete image features and removing the color style information of the original image, and to generate a method for complex color transfer of realistic images without losing or even enhancing the features of the input image.

[0006] In order to achieve the above object, the technical solution adopted by the present invention is:

[0007] A high-quality realistic image style transfer method based on composite filtering, comprising the following steps:

[0008] S1. Perform semantic segmentation on the input image and the reference image;

[0009] S2. Decouple the input image and the reference image using the ILS filter to obtain the ILS structure layer and the ILS texture layer of the input image and the reference image;

[0010] S3. Perform R, G, B channel-by-channel color transfer on the ILS structure layer of the reference image in step S2 to the structure layer of the input image to obtain a new ILS structure layer, and combine the new ILS structure layer with the ILS texture layer of the input image to obtain a new image;

[0011] S4. Use the TE module to repair the boundary features of the ILS structure layer of the new image in step S3 to obtain the ILS structure layer of the final image;

[0012] S5. Estimate the loss of texture features caused by the boundary feature repair in step S4, and convert the loss caused in the structure layer of the final image into the texture feature loss of the texture layer of the final image;

[0013] S6. Compensate the texture feature loss of the texture layer of the final image in step S5 to obtain the ILS texture layer of the final image;

[0014] S7. Fuse the ILS structure layer of the final image in step S4 with the final ILS texture layer in step S6 to obtain an image with high-quality realistic image style transfer.

[0015] Further, in step S3, by taking as the input image, R as the reference image, taking and R and its semantic segmentation input encoder-decoder, performing color transfer based on the principle of WCT, and extracting the texture features of are as follows:

[0016]

[0017] where is a diagonal matrix with the covariance matrix eigenvalues; is the vectorized VGG feature of; Ch is the number of channels; is the orthogonal matrix corresponding to the eigenvector, then there is:

[0018]

[0019] Transfer the color of to There are:

[0020]

[0021] where is the color transfer result; is a diagonal matrix with the eigenvalues of the covariance matrix ; is the orthogonal matrix corresponding to the eigenvectors, satisfying:

[0022]

[0023] where m is the average value of ; The decoder obtains the color transfer result of the image in the RGB channels for the input ;

[0024] In formulas (3.1)-(3.4), replace with the structural layer obtained by filtering the j channel of using the ILS filter; Replace with the structural layer obtained by filtering the j channel of using the ILS filter to obtain the structural layer S of the color transfer result in the RGB channels j ′, where j ∈ {R, G, B}; Combine the new ILS structural layer with the texture layer of the input image to obtain a new image W.

[0025] Furthermore, in step S4, for boundary feature repair of the structural layer of the new image W, there are:

[0026]

[0027] where W′ is the feature repair result; represents the WLS filter operation, represents the structural layer obtained by filtering W using the WLS filter, represents the texture layer obtained by filtering using the WLS filter; The AWLS texture layer of

[0028]

[0029] where k1 and k1′ are the weights of and respectively;

[0030] According to formula (4.2), for the new image W in step S3, the AWLS texture layer of W obtained by using the AWLS filter is as follows:

[0031]

[0032] According to formula (4.2), the ILS texture layer without the color of the original image obtained by the ILS filter is as follows: There are:

[0033]

[0034] Among them, represents the filtering operation of the ILS filter,

[0035] and respectively represent the structure layer and the texture layer obtained by filtering with the ILS filter; k2 and k2′ are the weights of and respectively, and are set to k2 = k2′ = 1; obtained according to formula (4.3) and formula (4.4):

[0036]

[0037] Among them, and respectively represent the structure layer and the texture layer obtained by filtering W with the ILS filter;

[0038] According to formula (4.1), when repairing the j channel of S′, the repair result of S j ′ can be obtained as:

[0039]

[0040] Among them, S j is the repair result of the output W of the ILS structure layer of WCT by the TE module on the j channel; 2 is the structure layer obtained by filtering the structure layer obtained by filtering W j with the ILS filter and then filtering with the AWLS filter; represents the structure layer obtained by filtering the j channel of with the ILS filter; represents the structure layer obtained by filtering

[0041]

[0042] Further, in step S2, the AWLS filter is a filter designed specifically for image style transfer based on the WLS filter.

[0042] In the WLS filter, the factors controlling the image smoothing degree are as follows:

[0043]

[0044] In formula (1.1), f is the input image; l is the luminance information of image f; s is the pixel position; ε′ is a very small constant value; α′ is the smoothing weight value.

[0045] In the ILS filter, the factors controlling the image smoothing degree are as follows:

[0046]

[0047] Among them, p is the smoothing factor.

[0048] When ε and ε′ are ignored, we obtain:

[0049]

[0050] Compare the factors controlling the image smoothing degree in the WLS filter with those in the ILS filter, and by judging the edge-preserving smoothing capabilities of the ILS filter and the WLS filter, use the ILS filter to decouple the input image and the reference image, so as to obtain an ILS structure layer with a stronger edge-preserving effect and an ILS texture layer with a better effect of removing the color of the input image.

[0051] Furthermore, in step S5, the calculation method of the loss estimation is as follows:

[0052]

[0053] Converting the loss caused in the structure layer of the final image into the texture feature loss of the final image has:

[0054]

[0055] Among them, is to extract the structure layer of W using the WLS filter, and obtain according to formula (5.1):

[0056]

[0057] According to formula (1.3), when ε and ε′ are ignored, the result of filtering the image by the WLS filter is approximately equal to the result of filtering the image by the ILS filter first and then by the WLS filter, and obtain according to formula (5.3):

[0058]

[0059] Substitute formula (5.4) into formula (5.1) to get:

[0060]

[0061] Among them, δ represents the feature loss of the final image structure layer.

[0062] Furthermore, in step S6, texture feature loss compensation is performed on the texture of the final image described in step S5:

[0063] W j ′ extracts the WLS filter features and is obtained through formula (5.1):

[0064]

[0065] Approximately represent δ according to formula (5.4) and formula (6.1) to obtain:

[0066]

[0067] Among them, δ' is the compensated texture feature loss, and K1 and K”1 are weights respectively. The difference between δ and δ' can be understood as the texture feature difference between the problem boundary and the natural boundary.

[0068] Furthermore, in step S6, after the texture layer of the final image is compensated for texture feature loss, the ILS texture layer of the final image is obtained;

[0069] is in formula (6.2) Combined with that in step 2 Decoupled by the ILS filter to obtain Let K1 be less than or equal to 0.4 and K1′ be greater than or equal to 0.6.

[0070] Furthermore, in step S7, the calculation method for fusing the ILS structure layer of the final image and the ILS texture layer of the final image is:

[0071] O j = γ1S j + γ2F j Formula (7.1)

[0072] Among them, γ1 and γ2 are the weights of S j and F j respectively.

[0073] Furthermore, obtain according to formula (7.1)(6.3)(4.2)

[0074]

[0075] Among them, the sum of the weights of the two structural layers is 0, and the weights of other items are 1. Then we have: γ1 - γ2K1k1′ = 0, γ2K1k1 + γ2K1′ = 0, γ1 = k1′, γ2 = K1 = 1, k1 = 0.8, K1′ = 0.2; Different degrees of color transfer results are obtained by adjusting the values of γ1 and -k1′.

[0076] A realistic image style transfer system based on composite filtering includes a processor and a storage device. One or more processors are provided, and the storage device is used to store one or more programs to execute the method of steps S1 - S7.

[0077] The beneficial effects of the present invention are:

[0078] In the present invention, ILS is used to extract the texture layer with less features and no color information of the input image, and the structural layer containing most of the image information is extracted. Color transfer is performed on the structural layer, and WLS is used to repair the boundaries of the structural layer, and the feature loss caused by the repair is estimated and compensated, so as to solve the contradictory problem of removing the color style information of the original image and highly maintaining the features of the original image, and generate a method for realistic and complex image color transfer. Compared with the existing image style transfer methods, the present invention can not only process input images or reference style images with bright colors with high quality, but also transfer the reference style containing complex styles (such as containing multiple colors) to the input image with high quality. BRIEF DESCRIPTION OF THE DRAWINGS

[0079] Figure 1 It is a flowchart of the steps of the transfer method in a preferred embodiment of the present invention.

[0080] Figure 2 It is an algorithm framework diagram of the transfer method in a preferred embodiment of the present invention.

[0081] Figure 3 It is a feature retention and ECSL algorithm framework diagram of the transfer method in a preferred embodiment of the present invention.

[0082] Figure 4 It is an ablation experiment effect diagram of the transfer method in a preferred embodiment of the present invention.

[0083] Figure 5 It is a comparison diagram of the filtering results of multiple filters of the transfer method in a preferred embodiment of the present invention.

[0084] Figure 6 It is a comparison diagram of the feature retention and style transfer effects before and after adding ECSL of the transfer method in a preferred embodiment of the present invention.

[0085] Figure 7 It is a style transfer effect diagram of different weights of the transfer method in a preferred embodiment of the present invention.

[0086] Figure 8 It is the color transfer effect diagram with different weights of the transfer method in a preferred embodiment of the present invention.

[0087] Figure 9 It is the comparison diagram of the transfer method and the incomplete transfer effect to be solved in a preferred embodiment of the present invention.

[0088] Figure 10 It is a schematic diagram of incomplete style transfer caused by the large correlation between the color of the input image and the color of the reference image.

[0089] Figure 11 It is the comparison diagram of the transfer effects of different transfer methods. Specific Embodiment

[0090] Next, the technical solutions in the embodiments of the present invention will be clearly and completely described in conjunction with the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all the embodiments. All other embodiments obtained by those of ordinary skill in the art based on the embodiments of the present invention without creative efforts shall fall within the protection scope of the present invention.

[0091] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by those of ordinary skill in the technical field to which the present invention belongs. The terms used in the description of the present invention in this specification are only for the purpose of describing specific embodiments and are not intended to limit the present invention. The term "and / or" used herein includes any and all combinations of one or more of the related listed items.

[0092] A high-quality realistic image style transfer method based on composite filtering in a preferred embodiment of this example includes the following steps:

[0093] S1. Perform semantic segmentation on the input image and the reference image.

[0094] The semantic segmentation method of the image is as follows: The DeepLab semantic segmentation algorithm is used to perform semantic segmentation on the image and the reference style image. There are 150 basic classifications in total for semantics, and a merging and simplification method is used to streamline the categories. For example, lakes, rivers, oceans, and water currents are classified into one category, etc. Thus, a set of streamlined classifications are generated to produce a clearer and simpler segmentation, thereby generating a more stable output.

[0095] S2. Decouple the input image and the reference image using the ILS filter to obtain the ILS structure layer and the ILS texture layer of the input image and the reference image.

[0096] In step S2, a weighted least squares filter is adopted. The weighted least squares filter is an edge-preserving filter, and its goal is to make the filtering result as close as possible to the original image, while being as smooth as possible in areas with small gradients and keeping the edge parts with strong gradients as much as possible. Denote the original image as f, the filtering result to be solved as u, a x 、a y are the weight matrices of the gradients in the x and y directions respectively, then the loss function can be expressed as:

[0097]

[0098] where f and u are the vector representations of the input image and the image filtering result; s is the pixel position.

[0099] According to formula (1.4), we have:

[0100]

[0101] where l is the luminance information of the image f. The value of α′ is between 1.2 and 2.0. ε′ is a very small constant value. In fact, we can use other matrices to replace the parameter l.

[0102] When using L(f) to represent the matrix closely related to the source image f, which has the same dimension as f, the solution of formula (1.4) is

[0103] (I + λL(f))u = f Formula (1.6)

[0104] where u is the result of image smoothing filtering. The default value of L(f) is log(f). I is the identity matrix. λ is used to balance the data term and the smoothing term.

[0105] The adaptive filter is as follows: When f is , let be (the illumination information of C) and the sum of the logarithms of

[0106]

[0107] where ΔL = |mean(L(C)) - mean(L(R))|. mean(·) refers to the average value of the variable. τ is the threshold for controlling the use of log(L(C)) and log(C). β is a trigger with a value of 0 or 1. Among them, when ΔL > τ, β = 1; when ΔL < τ, β = 0; let τ = 0.5, when f is R, calculate L(R) in the same way as calculating

[0108] The energy equation of the ILS filter is as follows: ​

[0109]

[0110] Among them, f and u are the vector representations of the input image and the image filtering result. s is the pixel position. Denote the gradients of u in the x and y directions. The smoothing factor p satisfies 0 < p < 1. ε = 0.0001.

[0111] The solution of formula (1.8) is:

[0112]

[0113] Among them, and represent the FFT and IFFT operations respectively. Denote as the complex conjugate of,

[0114] Therefore, in the WLS filter, the factors controlling the image smoothing degree are:

[0115]

[0116] Among them, f is the input image; l is the luminance information of the image f; s is the pixel position; ε′ is a very small constant value; α′ is the smoothing weight;

[0117] In the ILS filter, the factors controlling the image smoothing degree are:

[0118]

[0119] Among them, p is the smoothing factor;

[0120] Considering that the filtering guided by L can only obtain the information of luminance change, and L is only a part of the image (because the image is composed of L, A, and B), therefore, the image filtering result guided by L is not as close to the input image as the image filtering result guided by itself. That is to say, the ILS result retains more information about the input image than the WLS result. In addition, when the WLS filter is guided by the image itself and ε and ε′ are ignored, we obtain:

[0121]

[0122] From this, we get φ p (x) > a x,s , which shows that compared with a in the WLS filter, φ p (x) in the ILS has a stronger edge-preserving smoothing effect.

[0123] Therefore, in step S2, the degree of image smoothing control in the WLS filter is compared with that in the ILS filter, and by judging the edge-preserving smoothing capabilities of the ILS filter and the WLS filter, the ILS filter is used to decouple the input image and the reference image.

[0124] The degree of image smoothing control in the WLS filter is compared with that in the ILS filter, and by judging the edge-preserving smoothing capabilities of the ILS filter and the WLS filter, the ILS filter is used to decouple the input image and the reference image to obtain an ILS structure layer with a stronger edge-preserving effect and an ILS texture layer with a better effect of removing the color of the input image.

[0125] S3. Perform R, G, B channel-by-channel color transfer of the ILS structure layer of the reference image in step S2 to the structure layer of the input image to obtain a new ILS structure layer, and combine the new ILS structure layer with the ILS texture layer of the input image to obtain a new image.

[0126] In step S3, by taking as the input image and R as the reference image, taking and R and its semantic segmentation input encoder-decoder, performing color transfer based on the principle of WCT, and extracting the texture features of are as follows:

[0127]

[0128] where is a diagonal matrix with the covariance matrix eigenvalues; is the vectorized VGG feature of; Ch is the number of channels; is the orthogonal matrix corresponding to the eigenvector, then there is:

[0129]

[0130] Transfer the color of to to get:

[0131]

[0132] where is the color transfer result; is a diagonal matrix with the covariance matrix eigenvalues; is the orthogonal matrix corresponding to the eigenvector, satisfying:

[0133]

[0134] Among them, m is the average value; the input The decoder obtains the color transfer result of the image in the RGB channels;

[0135] In formulas (3.1) - (3.4), replace with the structural layer obtained by filtering the j channel of by the ILS filter Replace with the structural layer obtained by filtering the j channel of j ′, where j ∈ {R, G, B}; Combine the new ILS structural layer with the texture layer of the input image to obtain the new image W.

[0136] S4. Use the TE module to repair the boundary features of the ILS structural layer of the new image in step S3 to obtain the ILS structural layer of the final image. The TE module mainly includes the AWLS filter.

[0137] As Figure 4 shown, step S4 designs a feature repair module to improve the unnatural boundary problem that is prone to occur in the original network structure.

[0138] In step S4, the boundary feature repair of the structural layer of the new image W is as follows:

[0139]

[0140] Among them, W′ is the feature repair result; represents the WLS filter operation, represents the structural layer obtained by filtering W with the WLS filter, represents the texture layer obtained by filtering with the WLS filter;

[0141]

[0142] Among them, k1 and k1′ are respectively and the weights of;

[0143] According to formula (4.2), for the new image W in step S3, the AWLS texture layer of W obtained by using the AWLS filter is:

[0144]

[0145] According to formula (4.2), the ILS texture layer without the color of the original image obtained through the ILS filter is: There is:

[0146]

[0147] Among them, represents the ILS filter operation, and

[0148] and respectively represent the structure layer and the texture layer obtained by performing ILS filter on ; k2 and k2' are the weights of and respectively, and are set to k2 = k2' = 1; according to formula (4.3) and formula (4.4), we obtain:

[0149]

[0150] Among them, and respectively represent the structure layer and the texture layer obtained by performing ILS filter on W;

[0151] According to formula (4.1), when repairing the j-th channel of S', S j 's repair result is:

[0152]

[0153] Among them, S j is the repair result of the ILS structure layer of W output by the TE module on the j-th channel of WCT 2 ; is the structure layer obtained by performing AWLS filter on the structure layer obtained by performing ILS filter on W j ; represents the structure layer obtained by performing ILS filter on the j-th channel of ; represents the structure layer obtained by performing AWLS filter on .

[0154] S5. Estimate the loss caused by the boundary feature repair in step S4, and convert the loss caused in the structure layer of the final image into the texture feature loss of the texture layer of the final image.

[0155] In step S5, the calculation method of loss estimation is:

[0156]

[0157] Converting the loss caused in the structural layer of the final image into the texture feature loss of the final image includes:

[0158]

[0159] Among them, is to extract the structural layer of W using the WLS filter and obtain it according to formula (5.1):

[0160]

[0161] According to formula (1.3), when ignoring ε and ε′, the result of filtering the image by the WLS filter is approximately equal to the result of filtering the image by the ILS filter and then by the WLS filter, and is obtained according to formula (5.3):

[0162]

[0163] Substituting formula (5.4) into formula (5.1) gives:

[0164]

[0165] Among them, δ represents the feature loss of the structural layer of the final image.

[0166] In step S6, perform texture feature loss compensation on the texture of the final image in step S5:

[0167] W j ′ extracts the WLS filter features and is obtained through formula (5.1):

[0168]

[0169] Approximately representing δ according to formula (5.4) and formula (6.1), we get:

[0170]

[0171] Among them, δ' is the compensated texture feature loss, and K1 and K”1 are weight values respectively. The difference between δ and δ' can be understood as the texture feature difference between the problem boundary and the natural boundary.

[0172] S6. Perform texture feature loss compensation on the texture layer of the final image in step S5 to obtain the ILS texture layer of the final image.

[0173] In step S6, after performing texture feature loss compensation on the texture layer of the final image, the ILS texture layer of the final image is obtained;

[0174] is in formula (6.2) In combination with that in Step 2 decoupled by the ILS filter to obtain ; let K1 be less than or equal to 0.4 and K1' be greater than or equal to 0.6.

[0175] According to formula (5.5), representing the structural layer of the WCT 2 in the j-channel component, and combining with formula (5.3) to obtain the approximate feature loss caused by feature repair in Step S4. The structural layer of the feature repair image is extracted by the WLS filter for feature loss compensation to obtain The texture layer of the input image is extracted by the ILS filter to obtain Combine with to obtain the final ILS texture layer F j .

[0176] Figure 6 For increasing the comparison of feature preservation and style transfer before and after ECSL.

[0177] The image texture layer extracted by WLS has more information on image brightness changes, while the image texture layer extracted by ILS basically has no brightness information. Therefore, we can obtain the image style transfer results with different brightness by adjusting the weights of K1 and K1' in the above formula, as Figure 7 shown.

[0178] S7. Fuse the ILS structural layer of the final image in Step S4 with the final ILS texture layer in Step S6 to obtain an image with high-quality realistic image style transfer.

[0179] In Step S7, the calculation method for fusing the structural layer of the feature repair image with the final texture layer is:

[0180] O j = γ1S j + γ2F j Formula (7.1)

[0181] where γ1 and γ2 are the weights of S j and F j .

[0182] Obtain according to formulas (7.1)(6.3)(4.2)

[0183]

[0184] Among them, the sum of the weights of the two structural layers is 0, and the weights of other terms are 1. Then we have: γ1 - γ2K1k1' = 0, γ2K1k1 + γ2K1' = 0, γ1 = k1', γ2 = K1 = 1, k1 = 0.8, K1' = 0.2; Different degrees of color transfer results are obtained by adjusting the values of γ1 and -k1'. As Figure 8 shown.

[0185] This embodiment also includes a realistic image style transfer system based on compound filtering, which includes a processor and a storage device. One or more processors are provided, and the storage device is used for storing one or more programs to execute the method of steps S1 - S7.

Claims

1. A high-quality realistic image style transfer method based on composite filtering, characterized in that, Including the following steps: S1. Perform semantic segmentation on the input image and the reference image; S2. Decouple the input image and the reference image using an ILS filter to obtain the ILS structure layer and the ILS texture layer of the input image and the reference image; In step S2, The parameters in the ILS filter that control the image smoothness are: where p is the smoothness factor; ε is a constant value; x is the abscissa of the x-axis; The energy equation of the ILS filter is as follows: where f and u are vector representations of the input image and the result of image filtering, respectively; s is the pixel position; denote the gradients of u in the x and y directions, respectively; the smoothing factor p satisfies 0 < p < 1; ε = 0.0001; λ is used to balance the data term and the smoothing term; S3. Perform R, G, B channel-by-channel color transfer on the ILS structure layer of the reference image in step S2 to the structure layer of the input image to obtain a new ILS structure layer, and combine the new ILS structure layer with the ILS texture layer of the input image to obtain a new image; S4. Use a TE module to repair the boundary features of the ILS structure layer of the new image in step S3 to obtain the ILS structure layer of the final image; S5. Estimate the loss caused by the boundary feature repair in step S4, and convert the loss caused in the structure layer of the final image into the texture feature loss of the texture layer of the final image; S6. Compensate the texture feature loss of the texture layer of the final image in step S5 to obtain the ILS texture layer of the final image; S7. Fuse the ILS structure layer of the final image in step S4 with the final ILS texture layer in step S6 to obtain an image with high-quality realistic image style transfer.

2. A high-quality realistic image style transfer method based on compound filtering according to claim 1, characterized in that: In step S3, by taking as the input image and R as the reference image, and its semantic segmentation input encoder-decoder for R, color transfer is performed based on the principle of WCT, and the texture features are extracted as follows: Among them is a diagonal matrix with the covariance matrix eigenvalues; is the vectorized VGG feature of Ch is the number of channels; is the orthogonal matrix corresponding to the eigenvector, then there is: Transfer 's color to There are: Among them is the color transfer result; is a diagonal matrix with covariance matrix eigenvalues; is the orthogonal matrix corresponding to the eigenvectors, satisfying: where m is the average value; the input decoder obtains the color transfer result of the image in the RGB channel; In formulas (3.1) - (3.4), replace with The structural layer obtained by filtering the j-channel of Replace with The structural layer obtained by filtering the j-channel of To obtain the color transfer result structural layer S' of the RGB channels j , where j ∈ {R, G, B}; Combine the new ILS structural layer with the texture layer of the input image to obtain a new image W.

3. A high-quality realistic image style transfer method based on composite filtering according to claim 2, characterized in that: In step S4, the boundary feature repair of the structure layer of the new image W is as follows: Among them, W′ is the result of feature repair; Represents the WLS filter filtering operation, Represents the structural layer obtained by filtering W with the WLS filter; Represents The texture layer obtained by filtering with the WLS filter; The texture layer of AWLS of where k1 and k′1 are the weights of and respectively; According to formula (4.2), for the new image W in step S3, use an AWLS filter to obtain the AWLS texture layer of W as: According to formula (4.2), the ILS texture layer without the color of the original image obtained by the ILS filter is: ​ Among them, represents the ILS filter filtering operation, and respectively represent the structural layer and the texture layer obtained by performing ILS filter filtering on ; k2 and k2' are the weights of and respectively, and are set to k2 = k'2 = 1; obtained according to formula (4.3) and formula (4.4): Among them, and respectively represent the structure layer and the texture layer obtained by filtering W with an ILS filter; When repairing the j-th channel of S' according to formula (4.1), the repaired result of S' can be obtained j is as follows: Among them, S j is the repair result of the ILS structure layer of WCT 2 output W in the j channel; is the structure layer obtained by filtering the structure layer obtained by filtering W j with an ILS filter and then filtering with an AWLS filter; represents the structure layer obtained by filtering the j channel of with an ILS filter; represents the structure layer obtained by filtering with an AWLS filter.

4. A high-quality realistic image style transfer method based on composite filtering according to claim 1, characterized in that: In step S5, the calculation method of loss estimation is: The conversion of the loss caused in the structure layer of the final image into the texture feature loss of the final image is as follows: Among them, is the structural layer that extracts W using the WLS filter and is obtained according to formula (5.1): When ignoring ε and ε′, the result of filtering the image by the WLS filter is approximately equal to the result of filtering the image by the ILS filter and then by the WLS filter, and is obtained according to formula (5.3): Substitute formula (5.4) into formula (5.1) to get: where δ represents the feature loss of the structure layer of the final image.

5. A high-quality realistic image style transfer method based on composite filtering according to claim 4, characterized in that: In step S6, perform texture feature loss compensation on the texture of the final image in step S5: W j The features of the WLS filter are extracted and obtained by Equation (5.1): Approximately represent δ according to formula (5.4) and formula (6.1) to obtain: where δ' is the compensated texture feature loss, and K1 and K”1 are weights respectively; the difference between δ and δ' can be understood as the texture feature difference between the problem boundary and the natural boundary.

6. A high-quality realistic image style transfer method based on composite filtering according to claim 5, characterized in that: In step S6, after the texture layer of the final image is compensated for texture feature loss, the ILS texture layer of the final image is obtained; in formula (6.2) in step 2 decoupled by the ILS filter Combination; let K1 be less than or equal to 0.4 and K1' be greater than or equal to 0.

6.

7. A high-quality realistic image style transfer method based on compound filtering according to claim 6, characterized in that: In step S7, the calculation method for fusing the ILS structure layer of the final image with the ILS texture layer of the final image is: O j = γ1S j + γ2F j Equation (7.1) Among them, where γ1 and γ2 are the weights of S j and F j respectively.

8. A high-quality realistic image style transfer method based on composite filtering according to claim 7, characterized in that: Obtained according to formula (7.1)(6.3)(4.2) Among them, the sum of the weights of the two structural layers is 0, and the weights of other items are 1. Then we have: γ1 - γ2K1k1' = 0, γ2K1k1 + γ2K1' = 0, γ1 = k1', γ2 = K1 = 1, k1 = 0.8, K1' = 0.2; Different degrees of color transfer results are obtained by adjusting the values of γ1 and -k1'.

9. A realistic image style transfer system based on composite filtering, characterized in that: It includes a processor and a storage device. One or more processors are provided, and the storage device is used for storing one or more programs to execute the high-quality realistic image style transfer method based on composite filtering described in claim 1.

Images