A realistic image style transfer system

Abstract

The present application relates to the technical field of computer image processing, and particularly relates to a realistic image style transfer system, a style transfer module, input for a content image, a reference image and corresponding semantic segmentation, the style transfer module transfers the style of the reference image to the content image through an encoder and a decoder, and performs repairing in the transfer process to obtain a new image, a feature extraction module, used for obtaining the content image, the reference image and corresponding semantic segmentation to extract boundary features and image features, a repairing module, used for repairing through an AWLS filter and an ILS filter to obtain a transfer result, and an EDE quantitative evaluation module, used for evaluating the transfer result, the present application can completely remove the original color of the content image while retaining the clear features of the content image, can produce more significant image style transfer effects, and the EDE quantitative evaluation index can quantitatively evaluate the significance of the realistic image style transfer using semantic segmentation.

Description

Technical Field

[0001] This invention relates to the field of computer image processing technology, and in particular to a realistic image style transfer system. Background Technology

[0002] Realistic image style transfer is a fundamental task in image processing, requiring effective editing of image color and texture, and is widely used in digital art creation, film and television special effects, virtual reality environments, etc. However, the color and texture of an image are often mixed together. This mixing leads to blurry artifacts between texture preservation and style transfer. To achieve impressive realistic image style transfer effects, it is necessary to use a reference image ( The reference style of (C) completely replaces the original style of the content image (C), while preserving as much as possible. Important textures.

[0003] A typical approach uses the Gram matrix and feature matrix to effectively distinguish style and texture, respectively, and produces art style transfer results. However, the loss function includes... Content and The style of the image is often obscured. Therefore, this method leads to entanglement between style and texture, failing to produce good image style transfer results. Subsequent methods, such as Luan F, Paris S, Shechtman E, and Bala K, "Deep Photo Style Transfer," published in the *IEEE Conference on Computer Vision and Pattern Recognition Proceedings*, and Li JY and Chen Fang, "Diverse Texture Synthesis Using Feedforward Networks," published in the *IEEE Conference on Computer Vision and Pattern Recognition Proceedings*, improved upon the method proposed by Gatiss LA, Eck AS, and Besger M, "Image Style Transfer Using Convolutional Neural Networks." However, their effectiveness in feature preservation remains limited.

[0004] Another important approach is to use whitening and color transformation (WCT) to transfer the reference style. WCT can perform image style transfer and has the advantage of fast computation speed. However, WCT fails to effectively preserve the texture of the content image. Many techniques have been developed to improve WCT to generate artistic or realistic images. Traditional methods and their subsequent improvements often affect the preservation of texture details during color transfer. To address the challenge of color and texture entanglement, Ding et al.

[18] proposed an adaptive weighted least squares (AWLS) filter and channel separation (AFCS) framework to separate the color and texture. and The structure layer and texture layer. However, the image features extracted by the AWLS filter also retain some color information, which leads to incomplete image style transfer. Ding et al. [7] used a post-processing module, including AWLS filter and ILS filter, to obtain effective style transfer results. In this method, The network parameters cannot be further optimized. Unlike methods based on Gram matrices or covariance matrices, the diffusion model does not require matching. and They have feature statistics, but they require detailed text input and generate style images directly through end-to-end learning. Summary of the Invention

[0005] To address the aforementioned problems, this invention provides a realistic image style transfer system that can completely remove the original colors of a content image while preserving its clear features, thereby producing a more significant image style transfer effect.

[0006] To achieve the above objectives, the technical solution adopted by the present invention is as follows:

[0007] A realistic image style transfer system, comprising:

[0008] The style transfer module is used as input for the content image, the reference image, and the corresponding semantic segmentation. The style transfer module transfers the style of the reference image to the content image through an encoder and a decoder, and performs repairs during the transfer process to obtain a new image.

[0009] The feature extraction module is used to acquire the content image, the reference image, and the corresponding semantic segmentation, so as to extract boundary features for image boundary repair of the new image and image features for image feature recovery of the new image;

[0010] The repair module is used for data acquisition by the style transfer module and the feature extraction module. The repair module repairs the texture features of the new image by using AWLS and ILS filters and based on the boundary features and the image features to obtain the transfer result.

[0011] The EDE quantization evaluation module is used to compare the color style of the migration result with the color style of the reference image, so as to optimize the weights of the ILS filter feature layer based on the comparison result.

[0012] Furthermore, the encoder is a VGG-19 network pre-trained on ImageNet, with Haar wavelet pooling incorporated from the conv1_1 layer to the conv4_1 layer to form the WCT. 2 WCT of coding architecture 2 The encoder, the decoder mirrors the WCT2 Encoder, and is WCT 2 The decoder, the style transfer module in the WCT 2 After the WCT output of the last block of the decoder, it is repaired by the repair module.

[0013] Furthermore, the WCT 2 The last block of the decoder corresponds to the LL layer of wavelet decomposition. The style transfer module supplements the boundaries of the subgraph style transfer results of the LL layer through the repair module.

[0014] Furthermore, in the boundary feature extraction, the feature extraction module will extract the content image. As , also as The semantic segmentation, and its semantic segmentation, are respectively input into the WCT. 2 Encoder, and in the WCT 2 The last block of the decoder extracts the corresponding boundary features. ;

[0015] The feature extraction module, in the image feature extraction process, will... , and the corresponding semantic segmentation input to the WCT 2 The encoder performs style transfer to obtain Through WCT 2 AWLS boundary repair module of the decoder ,right AWLS-based boundary repair is performed, and by using... AWLS texture layer replacement AWLS texture layer to repair To obtain :

[0016]

[0017] Formula (2.1)

[0018] Formula (2.2)

[0019] Among them, among them, The structural layers obtained for the AWLS filter; The texture layer obtained by the AWLS filter; the parameters in parentheses represent the image before smoothing.

[0020] Will As and Input to In the module, and in the WCT 2 The last decoder layer extracts the corresponding image features. .

[0021] Furthermore, the repair module for the The repair method is the same as the image feature extraction method:

[0022] Formula (3.1)

[0023] in, The result of boundary feature repair; for The result of style transfer;

[0024] The Enhanced style transfer is performed using the image features and the AWLS filter to obtain... The Through ILS filter and Enhance texture features to obtain ;

[0025] Regarding the Decode the data to obtain the migration results. .

[0026] Furthermore, the repair module for the The repair method is as follows: [The rest of the text appears to be incomplete and requires further context.] The input is fed into the AWLS filter to obtain the structural layer. ; Regarding the minus To obtain a boundary-free problem .

[0027] Furthermore, the repair module applies an AWLS filter to the image features. Enhanced style transfer:

[0028] Formula (3.2)

[0029] in, for Enhanced results; for AWLS feature layer, yes AWLS structure layers; This is for AWLS feature layer; for The weight value; for The weight value; for The weight value;

[0030] The repair module passes through ILS texture features to enhance Furthermore, the texture features of the AWLS repair result are supplemented by an ILS texture layer:

[0031] Formula (3.3)

[0032] in, Supplementing results for ILS; The texture layer obtained by ILS smoothing; for The weight value.

[0033] Furthermore, the repair module performs boundary repair and feature enhancement based on the AWLS filter and the ILS filter, and its loss function includes... and :

[0034] Formula (3.4)

[0035] Formula (3.5)

[0036] in, , These are the weights of the loss function; This is for and The variance; for and The variance; for Features of wavelet coding at layer 4; To reconstruct the image Features of wavelet coding layer 4.

[0037] Furthermore, the EDE quantification evaluation module includes a color annotation submodule and an EDE difference calculation submodule, and uses the migration result as the detection style image and the reference image as the reference style image.

[0038] The color annotation submodule is used for annotation of the reference style image and the detection style image. The color annotation submodule has a `color_dict` set to define semantic content. The color annotation submodule performs pixel-by-pixel detection on the colors of the semantic segmentation maps of the reference style image and the detection style image. When a pixel's color belongs to the `color_dict` set, the corresponding color is marked on the semantic segmentation maps of the reference style image and the detection style image. When a pixel's color does not belong to the `color_dict` set, the closest color in the `color_dict` set is calculated using Manhattan distance, and the corresponding closest color is marked on the semantic segmentation maps of the reference style image and the detection style image.

[0039] The EDE difference calculation submodule is used for proximity analysis between the reference style image and the detected style image. After standardizing the format of the reference style image and the detected style image, the EDE difference calculation submodule decomposes all semantic regions of both images using singular value decomposition based on the labeled colors, thereby obtaining the reference style image. and the detected style image ; It is a diagonal matrix of the reference style image and the detected style image, and each contains singular values; and These are the right singular vector transposes of the reference style image and the detected style image, respectively.

[0040] The EDE difference calculation submodule for and The reference style image is obtained by reconstructing and weighting. Matrix and the detected style image Matrix, calculate the Matrix and the stated The root mean square of the differences between the matrices is used to obtain the degree of similarity between the reference style image and the detected style image, and a comparison result is obtained based on the degree of similarity.

[0041] Furthermore, in the EDE difference calculation submodule, the style covariance matrix is ​​decomposed into singular values:

[0042] Formula (4.1)

[0043] in, The style covariance matrix; It is a left singular matrix; It is a right singular matrix. The diagonal elements are the singular value diagonal matrix; the reference style image and the detected style image are input to obtain the style covariance matrix: ,

[0044] The effective rank of the diagonal elements of the singular value diagonal matrix is:

[0045] Formula (4.2)

[0046] in, For effective rank; Singular values ​​representing style characteristics; The energy conservation threshold; It is a matrix The number of rows represents the dimension of the feature vector;

[0047] Take before The square root of each singular value, multiplied by the weight. To obtain weighted scaling :

[0048] Formula (4.3)

[0049] Will Set to 1 to obtain the principal component directions:

[0050] Formula (4.4)

[0051] Will Multiply Then multiply by The transpose of the matrix is ​​used to reconstruct the symmetric matrix. :

[0052] Formula (4.5)

[0053] in, For right singular matrices List. for A diagonal matrix of singular values; These are weights, used to control the intensity of style transfer;

[0054] EDE achieves low-rank embedding in the feature space by truncating singular values, and... Set as a low-rank approximation with the style covariance matrix:

[0055] Formula (4.6)

[0056] The semantic segmentation map of the reference style image, the semantic segmentation map of the detected style image, and the semantic segmentation map of the reference style image are calculated according to formula (4.6). Matrix, the detected style image The low-rank approximation of the covariance matrix of the matrix, and the style difference between the reference style image and the detected style image is:

[0057] Formula (4.8)

[0058] The difference in style between the semantic segmentation images of the reference style image and the detected style image is as follows:

[0059] Formula (4.9)

[0060] in, This represents the number of semantically segmented regions within the image.

[0061] The beneficial effects of this invention are:

[0062] 1. The style transfer module, in order to repair the reconstruction offset of WCT, only repairs the last layer of the decoder, which corresponds to the LL layer of wavelet decomposition and contains most of the feature information of the style transfer result. The other three sub-layers contain fewer image features from the input image, resulting in insufficient boundary repair and consuming significant time and computation. Therefore, this application does not repair boundaries at every layer of the decoder, but only repairs boundaries and supplements texture features at the last layer. Two features are extracted from the feature space using the feature extraction module. Then, in the repair module, these two features are used to repair the features of the last layer of the style transfer decoder. The adaptive weighted least squares filter and iterative least squares filter repair modules repair the unnatural boundaries and feature loss of the content image during style transfer. This invention, through... Adaptive weighted least squares (AWLS) filters and iterative least squares (ILS) filters can achieve effective style transfer while reducing the unique features of the content. Even when trained with a simple loss function, they can obtain more significant style transfer results and produce more significant image style transfer effects.

[0063] 2. The EDE quantification evaluation module of this invention is based on the evaluation index of the diagonal elements of the right singular matrix and the singular value diagonal matrix. On the basis of semantic segmentation, the EDE index can obtain quantitative evaluation results that are close to the visual effect. The EDE evaluation metric uses the covariance matrix to calculate the image style, thereby finding the style difference between the output image and the reference image. The EDE quantification evaluation module calculates the difference between the style transfer generated image and the reference style image by constructing the covariance matrix between them, thereby quantitatively measuring the significance of style transfer. Attached Figure Description

[0064] Figure 1 This is a block diagram of a preferred embodiment of the present invention.

[0065] Figure 2 This is a preferred embodiment of the image style transfer framework diagram of the present invention.

[0066] Figure 3 This is a structural diagram of an encoder-decoder according to a preferred embodiment of the present invention.

[0067] Figure 4 This is a preferred embodiment of the present invention: a single-scale image decomposition diagram based on Haar wavelets.

[0068] Figure 5 This is a fix for a preferred embodiment of the present invention. The process diagram.

[0069] Figure 6 This is a comparison chart of style transfer results under different c values ​​according to a preferred embodiment of the present invention.

[0070] Figure 7 This is a loss convergence diagram of a preferred embodiment of the present invention.

[0071] Figure 8 This is a process diagram of EDE according to a preferred embodiment of the present invention.

[0072] Figure 9 This is a comparison diagram of a preferred embodiment of the present invention and a realistic method with semantic segmentation.

[0073] Figure 10 This is a comparison diagram of a preferred embodiment of the present invention and a realistic method without semantic segmentation.

[0074] Figure 11 This is a comparison diagram of a preferred embodiment of the present invention and an artistic method.

[0075] Figure 12 This is a comparison diagram of a preferred embodiment of the present invention with 12 non-segmented image style transfer methods.

[0076] Figure 13 This is a schematic diagram of user voting rates for the effectiveness of EDE evaluation in a preferred embodiment of the present invention.

[0077] Figure 14 This is a preferred embodiment of the invention. A schematic diagram of an ablation experiment for boundary repair.

[0078] Figure 15 This is a preferred embodiment of the present invention. Schematic diagram of feature-enhanced ablation experiment.

[0079] Figure 16 This is a preferred embodiment of the present invention. and A schematic diagram of the ablation experiment.

[0080] Figure 17 This is an image style transfer result diagram of a preferred embodiment of the present invention.

[0081] In the diagram, 1-style transfer module, 2-feature extraction module, 3-repair module, 4-DE evaluation module, 41-color annotation sub-module, and 42-EDE difference calculation sub-module. Detailed Implementation

[0082] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0083] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention pertains. The terminology used herein in the description of the invention is for the purpose of describing particular embodiments only and is not intended to be limiting of the invention. The term "and / or" as used herein includes any and all combinations of one or more of the associated listed items.

[0084] Please also see Figures 1 to 17 A preferred embodiment of the present invention provides a realistic image style transfer system, comprising: a style transfer module 1, a feature extraction module 2, a repair module 3, and an EDE quantization evaluation module 4.

[0085] Style transfer module 1 is used as input for content image, reference image and corresponding semantic segmentation. Style transfer module 1 transfers the style of reference image to content image through encoder and decoder, and repairs it during the transfer process to obtain a new image.

[0086] The encoder is a VGG-19 network pre-trained on ImageNet, with Haar wavelet pooling incorporated from conv1_1 to conv4_1 layers to form the WCT. 2 WCT of Coding Architecture 2 Encoder, decoder mirror WCT 2 Encoder, and is WCT 2 Decoder, Style Transfer Module 1 in WCT 2 After the WCT output of the last block of the decoder, it is repaired by repair module 3.

[0087] WCT 2 The last block of the decoder corresponds to the LL layer of wavelet decomposition. Style transfer module 1 supplements the boundary of the subgraph style transfer result of the LL layer through repair module 3.

[0088] In this embodiment, to correct the reconstruction offset caused by the decoder and supplement texture features, we added a repair module 3 to the decoder. Our style transfer model mainly consists of an encoder and a decoder module. We use WCT. 2 The encoder architecture in the code, the decoder mirror encoder structure, such as Figure 3 As shown, AWLS and ILS are used to help the module solve boundary problems and supplement features in the final decoder.

[0089] To correct the reconstruction offset of WCT, only the last layer of the decoder is repaired, which corresponds to the LL layer of wavelet decomposition and contains most of the feature information of the style transfer result. When the image is decomposed by Haar wavelet, four submaps are obtained: low-frequency submap (LL), horizontal high-frequency submap (LH), vertical high-frequency submap (HL), and diagonal high-frequency submap (HH). The result of single-scale Haar wavelet decomposition of the image is as follows: Figure 4 As shown, Figure 4 In the diagram, (a) represents the input image. (b) - (e) represent the LL, LH, HL, and HH sub-bands, respectively. It can be seen that most of the information in the image is contained within the LL sub-image. Therefore, the boundary of the style transfer result obtained through Haar wavelet decomposition is repaired, and its features are supplemented. When the Haar wavelet transform is embedded in the encoder and decoder networks, the LL sub-band (low-frequency approximation component) corresponds to the first layer (lowest layer) of the encoder and the fourth layer (highest layer) of the decoder. The other three sub-layers contain fewer image features from the input image, resulting in insufficient boundary repair. The LL sub-band is responsible for preserving the main structure and overall style features of the image. The other three sub-layers contain fewer image features from the input image, resulting in insufficient boundary repair and consuming significant time and computation. Therefore, this embodiment does not repair the boundary at every layer of the decoder, but only repairs the boundary and supplements texture features at the last layer.

[0090] Feature extraction module 2 is used to acquire content image, reference image and corresponding semantic segmentation, so as to extract boundary features for image boundary repair of new image and image features for image feature recovery of new image.

[0091] Feature extraction module 2 extracts the content image during boundary feature extraction. As , also as The semantic segmentation and its corresponding boundary features are input into the WCT2 encoder, and the corresponding boundary features are extracted in the last block of the WCT2 decoder. ; For reference image; by using content image As , also as The feature extraction method makes There are no boundary issues.

[0092] Feature extraction module 2, in image feature extraction, will... , The corresponding semantic segmentation input is fed into the WCT2 encoder for style transfer to obtain... AWLS boundary repair module 3 via WCT2 decoder ,right AWLS-based boundary repair is performed, and by using... AWLS texture layer replacement AWLS texture layer to repair To obtain :

[0093]

[0094] Formula (2.1)

[0095] Formula (2.2)

[0096] Among them, among them, The structural layers obtained for the AWLS filter; The texture layer obtained by the AWLS filter; the parameters in parentheses represent the image before smoothing.

[0097] In this embodiment, It is an AWLS boundary repair module 3 of the WCT2 encoder. The encoder is The encoder in the text. The decoder in Each decoder layer's convolutional layer is followed by AWLS boundary repair, therefore we will Its semantic segmentation graph is input to Style transfer is performed in the module to obtain And perform AWLS-based boundary repair to obtain .

[0098] Will As and Input to In the module, and in the last decoder layer of the WCT2 decoder, the corresponding image features are extracted. .from The obtained AWLS texture features are compared to It contains far fewer original styles.

[0099] Repair module 3 is used for data acquisition of style transfer module 1 and feature extraction module 2. Repair module 3 repairs the texture features of the new image through AWLS filter and ILS filter, and based on boundary features and image features, to obtain the transfer result.

[0100] Repair module 3 pairs The repair method is the same as the image feature extraction method:

[0101] Formula (3.1)

[0102] in, The result of boundary feature repair; for The result of style transfer.

[0103] like Figure 5 As shown, using Repair style transfer results A simple process. Figure 5 It also describes Figure 1 middle The generation process. and The fusion result right The repair results.

[0104] Repair module 3 pairs The repair method is as follows: The input is fed into an AWLS filter to obtain the structural layers. ;right minus To obtain a boundary-free problem In the WCT2 model, the image itself migrates without creating boundary problems, so this relationship is used to extract features without boundary problems.

[0105] Enhanced style transfer is achieved by using image features and AWLS filters to obtain... ; Through ILS filter and Enhance texture features to obtain ;

[0106] right Decode the data to obtain the migration results. .

[0107] Repair module 3 uses an AWLS filter to process image features. Enhanced style transfer:

[0108] Formula (3.2)

[0109] in, for Enhanced results; for AWLS feature layer, yes AWLS structure layers; This is for AWLS feature layer; for The weight value; for The weight value; for The weight value; in this embodiment, . The greater the weight, the clearer the style transfer result, but the more obvious the original style becomes. This can be achieved by modifying... Feature weights can not only remove The original style can be preserved, and the brightness of the final style transfer result can be adjusted.

[0110] based on Feature enhancement: We found that during image style transfer training based on formula (3.1), the generated images are dark and the texture is obviously lost. This problem cannot be solved by simply adjusting the weight of the feature preservation term in the loss function. Include and The texture and brightness information, therefore, this embodiment uses Enhance style transfer results using Formula 4 . The enhancement result is .

[0111] Repair module 3 passed ILS texture features to enhance Furthermore, the texture features of the AWLS repair result are supplemented by an ILS texture layer:

[0112] Formula (3.3)

[0113] in, Supplementing results for ILS; The texture layer obtained by ILS smoothing; for The weight value; in this embodiment, .

[0114] Based on AWLS boundary restoration can restore the original style of the input image. To further supplement features, this embodiment uses an ILS filter to extract... The feature layer. The ILS feature layer has almost no The original style, therefore, we use ILS texture features to enhance .

[0115] like Figure 6 As shown, the effect of different values ​​of 'c' on the output is illustrated. Experimental results show that the style of the network output is closer to the reference image after adding appropriate ILS features than when there are no ILS features, such as... Figure 6 As shown in (b), (c), and (d), when the weights of the ILS features are too large, the final output will exhibit significant noise, such as... Figure 6 As shown in (g). Figure 6 In the middle, (a) is (top), (Bottom) and its semantic segmentation map (bottom right). (b) - (g) are the image style transfer results under different c values. Table 1 compares the EDE index under different c values.

[0116]

[0117] Table 1 Comparison of EDE indices under different c values

[0118] Repair module 3 performs boundary repair and feature enhancement based on AWLS and ILS filters, and its loss function includes... and :

[0119] Formula (3.4)

[0120] Formula (3.5)

[0121] in, , These are the weights of the loss function; This is for and The variance (reconstruction loss); for and The variance (feature loss); for Features of wavelet coding at layer 4; To reconstruct the image Features of the fourth layer of wavelet coding. For example... Figure 7 As shown, the loss function and All convergences were effective, indicating that the model is effective. Figure 7 In the diagram, (a) is the loss convergence plot, and (b) is the corresponding plot. The reconstruction loss convergence plot, (c) is the corresponding The feature loss convergence plot.

[0122] like Figure 2 As shown, the WAI network in this embodiment is used to achieve realistic image style transfer. Figure 17 The image style transfer result of this embodiment is shown, where (a) includes the content image (top) and the reference image (bottom) and the corresponding semantic segmentation (bottom left corner of the image). (b) shows our image style transfer result.

[0123] EDE quantization evaluation module 4 is used to compare the color style of the transfer result with the color style of the reference image, so as to optimize the weights of the ILS filter feature layer based on the comparison result. EDE quantization evaluation module 4 includes a color annotation submodule 41 and an EDE difference calculation submodule 42, and uses the transfer result as the detection style image and the reference image as the reference style image;

[0124] The color annotation submodule 41 is used for annotation of the reference style image and the detection style image. The color annotation submodule 41 has a color_dict set to define semantic content. The color annotation submodule 41 performs pixel-by-pixel detection on the color of the semantic segmentation map of the reference style image and the detection style image. When the color of a pixel belongs to the color_dict set, the corresponding color is marked on the semantic segmentation map of the reference style image and the detection style image. When the color of a pixel does not belong to the color_dict set, the closest color in the color_dict set is calculated by Manhattan distance, and the corresponding closest color is marked on the semantic segmentation map of the reference style image and the detection style image.

[0125] In the color annotation submodule 41, input a reference style image. and detection style images The semantic segmentation map is then used for pixel-by-pixel detection. and The color of a pixel is determined. If the pixel's color belongs to the `color_dict` set, that color is assigned. Otherwise, the closest color in the `color_dict` set is calculated using Manhattan distance before color labeling.

[0126] The EDE difference calculation submodule 42 is used for proximity analysis between the reference style image and the detected style image. After standardizing the format of the reference style image and the detected style image, the EDE difference calculation submodule 42 decomposes all semantic regions of the reference style image and the detected style image using singular value decomposition according to the labeled colors, to obtain the reference style image. and style detection images ; The diagonal matrix of the reference style image and the detected style image contains singular values. and These are the right singular vector transposes of the reference style image and the detected style image, respectively;

[0127] EDE Difference Calculation Submodule 42 pairs and Reconstruction and weighting are performed to obtain the reference style image. Matrix and style detection images Matrix, calculation Matrix and The root mean square of the differences between the matrices is used to obtain the similarity between the reference style image and the detected style image, and the comparison result is obtained based on the similarity.

[0128] The reference style image in this embodiment is The detected style image is .

[0129] like Figure 8 As shown, this illustrates the EDE process. In the EDE difference calculation submodule 42, the style covariance matrix is ​​decomposed into singular values:

[0130] Formula (4.1)

[0131] in, The style covariance matrix; It is a left singular matrix; It is a right singular matrix. These are the diagonal elements of the singular value diagonal matrix; the reference style image and the detected style image are input to obtain the style covariance matrix: ,

[0132] The left singular vector matrix is ​​an orthogonal basis matrix associated with the row space of the input matrix in singular value decomposition, and its column vectors form the projection basis of the input matrix's row space. In image processing, left singular vectors can be used for feature extraction and noise reduction. The column vectors of the right singular vector matrix form an orthogonal basis for the image feature space, and these basis vectors correspond to the main directions of change in the image data. In style transfer tasks, the column directions of the right singular vector matrix (columns of V) directly encode the distribution pattern of the style image in the feature space. In style transfer tasks, right singular vectors are often used to construct color transformation matrices, and the square roots of their diagonal elements can be directly used as scaling factors.

[0133] The effective rank of the diagonal elements of a singular valued diagonal matrix is:

[0134] Formula (4.2)

[0135] in, For effective rank; Singular values ​​representing style characteristics; The energy conservation threshold; It is a matrix The number of rows represents the dimension of the feature vector;

[0136] Take before The square root of each singular value, multiplied by the weight. To obtain weighted scaling :

[0137] Formula (4.3)

[0138] Will Set to 1 to obtain the principal component directions:

[0139] Formula (4.4)

[0140] Will Multiply Then multiply by The transpose of the matrix is ​​used to reconstruct the symmetric matrix. :

[0141] Formula (4.5)

[0142] in, For right singular matrices List. for A diagonal matrix of singular values; These are weights used to control the intensity of style transfer; singular value decomposition is a generalized form of feature decomposition, while EDE achieves low-rank embedding in the feature space by truncating singular values.

[0143] EDE achieves low-rank embedding in the feature space by truncating singular values, and... Set as a low-rank approximation with the style covariance matrix:

[0144] Formula (4.6)

[0145] The semantic segmentation map of the reference style image, the semantic segmentation map of the detected style image, and the semantic segmentation map of the reference style image are calculated according to formula (4.6). Matrix, style detection image The low-rank approximation of the covariance matrix of the matrix, and the style difference between the reference style image and the detected style image is:

[0146] Formula (4.8)

[0147] The differences in style between the semantic segmentation images of the reference style image and the detected style image are as follows:

[0148] Formula (4.9)

[0149] in, This represents the number of semantically segmented regions within the image.

[0150] like Figure 6 As shown, in (b)-(g), the girl's hair exhibits a complex phenomenon where color and texture intertwine and blend. Intuitively, the color information of the girl's hair in (c) is closest to the reference style. This means that... Figure 6 (c) It provides the best visualization, consistent with the optimal EDE value of c=1 in Table I. Compared to the Gram matrix (a non-centered covariance matrix), the covariance matrix more accurately describes the feature distribution through mean normalization and reduces content interference.

[0151] In this embodiment, the Adam optimizer's learning rate was 0.001, the beta parameter was set to (0.9, 0.999), the number of training iterations was 320, and the batch size was 1. We conducted the experiments on a workstation equipped with the following hardware: Intel Xeon® Platinum 8352V, CPU @ 2.10GHz, RAM: 128.0 GB, 64-bit OS, llvmpipe (LLVM 15.0.7, 256-bit), NVIDIA RTX A6000 ×1, and Ubuntu 22.04LTS. The ILS filter was written in MATLAB 2024B, and the remaining code was written in Python 3.9.

[0152] To illustrate the migration effect of this embodiment, a comparison with existing technical methods is made:

[0153] Existing technologies include:

[0154] [1] J. Yoo, Y. Uh, S. Chun, B. Kang, and J.-W. Ha, “Photorealisticstyle transfer via wavelet transforms,” in Proceedings of the IEEE / CVFInternational Conference on Computer Vision, 2019, pp. 9036–9045.

[0155] [2] X. Li, S. Liu, J. Kautz, and M.-H. Yang, “Learning lineartransformations for fast image and video style transfer,” in Proceedings of the IEEE conference on computer vision and pattern recognition, 2019, pp.3809 3817.

[0156] [3] F. Luan, S. Paris, E. Shechtman, and K. Bala, “Deep photo styletransfer,” in Proceedings of the IEEE conference on computer vision and pattern recognition, 2017, pp. 6997–7005.

[0157] [4] J. An, S. Huang, Y. Song, D. Dou, W. Liu, and J. Luo, “Artflow: Unbi ased image style transfer via reversible neural flows,” in Proceedings of the IEEE conference on computer vision and pattern recognition, 2021

[0158] [5] K. Hong, S. Jeon, H. Yang, J. Fu, and H. Byun, “Domain-awareuniversal style transfer,” in Proceedings of the IEEE / CVF InternationalConference on Computer Vision, 2021, pp. 14609–14617.

[0159] [6] L. Wen, C. Gao, and C. Zou, “Cap-vstnet: Content affinitypreserved versatile style transfer,” in Proceedings of the IEEE / CVFConference on Computer Vision and Pattern Recognition, 2023, pp. 18300–18309.

[0160] [7] H. Ding, H. Zhang, G. Fu, C. Jiang, F. Luo, C. Xiao, and M. XU,“Towards high-quality photorealistic image style transfer,” IEEE Transactions on Multimedia, vol. 26, no. 2024, pp. 9892–9905, 2024.

[0161] [8] Y. Zhang, N. Huang, F. Tang, H. Huang, C. Ma, W. Dong, and C. Xu,“Inversion-based style transfer with diffusion models,” in Proceedings of theIEEE / CVF conference on computer vision and pattern recognition, 2023, pp.10146–10156.

[0162] [9] J. Chung, S. Hyun, and J.-P. Heo, “Style injection in diffusion: A training-free approach for adapting large-scale diffusion models for styletransfer,” in Proceedings of the IEEE / CVF Conference on Computer Vision andPattern Recognition, 2024, pp. 8795–8805.

[0163]

[10] R. Xu, W. Xi, X. Wang, Y. Mao, and Z. Cheng, “Stylesp: Samplingstartpoint enhancement for training-free diffusion-based method for styletransfer,” in Proceedings of the Computer Vision and Pattern Recognition Conference, 2025, pp. 18260–18269.

[0164]

[11] H. Chen, Z. Wang, and H. Liao, “Ccpl: Contrastive coherencepreserving loss for versatile style transfer,” in Computer Vision–ECCV 2022:17th European Conference, Tel Aviv, Israel, October 23–27, 2022, Proceedings, Part XXXVII. Springer, 2022, pp. 189–205.

[0165] Comparison Methods: Currently, the main methods of image style transfer include realistic image style transfer and artistic image style transfer. Among realistic style transfer methods, only a few methods use semantic segmentation to obtain more accurate style transfer results. This embodiment compares with style transfer methods that use GRAM matrices, such as depth photo style transfer [3]. At the same time, this embodiment also compares with style transfer methods that use WCT, such as [1] Learning linear transformation methods [2] and based on The post-processing joint filter structure [7]. The comparison results are as follows: Figure 9 As shown. In style transfer methods without semantic segmentation, this embodiment compares domain-aware general style transfer [5], the Artflow method [4], and CAP-VSTNet [6]. The comparison results are as follows. Figure 10 As shown.

[0166] Visual comparison: Figure 9 , 10 Figures 11 and 12 illustrate the visual style transfer results of different methods. This embodiment demonstrates that the method presented achieves better results. Figure 9 In most results, deep photo style transfer methods [3] and learned linear transformation methods [2] failed to preserve features well. WCT [1] The method exhibits noticeably unnatural boundaries in the facial image style transfer results. The AFCS framework

[18] failed to transfer complex colors and textures. WCT Combining with a post-processing joint filter framework [7] results in color overflow issues in some outcomes, such as facial results. In contrast, the method in this embodiment achieves the best visual results. Figure 10 In some results, domain-aware general style transfer[5] failed to achieve the desired results. The Artflow method [4] has low resolution in its image style transfer results. The results are prone to mosaic effects. CAP-VSTNet [6] is not as good as the method in this embodiment in terms of feature preservation.

[0167] exist Figure 11 Among them, the CCPL method

[11] suffers from color overflow, and their style transfer results are not good enough in terms of feature preservation. The INST method [8] will The characteristics are transformed into artistic shapes that approximate them. Therefore, the content of the style transfer result may be similar to... Completely different. The S2WAT method

[27] failed to preserve features well in some image style transfer results. For example, the texture of porcelain changed. The StyleID method [9] generated similar textures based on the original features when transferring styles. Therefore, their results are also prone to change. The content of this image leads to significant changes or blurring of the generated features, as shown in the donut image. The results of this embodiment demonstrate the best overall performance. Figure 12 In this embodiment, the results of glasses style transfer are compared with those of these 12 methods. The results show that the results of this embodiment are closer to the original methods in terms of feature preservation. The saliency of style transfer in this embodiment is also impressive.

[0168] Quantitative assessment: This embodiment uses The results of image style transfer were evaluated using SSIM, PSNR, NIQE, and AG metrics. The smaller the value, the closer the image style transfer result is to the reference style. SSIM measures the similarity between the style transfer result and the reference style. Similarity in brightness, contrast, and structure. PSNR measures the degree of image distortion. NIQE is based on a statistical model of natural scenes. The lower the value, the more natural the image style transfer result. AG measures image sharpness and detail. The higher the value, the richer the image detail and the more pronounced the contrast. Tables 2, 3, and 4 present the quantitative comparison results of image style transfer.

[0169] In Table 2, the results of this embodiment are the best in most metrics. In the first group, the EDE metric of [2] is slightly better than the results of this embodiment, and the results of [1] and [7] are the same as those of this embodiment. However, the other metrics and visual effects of this embodiment are the best. In the second group, the SSIM value of [7] is better than that of this embodiment. However, the results of [7] show obvious color overflow, such as in necklaces around the neck. The PSNR and NIQE of [2] are better than those of this embodiment. However, the feature retention results of [2] are not as good as those of this embodiment. In the third group, the SSIM value of [7] is worse than that of this embodiment, and the PSNR value of [2] is worse than that of this embodiment. However, the feature retention of the above two results is not as good as that of this embodiment. In the fourth group, the SSIM value of [7] is better than that of this embodiment, and the PSNR value of [1] is better than that of this embodiment. However, the style transfer effect of this embodiment is more significant.

[0170]

[0171] Table 2 Comparison of EDE, SSIM, PSNR, NIQE and AG Indicators

[0172] Table 3 shows the average comparison results for EDE, SSIM, PSNR, NIQE, and AG. This embodiment has the best ΔEDE and SSIM values. In the comparison of PSNR and NIQE, this embodiment's results are the second best. Overall, this embodiment outperforms the other four.

[0173]

[0174] Table 3. Average comparison results of the four indicators

[0175] In Table 4, although the PSNR value of [6] is higher than that of this embodiment in the first group, the overall effect of the results of this embodiment is better than that of [6]. Sometimes, the values ​​of SSIM and PSNR are easily affected by the significance of image style transfer. The more significant the style transfer result, The greater the change in color and brightness, the more the measurements of SSIM and PSNR are affected. In the second group, the PSNR value of [6] and the NIQE value of [5] are better than those of this embodiment. However, the overall effect of the results of this embodiment is better than that of [6] and [5]. For example, the results of this embodiment not only have significant style transfer, but also appropriate brightness and better visual effect. Figure 11 and Figure 12 In China, the content results generated by image art style transfer methods are often related to... The results are distinctly different, although they can achieve significant style transfer effects. The results of this embodiment not only achieve significant style transfer, but also better preserve... The original and important characteristics.

[0176]

[0177] Table 4 Comparison of EDE, SSIM, PSNR, NIQE and AG Indicators

[0178] In this implementation, Figure 9 In the image, (a) and (b) contain the reference image and the content image with corresponding semantic segmentation (bottom left corner of the image), respectively. (c) - (g) are the style transfer results of Luan et al. [3], Yoo et al. [1], Li et al. [2], Ding et al. [7] and our work, respectively.

[0179] Figure 10 In the middle, (a)-(f) are respectively And An et al. [4], Hong et al. [5], Wen et al. [6] and our style transfer results.

[0180] Figure 11 In the middle, (a) is (top) and (Bottom). (b)-(g) are the style transfer results of Chen et al.

[11] , Zhang et al. [8], Zhang et al.

[27] , Chung et al. [9], Xu et al.

[10] and ours, respectively.

[0181] Figure 12 In the middle, (a) is and (in the upper right corner). (b) - (n) are the style transfer results of Yoo et al. [1], Li et al. [2], Luan et al. [3], Ding et al. [7], Chen et al.

[11] , Hong et al. [5], An et al. [4], Wen et al. [6], Zhang et al. [8], Zhang et al.

[27] , Chun et al. [9], Xu

[10] and ours.

[0182] In this embodiment, two user studies were conducted on 50 randomized volunteers to verify the effectiveness of the migration method and EDE assessment in this embodiment.

[0183] This embodiment incorporates a user study to evaluate the effectiveness of style transfer methods. The four quantitative comparison methods may not be suitable for assessing image style transfer methods because they do not consider the saliency of style transfer. When style transfer is more effective, the result may cause greater substantial changes to the original content image. These changes can sometimes lead to unfair evaluations. Therefore, this embodiment introduces a user study to provide a more comprehensive comparison of various style transfer methods. In this embodiment, each volunteer is randomly shown 80 sets of style transfer results obtained using this embodiment and other comparative methods: Deep Photo Style Transfer [3], Photorealistic style transfer via wavelet transforms [1], Learning Linear Transformations for Fast Image and Video Style Transfer [2], Towards High-Quality Realistic Image Style Transfer [7], Ccpl: Contrastivecoherence preserving loss for versatile style transfer

[11] , Domain-Aware General Style Transfer [5], Artflow: Unbiased Image Style Transfer via Reversible Neural Flow [4], Cap-vstnet: General Style Transfer that Preserves Content Affinity [6], Inversion-Based Style Transfer Based on Diffusion Model [8], S2wat: Image Style Transfer via Hierarchical Visual Transformers Using Striped Window Attention

[27] , Style Injection in Diffusion: A Training-Free Method for Adapting Large-Scale Diffusion Models to Style Transfer [9], and Stylessp: Sampling Start Point Enhancement for Diffusion-Based Training-Free Style Transfer

[10] . Each volunteer browses labeled images. A survey was conducted to gather feedback on the following issues: I) accurate detail and vivid contrast; II) natural and rich color accuracy; III) realistic texture preservation; IV) consistent photographic realism. To ensure fairness, the results generated by the five methods were labeled as follows: and The result of this embodiment is labeled as The final results are shown in Table 5.

[0184] set up express exist The total number of votes on the question. To evaluate each method for each question, this embodiment uses... calculate exist The percentage of votes (PoV) received on the issue. To provide an overall evaluation of the different methods, this embodiment further uses... calculate The percentage of votes received. Table V in this embodiment shows the percentage of votes received by different methods in the survey, where Qu.x represents... The problem is that the method in this embodiment scored the highest, meaning that human subjects preferred the results of this embodiment.

[0185]

[0186] Table 5. Voting Results for Comparison Methods

[0187] User study evaluating the effectiveness of EDE: Forty style transfer results were selected, representing the outputs of five methods using semantic segmentation (including [3], [1], [2], [7], and the method described in this embodiment). Each volunteer viewed labeled images. A survey was conducted to gather feedback on the effectiveness of style transfer. Final results are available in […]. Figure 13 This example shows that the trend of the EDE evaluation curve is opposite to the trend of the percentage selected by the user. The opposite trend is more pronounced when the EDE value is small. This indicates that when using semantic segmentation to evaluate EDE, the evaluation results are consistent with the visual results.

[0188] In this embodiment, Ablation experiments were conducted using ILS filtering and loss functions to verify... The impact of ILS filtering on style transfer and the role of each component of the loss function.

[0189] Ablation experiment: Figure 14 Demonstrated the use of Ablation experiments were conducted to repair the boundary. Due to WCT The module has not yet addressed the image reconstruction errors and biased image restoration issues in the WCT model; the image style transfer results in (c) show artificial boundaries. This can be achieved by using... The texture layer repairs the output of WCT, and the WAI network improves the boundary problem, as shown in (d).

[0190] Ablation experiment: Figure 15 Showing Feature-enhanced ablation experiments. Based on the simple loss function shown in Equations (3.4) and (3.5), without... In the case of feature enhancement, the image style transfer results are darker and the features are not clear. Feature enhancement improves this problem.

[0191] ILS ablation experiment: Figure 6 Ablation experiments demonstrating enhanced ILS characteristics were presented. Figure 6 In (b), c = 0 indicates that the image style transfer result does not have ILS feature enhancement. Using ILS feature enhancement, this embodiment can obtain a result that is closer to... Image style transfer results, such as Figure 6 As shown in (c), (d), etc.

[0192] Ablation experiment of loss function: Reconstruction loss and feature loss Style saliency and feature integrity of the style transfer results are controlled separately. The ablation experiment of the loss function is as follows: Figure 16 As shown. When this embodiment performs image style transfer without a loss function, the result is darker and lacks style, as... Figure 16 As shown in (c). When this embodiment only uses... Instead of using When performing image style transfer, the resulting brightness is uneven, such as... Figure 16 As shown in (d). When this embodiment only uses... Instead of using When performing image style transfer, the result suffers from feature loss, such as Figure 16 As shown in (e). When this embodiment uses simultaneously and The best results are achieved when performing image style transfer, such as Figure 16 As shown in (f).

[0193] In this embodiment, Figure 13 In the diagram, the left vertical axis represents the EDE evaluation value, and the right vertical axis represents the user survey value. The blue dashed line represents the EDE evaluation result. The red dashed line represents the percentage of users selected by each method out of all users. The Li, Luan, Yoo, and Ding methods correspond to methods [2], [3], [1], and [7], respectively.

[0194] Figure 14 In the image, (a) to (d) represent the reference image, the content image, and the... Style transfer results and usage The result.

[0195] Figure 15 In the image, (a) to (d) represent the reference image, the content image, and the image not used, respectively. Style transfer results and usage The result.

[0196] Figure 16 In the image, (a) to (f) represent the reference image, the content image, and the image not used, respectively. and Style transfer results, usage and Style transfer results, usage and Style transfer results, usage and The style transfer results.

[0197] This embodiment discloses an effective filtering structure, the WAI network, and its evaluation metric, EDE. The WAI network consists of WCT2, AWLS, and ILS filters, and can produce more significant image style transfer effects. The EDE metric provides an important indicator for evaluating realistic image style transfer results.

Claims

1. A realistic image style transfer system, characterized in that, include: The style transfer module is used as input for the content image, the reference image, and the corresponding semantic segmentation. The style transfer module transfers the style of the reference image to the content image through an encoder and a decoder, and performs repairs during the transfer process to obtain a new image. The feature extraction module is used to acquire the content image, the reference image, and the corresponding semantic segmentation, so as to extract boundary features for image boundary repair of the new image and image features for image feature recovery of the new image; The repair module is used for data acquisition by the style transfer module and the feature extraction module. The repair module repairs the texture features of the new image by using AWLS and ILS filters and based on the boundary features and the image features to obtain the transfer result. The EDE quantization evaluation module is used to compare the color style of the transfer result with the color style of the reference image, so as to optimize the weights of the ILS filter feature layer based on the comparison result. The EDE quantification evaluation module includes a color annotation submodule and an EDE difference calculation submodule, and uses the migration result as the detection style image and the reference image as the reference style image. In the EDE difference calculation submodule, the style covariance matrix is ​​decomposed into singular values: Official (4.1) in, The style covariance matrix; It is a left singular matrix; It is a right singular matrix. The diagonal elements are the singular value diagonal matrix; the reference style image and the detected style image are input to obtain the style covariance matrix: , in, and These are diagonal matrices representing the reference style image and the detected style image, respectively, and each contains singular values; and These are the right singular vector transposes of the reference style image and the detected style image, respectively; The effective rank of the diagonal elements of the singular value diagonal matrix is: Official (4.2) in, For effective rank; Singular values ​​representing style characteristics; The energy conservation threshold; It is a matrix The number of rows represents the dimension of the feature vector; Take before The square root of each singular value, multiplied by the weight. To obtain weighted scaling : Official (4.3) Will Set to 1 to obtain the principal component directions: Official (4.4) Will Multiply Then multiply by The transpose of the matrix is ​​used to reconstruct the symmetric matrix. : Official (4.5) in, For right singular matrices List; for A diagonal matrix of singular values; These are weights, used to control the intensity of style transfer; EDE achieves low-rank embedding in the feature space by truncating singular values, and... Set as a low-rank approximation with the style covariance matrix: Official (4.6) The semantic segmentation map of the reference style image, the semantic segmentation map of the detected style image, and the semantic segmentation map of the reference style image are calculated according to formula (4.6). Matrix, the detected style image The low-rank approximation of the covariance matrix of the matrix, and the style difference between the reference style image and the detected style image is: Official (4.8) The difference in style between the semantic segmentation images of the reference style image and the detected style image is as follows: Official (4.9) in, This represents the number of semantically segmented regions within the image.

2. The realistic image style transfer system according to claim 1, characterized in that: The encoder is a VGG-19 network pre-trained on ImageNet, which incorporates Haar wavelet pooling from the conv1_1 layer to the conv4_1 layer to form the WCT. 2 WCT of Coding Architecture 2 The encoder, the decoder mirrors the WCT 2 Encoder, and is WCT 2 Decoder, the style transfer module in the WCT 2 After the WCT output of the last block of the decoder, it is repaired by the repair module.

3. The realistic image style transfer system according to claim 2, characterized in that: The WCT 2 The last block of the decoder corresponds to the LL layer of wavelet decomposition. The style transfer module supplements the boundaries of the subgraph style transfer results of the LL layer through the repair module.

4. The realistic image style transfer system according to claim 2, characterized in that: The feature extraction module, in the boundary feature extraction, extracts the content image. As , also as The semantic segmentation, and its semantic segmentation, are respectively input into the WCT. 2 Encoder, and in the WCT 2 The last block of the decoder extracts the corresponding boundary features. ; The feature extraction module, in the image feature extraction process, will... , and the corresponding semantic segmentation input to the WCT 2 The encoder performs style transfer to obtain Through WCT 2 AWLS boundary repair module of the decoder ,right AWLS-based boundary repair is performed, and by using... AWLS texture layer replacement AWLS texture layer to repair To obtain : Official (2.1) Official (2.2) Among them, among them, The structural layers obtained for the AWLS filter; The texture layer obtained by the AWLS filter; the parameters in parentheses represent the image before smoothing. Will As and Input to In the module, and in the WCT 2 The last decoder layer extracts the corresponding image features. .

5. A realistic image style transfer system according to claim 4, characterized in that: The repair module is for the The repair method is the same as the image feature extraction method: Official (3.1) in, The result of boundary feature repair; for The result of style transfer; The Enhanced style transfer is performed using the image features and the AWLS filter to obtain... The Through ILS filter and Enhance texture features to obtain ; Regarding the Decode the data to obtain the migration results. .

6. A realistic image style transfer system according to claim 5, characterized in that: The repair module is for the The repair method is as follows: [The rest of the text appears to be incomplete and requires further context.] The input is fed into the AWLS filter to obtain the structural layer. ; Regarding the minus To obtain a boundary-free problem .

7. A realistic image style transfer system according to claim 5, characterized in that: The repair module uses an AWLS filter to process the image features. Enhanced style transfer: Official (3.2) in, for Enhanced results; for AWLS feature layer, yes AWLS structure layers; This is for AWLS feature layer; for The weight value; for The weight value; for The weight value; The repair module passes through ILS texture features to enhance Furthermore, the texture features of the AWLS repair result are supplemented by an ILS texture layer: Official (3.3) in, Supplementing results for ILS; The texture layer obtained by ILS smoothing; for The weight value.

8. A realistic image style transfer system according to claim 7, characterized in that: The repair module performs boundary repair and feature enhancement based on the AWLS filter and the ILS filter, and its loss function includes... and : Official (3.4) Official (3.5) in, , These are the weights of the loss function; This is for and The variance; for and The variance; for Features of wavelet coding layer 4; To reconstruct the image Features of wavelet coding layer 4.

9. A realistic image style transfer system according to claim 7, characterized in that: The color annotation submodule is used for annotation of the reference style image and the detection style image. The color annotation submodule has a `color_dict` set to define semantic content. The color annotation submodule performs pixel-by-pixel detection on the colors of the semantic segmentation maps of the reference style image and the detection style image. When a pixel's color belongs to the `color_dict` set, the corresponding color is marked on the semantic segmentation maps of the reference style image and the detection style image. When a pixel's color does not belong to the `color_dict` set, the closest color in the `color_dict` set is calculated using Manhattan distance, and the corresponding closest color is marked on the semantic segmentation maps of the reference style image and the detection style image. The EDE difference calculation submodule is used for proximity analysis between the reference style image and the detected style image. After standardizing the format of the reference style image and the detected style image, the EDE difference calculation submodule decomposes all semantic regions of both images using singular value decomposition based on the labeled colors, thereby obtaining the reference style image. and the detected style image ; It is a diagonal matrix of the reference style image and the detected style image, and each contains singular values; and These are the right singular vector transposes of the reference style image and the detected style image, respectively. The EDE difference calculation submodule for and The reference style image is obtained by reconstructing and weighting. Matrix and the detected style image Matrix, calculate the Matrix and the stated The root mean square of the differences between the matrices is used to obtain the degree of similarity between the reference style image and the detected style image, and a comparison result is obtained based on the degree of similarity.

Images