Image style migration method and device for guiding region self-adaption based on similarity measurement

Abstract

The invention relates to an image style migration method and device based on similarity measurement guide area self-adaption, which can generate a naturally coordinated stylized image, realize self-adaptive style migration based on a semantic area, enhance the detail capture capability of a model, and generate a visually satisfactory high-quality image while keeping the complete structure of the image. The method comprises the following steps: inputting features extracted from content and style images into a texture migration module guided by similarity measurement, and guiding a Transform decoder to fuse the content and style features by a cross attention module based on a Gaussian kernel function; the method comprises the following steps: respectively dividing a content image Ic and a style image Is into local image blocks, expanding each image block into a feature vector, respectively constructing a content feature matrix Vc and a style feature matrix Vs, sequentially calculating a similarity matrix # imgabs0 # # imgabs1 # between all content blocks and style blocks by using a Gaussian kernel function as an attention mask to modulate cross attention calculation, attention distribution is guided through semantic similarity, and accurate matching of the content and the style area is achieved.

Description

Technical Field

[0001] The present invention relates to the technical field of image processing, and in particular to an image style migration method based on similarity measurement-guided region adaptation, and an image style migration device based on similarity measurement-guided region adaptation. Background Art

[0002] With the rapid development of artificial intelligence (AI), image style transfer has become a key research area in computer vision. This technique aims to transfer features such as texture and color from style images to natural images, generating high-quality artistic images while preserving the content structure. Style transfer technology has a wide range of applications, spanning traditional artistic creation to modern digital entertainment. It demonstrates significant potential and application value in areas such as image generation, film and television production, virtual reality, and digital creativity.

[0003] Early style transfer methods primarily transferred style by constructing mathematical statistical models of textures, but these methods suffered from poor generalization and slow image reconstruction. With the development of deep learning and the introduction of multi-layer neural networks, style transfer methods based on convolutional neural networks have gradually become a research hotspot. Some achieve efficient style extraction and transfer by matching the statistical distribution of image features, others learn and fuse style representations by analyzing similarities between features, and some utilize projection flow networks to optimize stylization results. While these methods improve generation efficiency, convolutional neural networks have limitations in capturing global features, resulting in stylized images with missing content or distorted details. Furthermore, methods based on generative adversarial networks improve generation quality through adversarial training, but these models suffer from training instability, and generated images often exhibit artifacts or localized distortion. In recent years, methods based on diffusion models, while enabling highly flexible style generation through a gradual denoising process, suffer from high computational complexity and poor controllability.

[0004] To address the above issues, attention mechanisms and Transformer models have been further introduced into style transfer tasks to improve the model's content understanding and style expression capabilities. The attention mechanism enhances the model's ability to focus on key areas by dynamically allocating feature weights. Some methods incorporate soft attention into learnable networks, while others combine attention with adaptive normalization modules to better preserve the overall structure of the image. Vision Transformer models have also been introduced to effectively capture long-range image dependencies, effectively alleviating the problem of feature leakage and achieving unbiased style transfer. However, such methods do not consider the semantic characteristics of the image and are not effective in transferring specific styles. In addition, some methods use semantic segmentation models to divide regions to achieve local style control, but segmentation accuracy directly affects the transfer effect. Segmentation errors in complex scenes can cause style distortion, loss of details, and other problems. Summary of the Invention

[0005] In order to overcome the shortcomings of the existing technology, the technical problem to be solved by the present invention is to provide an image style transfer method based on similarity measurement-guided region adaptation, which can generate natural and coordinated stylized images, realize adaptive style transfer based on semantic regions, enhance the model's detail capture ability, and generate visually satisfactory high-quality images while preserving the complete structure of the image.

[0006] The technical solution of the present invention is: this method of image style transfer based on similarity measurement-guided region adaptation, which includes the following steps:

[0007] (1) Input content image and style image;

[0008] (2) Generate a signal-to-noise ratio map through the signal-to-noise ratio estimation module;

[0009] (3) Preliminary extraction of features of content image and style image through encoder;

[0010] (4) The global and local features of the content image and style image are extracted respectively through the Transformer model and the multi-layer attention module, where the attention module based on the signal-to-noise ratio guides

[0011] Transformer model for feature extraction;

[0012] (5) Generate content features and style features through a feature fusion module based on signal-to-noise ratio and projection operation;

[0013] (6) Input the content and style images and the extracted image features into the texture transfer module guided by the similarity metric, calculate the cross attention module based on the Gaussian kernel function, and guide the Transformer decoder to fuse the content features and style features;

[0014] (7) Restore the features to a stylized image through the decoder;

[0015] (8) Output stylized image;

[0016] In the step (6), the content image I c and style image I s Divided into local blocks, H and W represent the length and width of the image respectively, d represents the size of the local block, and each image block is expanded into a feature vector to construct the content feature matrix V c And style feature matrix V s , use the Gaussian kernel function to calculate the similarity matrix between all content blocks and style blocks in turn Similarity Matrix As an attention mask, it modulates the cross-attention calculation and guides the attention distribution through semantic similarity to achieve accurate matching of content and style regions.

[0017] The present invention achieves regional adaptive style transfer and generates natural and coordinated stylized images by introducing similarity metrics and multi-layer attention mechanisms. A Gaussian kernel function is used as a similarity metric to calculate the semantic similarity between the content image and the local region of the style image, generate an attention mask, and dynamically modulate the cross-attention mechanism to achieve adaptive style transfer based on semantic regions. In a dual-branch feature extraction network, a multi-layer attention module is constructed using convolution and self-attention mechanisms as short-range branches to extract local texture features and enhance the model's ability to capture details. Experimental results on the MS-COCO and WikiArt datasets show that the style transfer method proposed in the present invention can generate visually satisfactory high-quality images while retaining the complete structure of the image.

[0018] A similarity metric-based region-adaptive image style migration device is also provided, the device comprising:

[0019] an input module configured to input a content image and a style image;

[0020] a signal-to-noise ratio estimation module configured to generate a signal-to-noise ratio map;

[0021] An encoder, which is configured to initially extract features of the content image and the style image;

[0022] The extraction module is configured to extract global and local features of the content image and style image respectively through the Transformer model and the multi-layer attention module;

[0023] A feature fusion module, which is configured to fuse and project content features and style features;

[0024] The guidance module is configured to guide the Transformer decoder to fuse content features and style features through a cross-attention module based on the Gaussian kernel function, c and style image I s Divided into local blocks, H and W represent the length and width of the image respectively, d represents the size of the local block, and each image block is expanded into a feature vector to construct the content feature matrix V c And style feature matrix V s , use the Gaussian kernel function to calculate the similarity matrix between all content blocks and style blocks in turn Similarity Matrix As an attention mask, it modulates the cross-attention calculation and guides the attention distribution through semantic similarity to achieve accurate matching of content and style regions;

[0025] a decoder, which is configured to restore the features to a stylized image;

[0026] An output module configured to output a stylized image. BRIEF DESCRIPTION OF THE DRAWINGS

[0027] Figure 1 The figure is a flowchart of the image style transfer method based on similarity measurement guided region adaptation according to the present invention.

[0028] Figure 2 FIG. 1 is a diagram of a style transfer framework based on similarity metric-guided region adaptation according to the present invention.

[0029] Figure 3 Flowchart for calculating the attention mask in the cross-attention module based on the Gaussian kernel function according to the present invention.

[0030] Figure 4 a is the structure diagram of the multi-layer attention module; Figure 4 b is the structure diagram of the convolution module; Figure 4 c is the structure diagram of the self-attention module; Figure 4 d is the flow chart of the multi-head self-attention module.

[0031] Figure 5 a is a style image; Figure 5 b is the content image; Figure 5 c-5l is the generated images of 9 style transfer methods.

[0032] Figure 6 a is another style image; Figure 6 b is the content image; Figure 6 c-6l is the generated images of 9 style transfer methods.

[0033] Figure 7 a is another style image; Figure 7 b is the content image; Figure 7 c-7l is the generated images of 9 style transfer methods. DETAILED DESCRIPTION

[0034] like Figure 1 As shown in FIG, this image style transfer method based on similarity metric guided region adaptation includes the following steps:

[0035] (1) Input content image and style image;

[0036] (2) Generate a signal-to-noise ratio map through the signal-to-noise ratio estimation module;

[0037] (3) Preliminary extraction of features of content image and style image through encoder;

[0038] (4) The global and local features of the content image and style image are extracted respectively through the Transformer model and the multi-layer attention module, where the attention module based on the signal-to-noise ratio guides

[0039] Transformer model for feature extraction;

[0040] (5) Generate content features and style features through a feature fusion module based on signal-to-noise ratio and projection operation;

[0041] (6) Input the content and style images and the extracted image features into the texture transfer module guided by the similarity metric, calculate the cross attention module based on the Gaussian kernel function, and guide the Transformer decoder to fuse the content features and style features;

[0042] (7) Restore the features to a stylized image through the decoder;

[0043] (8) Output stylized image;

[0044] In the step (6), the content image I c and style image I s Divided into local blocks, H and W represent the length and width of the image respectively, d represents the size of the local block, and each image block is expanded into a feature vector to construct the content feature matrix V c And style feature matrix V s , use the Gaussian kernel function to calculate the similarity matrix between all content blocks and style blocks in turn Similarity Matrix As an attention mask, it modulates the cross-attention calculation and guides the attention distribution through semantic similarity to achieve accurate matching of content and style regions.

[0045] The present invention achieves regional adaptive style transfer and generates natural and coordinated stylized images by introducing similarity metrics and multi-layer attention mechanisms. A Gaussian kernel function is used as a similarity metric to calculate the semantic similarity between the content image and the local region of the style image, generate an attention mask, and dynamically modulate the cross-attention mechanism to achieve adaptive style transfer based on semantic regions. In a dual-branch feature extraction network, a multi-layer attention module is constructed using convolution and self-attention mechanisms as short-range branches to extract local texture features and enhance the model's ability to capture details. Experimental results on the MS-COCO and WikiArt datasets show that the style transfer method proposed in the present invention can generate visually satisfactory high-quality images while retaining the complete structure of the image.

[0046] Preferably, in step (6),

[0047]

[0048] Where V c i and V s j Represent the feature vectors of the i-th content block and the j-th style block respectively, ||·||2 represents the L2 norm, and σ is the Gaussian kernel scale parameter; the similarity matrix As the attention mask modulates the cross-attention calculation, the calculation process of the cross-attention module based on the similarity metric is:

[0049]

[0050] Among them, softmax is a normalization operation. is the scaling factor, α is a hyperparameter, Q, K, and V are query, key, and value features respectively; multiple attention heads are calculated in parallel through the multi-head mechanism, and the final result is expressed as:

[0051] MRCA(Q,K,V)=concat(RCA1(Q,K,V),RCA2(Q,K,V),...,RCA N (Q, K, V))W o (3)

[0052] Where W o is a learnable parameter, concat is the channel concatenation operation, and N is the number of attention heads.

[0053] Preferably, in step (6), the Transformer decoder uses a cross-attention module to input style features X″ c and content feature X″ s Adaptive decoding: First add content-aware position coding P to the content features c , get content sequence B c ={X″ c1 +P c1 ,x″ c2 +P c2 ,...,X″ cL +P cL} and style sequence B s ={X″ s1 , X″ s2 ,...,X″ sL}, L represents the sequence length; use the content sequence to generate query Q = B c W q , using the style sequence to generate the key K = B s W k Sum V = B s W v , where W q , Wk ,,W v is a weight matrix; each Transformer decoding layer consists of two cross-attention modules MRCA based on Gaussian kernel function and a feedforward network FFN. The calculation process is:

[0054] Y=MRCA(Q,K,V)+Q (4)

[0055] Y′=MRCA(Y+P c , K, V)+Y (5)

[0056] Y″=FFN(Y′)+Y′ (6)

[0057] Among them, layer normalization is applied after each module.

[0058] Preferably, in step (4), a multi-layer attention module is constructed as a short-range branch, which is composed of a convolutional module ConvBlock and a self-attention module SAM alternatingly. Each convolutional module includes a convolutional layer Conv, a batch normalization BN and a ReLU activation function. For the input feature F,

[0059] ConvBlock(F)=ReLU(BN(Conv(F)))+F (7);

[0060] The self-attention module contains multi-head self-attention MHSA, batch normalization BN and ReLU activation function. For the input feature F,

[0061] SAM(F)=ReLU(BN(MHSA(F)))+F (8).

[0062] Preferably, in step (4), in the multi-head self-attention module, the input feature F is first mapped to the query, key and value spaces respectively through three independent 1×1 convolutional layers,

[0063] Q f =query(F),K f =key(F), V f =value(F) (9)

[0064] Among them, query, key, and value all represent convolution operations;

[0065] In order to enhance the spatial variation ability of features, the content-aware position encoding Pos embed and query

[0066] Q f and key K f To perform the fusion:

[0067] Q′f =Q f +Pos embed , K′ f =K f +Pos embed (10)

[0068] Finally, the calculation process of multi-head self-attention is expressed as:

[0069]

[0070] Among them, softmax is a normalization operation. is the scaling factor.

[0071] Preferably, the method adopts a joint loss function Train the network, including content-aware loss Style-aware loss and consistency loss and

[0072]

[0073] where λ c ,λ s ,λ id1 ,λ id2 is a hyperparameter used to balance the degree of content preservation and style transfer; content-aware loss function Used to measure the generated image I cs and the original content image I c The differences in content and structure are as follows:

[0074]

[0075] represents the features extracted from the i-th layer of the pre-trained VGG-19 model, ||·||2 represents the L2 norm, and N l Indicates the total number of layers;

[0076] Style-aware loss function Used to measure the generated image I cs and the original style image I s The difference in color and texture between them is expressed as follows:

[0077]

[0078] μ(·) represents the mean of the feature, σ(·) represents the variance of the feature, ||·||2 represents the L2 norm, and the Gram matrix of the feature map extracted by VGG-19 is used to represent the image style features;

[0079] Consistency loss and The expression is as follows:

[0080]

[0081] Among them I cc and I ss Two identical content images I c or style image I s The output obtained from the input model;

[0082] Minimize the total loss function.

[0083] Those skilled in the art will appreciate that all or part of the steps in the above-described embodiment method can be implemented by instructing the relevant hardware through a program. The program can be stored in a computer-readable storage medium. When executed, the program includes the steps of the above-described embodiment method. The storage medium can be: ROM / RAM, a magnetic disk, an optical disk, a memory card, etc. Therefore, corresponding to the method of the present invention, the present invention also includes an image style transfer device based on similarity metric-guided region adaptation, which is generally represented in the form of functional modules corresponding to the steps of the method. The device includes:

[0084] an input module configured to input a content image and a style image;

[0085] a signal-to-noise ratio estimation module configured to generate a signal-to-noise ratio map;

[0086] An encoder, which is configured to initially extract features of the content image and the style image;

[0087] The extraction module is configured to extract global and local features of the content image and style image respectively through the Transformer model and the multi-layer attention module;

[0088] A feature fusion module, which is configured to fuse and project content features and style features;

[0089] The guidance module is configured to guide the Transformer decoder to fuse content features and style features through a cross-attention module based on the Gaussian kernel function, c and style image I s Divided into Local block, H and W represent the length and width of the image respectively, d represents the local block size, each image block is expanded into a feature vector, and the content feature matrix V is constructed respectively c And style feature matrix V s , use the Gaussian kernel function to calculate the similarity matrix between all content blocks and style blocks in turn Similarity Matrix As an attention mask, it modulates the cross-attention calculation and guides the attention distribution through semantic similarity to achieve accurate matching of content and style regions;

[0090] a decoder, which is configured to restore the features to a stylized image;

[0091] An output module configured to output a stylized image.

[0092] Preferably, in the guidance module,

[0093]

[0094] Where V c i and V s j denote the feature vectors of the i-th content block and the j-th style block respectively, ||·||2 denotes the L2 norm, and σ is the Gaussian kernel scale parameter; the similarity matrix R is used as the attention mask to modulate the cross-attention calculation. The calculation process of the cross-attention module based on the similarity metric is:

[0095]

[0096] Among them, softmax is a normalization operation. is the scaling factor, α is a hyperparameter, Q, K, and V are query, key, and value features respectively; multiple attention heads are calculated in parallel through the multi-head mechanism, and the final result is expressed as:

[0097] MRCA(Q,K,V)=concat(RCA1(Q,K,V),RCA2(Q,K,V),...,RCA N (Q, K, V))W o (3)

[0098] Where W o is a learnable parameter, concat is the channel concatenation operation, and N is the number of attention heads.

[0099] Preferably, in the guidance module, the Transformer decoder uses a cross-attention module to input style features X″ c and content feature X″ s Adaptive decoding: First add content-aware position coding P to the content features c , get content sequence B c ={X″ c1 +P c1 , X″ c2 +P c2 ,...,X″ cL +PcL} and style sequence B s ={X″ s1 , X″ s2 ,...,X″ sL}, L represents the sequence length; use the content sequence to generate query Q = B c W q , using the style sequence to generate the key K = B s W k Sum V = B s W v , where W q , W k ,,W v is a weight matrix; each Transformer decoding layer consists of two cross-attention modules MRCA based on Gaussian kernel function and a feedforward network FFN. The calculation process is:

[0100] Y=MRCA(Q,K,V)+Q (4)

[0101] Y′=MRCA(Y+P c ,K,V)+Y (5)

[0102] Y″=FFN(Y′)+Y′ (6)

[0103] Among them, layer normalization is applied after each module.

[0104] Preferably, in the extraction module, a multi-layer attention module is constructed as a short-range branch. The module is composed of a convolution module ConvBlock and a self-attention module SAM alternately. Each convolution module includes a convolution layer Conv, a batch normalization BN and a ReLU activation function. For the input feature F,

[0105] ConvBlock(F)=ReLU(BN(Conv(F)))+F (7);

[0106] The self-attention module contains multi-head self-attention MHSA, batch normalization BN and ReLU activation function. For the input feature F,

[0107] SAM(F)=ReLU(BN(MHSA(F)))+F(8).

[0108] The present invention will be described in further detail below in conjunction with specific implementation methods.

[0109] 1. Dataset

[0110] In our experiments, we used MS-COCO as the content image dataset, which contains a rich collection of natural scene images, including people, objects, landscapes, architecture, animals, and plants. We also used WikiArt as the style image dataset, which contains paintings in various art forms, including oil paintings, watercolors, and sketches.

[0111] 2. Comparison of algorithms

[0112] The proposed method is compared with the global statistics-based methods AdaIN, WCT, ArtFlow, MCC, CAST, the attention mechanism-based methods SANet, AdaAttN, and the Transformer-based method StyTr on the test dataset. 2 , STRAT. All compared methods are tested using pre-trained models.

[0113] 3. Experimental Configuration

[0114] In the model architecture, each encoder layer consists of a 3×3 convolution, batch normalization, and ReLU activation function, and is downsampled by a maximum pooling operation. Each decoder layer consists of a 3×3 convolution, ReLU activation function, and a 2x upsampling operation to restore the image resolution to its original size. The image needs to be divided into blocks before being input into the Transformer model, and the local block size d is set to 8. During the training process, the weight λ in the loss function c ,λ s ,,λ id1 and λ id2 They are set to 10, 7, 50 and 7 respectively. All input images are randomly cropped to 224×224 size during training, while in the test phase, the model supports image input of any size.

[0115] In order to verify the effectiveness of the image style transfer method based on similarity metric guided region adaptation proposed in this invention, subjective and objective comparative experiments were conducted. The present invention uses content loss, LPIPS, and DISTS_C as content evaluation indicators, style loss, FID, and DISTS_S as style evaluation indicators, and CLIP_IQA as an overall quality evaluation indicator to evaluate the generated images from three aspects: content, style, and overall quality. Except for CLIP_IQA, the lower the results of all indicators, the better the effect. The present invention randomly selected 1,000 content images and 1,000 style images from the MS-COCO and WikiArt datasets to form a test dataset. All comparison methods generated 1,000 stylized images through the test dataset, and took the average calculation results of all images as the objective quality evaluation results. Table 1 shows the objective quality experimental results of 9 comparison methods, where bold indicates the best result and underline indicates the suboptimal result. From the table, we can see that the proposed method outperforms the comparison method in terms of content evaluation indicators such as LPIPS, DISTS_C, and content loss. It is also competitive in terms of style fidelity and outperforms other Transformer-based methods. The overall image quality CLIP-IQA is the highest, which shows that the proposed method is significantly better than the existing methods in terms of content fidelity, style naturalness, and detail preservation. In order to visually compare the stylization effects of different methods, Figures 5 to 7 Generated images of different styles are shown. The proposed method can adaptively transfer the style to more relevant content areas while maintaining the integrity of details and structure. The color and texture are more consistent with the reference style, and the transfer effect is more natural and harmonious, which is superior to the comparative method.

[0116] Table 1

[0117] method Content loss LPIPS DISTS_C Style Loss FID DISTS_S CLIP_IQA AdaIN 1.92 0.48 0.332 <![CDATA[ 0.788 ]]> 103.58 <![CDATA[ 0.311 ]]> 0.535 WCT 2.86 0.60 0.390 1.048 154.25 0.317 0.601 SANet 1.80 0.48 0.328 0.686 96.57 0.301 0.566 MCC 1.57 0.42 0.318 0.825 <![CDATA[ 98.63 ]]> 0.314 0.530 ArtFlow 1.58 0.44 0.318 1.106 106.09 0.325 0.520 AdaAttN 2.11 0.43 0.328 1.014 99.35 0.323 0.519 CAST 1.69 0.38 0.273 1.839 101.46 0.349 0.584 <![CDATA[StyTr 2 ]]> 1.41 0.40 0.279 0.827 107.34 0.334 <![CDATA[ 0.645 ]]> STRAT 1.27 <![CDATA[ 0.36 ]]> <![CDATA[ 0.271 ]]> 0.989 107.19 0.329 0.637 RBF-START <![CDATA[ 1.36 ]]> 0.35 0.270 1.309 105.89 0.316 0.681

[0118] The above description is merely a preferred embodiment of the present invention and does not constitute any form of limitation to the present invention. Any simple modifications, equivalent changes and modifications made to the above embodiments based on the technical essence of the present invention are still within the scope of protection of the technical solution of the present invention.

Claims

1. A similarity-metric guided region-adaptive image style transfer method, characterized by: It includes the following steps: (1) Input content image and style image; (2) Generate a signal-to-noise ratio map through the signal-to-noise ratio estimation module; (3) Preliminary extraction of features of content image and style image through encoder; (4) The global and local features of the content image and style image are extracted respectively through the Transformer model and the multi-layer attention module, where the signal-to-noise ratio-based attention module guides the Transformer model to perform feature extraction; (5) Generate content features and style features through a feature fusion module based on signal-to-noise ratio and projection operation; (6) Input the content and style images and the extracted image features into the texture transfer module guided by the similarity metric, calculate the cross attention module based on the Gaussian kernel function, and guide the Transformer decoder to fuse the content features and style features; (7) Restore the features to a stylized image through the decoder; (8) Output stylized image; In the step (6), the content image I c and style image I s Divided into local blocks, H and W represent the length and width of the image respectively, d represents the size of the local block, and each image block is expanded into a feature vector to construct the content feature matrix V c And style feature matrix V s , use the Gaussian kernel function to calculate the similarity matrix between all content blocks and style blocks in turn Similarity Matrix As an attention mask, it modulates the cross-attention calculation and guides the attention distribution through semantic similarity to achieve accurate matching of content and style regions.

2. The image style transfer method based on similarity metric guided region adaptation according to claim 1, characterized in that: In the step (6), in and Represent the feature vectors of the y-th content block and the j-th style block respectively, ||·||2 represents the L2 norm, and σ is the Gaussian kernel scale parameter; the similarity matrix As the attention mask modulates the cross-attention calculation, the calculation process of the cross-attention module based on the similarity metric is: Among them, softmax is a normalization operation. is the scaling factor, α is a hyperparameter, Q, K, and V are query, key, and value features respectively; multiple attention heads are calculated in parallel through the multi-head mechanism, and the final result is expressed as: MRCA(Q,K,V)=concat(RCA1(Q,K,V),RCA2(Q,K,V),...,RCA N (Q,K,V))W o (3) Where W o is a learnable parameter, concat is the channel concatenation operation, and N is the number of attention heads.

3. The image style transfer method based on similarity metric guided region adaptation according to claim 2, characterized in that: In step (6), the Transformer decoder uses the cross attention module to input style features X″ c and content feature X″ s Adaptive decoding: First add content-aware position coding P to the content features c , get content sequence B c ={X" c1 +P c1 , X" c2 +P c2 ,...,X" cL +P cL } and style sequence B s ={X" s11 ,X” s2 ,...,X" sL }, L represents the sequence length; use the content sequence to generate query Q = B c W q , using the style sequence to generate the key K = B s W k Sum V = B s W v , where W q , W k ,,W v is a weight matrix; each Transformer decoding layer consists of two cross-attention modules MRCA based on Gaussian kernel function and a feedforward network FFN. The calculation process is: Y=MRCA(Q,K,V)+Q (4) Y′=MRCA(Y+P c ,K,V)+Y(5) Y″=FFN(Y)+Y′ (6) Among them, layer normalization is applied after each module.

4. The image style transfer method based on similarity metric guided region adaptation according to claim 3, characterized in that: In step (4), a multi-layer attention module is constructed as a short-range branch. The module is composed of a convolutional module ConvBlock and a self-attention module SAM alternately. Each convolutional module contains a convolutional layer Conv, a batch normalization BN and a ReLU activation function. For the input feature F, ConvBlock(F)=ReLU(BN(Conv(F)))+F (7) The self-attention module contains multi-head self-attention MHSA, batch normalization BN and ReLU activation function. For the input feature F, SAM(F)=ReLU(BN(MHSA(F)))+F (8).

5. The image style transfer method based on similarity metric guided region adaptation according to claim 4, characterized in that: In step (4), in the multi-head self-attention module, the input feature F is first mapped to the query, key and value spaces respectively through three independent 1×1 convolutional layers. Q f =query(F),K f =key(F),V f =value(F) (9) Among them, query, key, and value all represent convolution operations; In order to enhance the spatial variation ability of features, the content-aware position encoding Pos embed With query Q f and key K f To perform the fusion: Q f ′=Q f +Pos embed ,K f ′=K f +Po sembed (10) Finally, the calculation process of multi-head self-attention is expressed as: Among them, softmax is a normalization operation. is the scaling factor.

6. The image style transfer method based on similarity metric guided region adaptation according to claim 5, characterized in that: This method adopts a joint loss function Train the network, including content-aware loss Style-aware loss and consistency loss and where λ c ,λ s ,λ id1 ,λ id2 is a hyperparameter used to balance the degree of content preservation and style transfer; content-aware loss function Used to measure the generated image I cs and the original content image I c The differences in content and structure are as follows: represents the features extracted from the i-th layer of the pre-trained VGG-19 model, ‖·‖2 represents the L2 norm, and N l Indicates the total number of layers; Style-aware loss function Used to measure the generated image I cs and the original style image I s The difference in color and texture between them is expressed as follows: μ(·) represents the mean of the feature, σ(·) represents the variance of the feature, ||·||2 represents the L2 norm, and the Gram matrix of the feature map extracted by VGG-19 is used to represent the image style features; Consistency loss and The expression is as follows: Among them I cc and I ss Two identical content images I c or style image I s The output obtained from the input model; Minimize the total loss function.

7. A similarity-metric-based, region-adaptive image style transfer device, characterized by: It includes: an input module configured to input a content image and a style image; a signal-to-noise ratio estimation module configured to generate a signal-to-noise ratio map; An encoder, which is configured to initially extract features of the content image and the style image; The extraction module is configured to extract global and local features of the content image and style image respectively through the Transformer model and the multi-layer attention module; A feature fusion module, which is configured to fuse and project content features and style features; The guidance module is configured to guide the Transformer decoder to fuse content features and style features through a cross-attention module based on the Gaussian kernel function, c and style image I s Divided into Local block, H and W represent the length and width of the image respectively, d represents the local block size, each image block is expanded into a feature vector, and the content feature matrix V is constructed respectively c And style feature matrix V s , use the Gaussian kernel function to calculate the similarity matrix between all content blocks and style blocks in turn Similarity Matrix As an attention mask, it modulates the cross-attention calculation and guides the attention distribution through semantic similarity to achieve accurate matching of content and style regions; a decoder, which is configured to restore the features to a stylized image; An output module configured to output a stylized image.

8. The image style transfer device based on similarity metric guided region adaptation according to claim 7, characterized in that: In the guidance module, in and Represent the feature vectors of the y-th content block and the j-th style block respectively, ||·||2 represents the L2 norm, and σ is the Gaussian kernel scale parameter; the similarity matrix As the attention mask modulates the cross-attention calculation, the calculation process of the cross-attention module based on the similarity metric is: Among them, softmax is a normalization operation. is the scaling factor, α is a hyperparameter, Q, K, and V are query, key, and value features respectively; multiple attention heads are calculated in parallel through the multi-head mechanism, and the final result is expressed as: MRCA(Q,K,V)=concat(RCA1(Q,K,V),RCA2(Q,K,V),...,RCA N (Q,K,V))W o (3) Where W o is a learnable parameter, concat is the channel concatenation operation, and N is the number of attention heads.

9. The image style transfer device based on similarity metric guided region adaptation according to claim 8, characterized in that: In the guidance module, the Transformer decoder uses the cross attention module to input style features X″ c and content feature X″ s Adaptive decoding: First add content-aware position coding P to the content features c , get content sequence B c ={X" c1 +P c1 , X" c2 +P c2 ,...,X" cL +P cL } and style sequence B s ={X" s1 ,X" s2 ,...,X" sL }, L represents the sequence length; use the content sequence to generate query Q = B c W q , using the style sequence to generate the key K = B s W k Sum V = B s W v , where W q , W k ,,W v is a weight matrix; each Transformer decoding layer consists of two cross-attention modules MRCA based on Gaussian kernel function and a feedforward network FFN. The calculation process is: Y=MRCA(Q,K,V)+Q (4) Y′=MRCA(Y+P c ,K,V)+Y(5) Y″=FFN(Y′)+Y′ (6) Among them, layer normalization is applied after each module.

10. The image style transfer device based on similarity metric guided region adaptation according to claim 9, characterized in that: In the extraction module, a multi-layer attention module is constructed as a short-range branch. The module is composed of a convolution module ConvBlock and a self-attention module SAM alternately. Each convolution module contains a convolution layer Conv, a batch normalization BN and a ReLU activation function. For the input feature F, ConvBlock(F)=ReLU(BN(Conv(F)))+F (7) The self-attention module contains multi-head self-attention MHSA, batch normalization BN and ReLU activation function. For the input feature F, SAM(F)=ReLU(BN(MHSA(F)))+F (8).

Images