Image art style transfer algorithm, storage medium and device

Abstract

This invention belongs to the field of image processing technology, and specifically relates to an algorithm for transferring artistic image style. The algorithm comprises the following steps: Step S1, constructing a generative network G, whose basic framework includes an encoder and a decoder; Step S2, constructing a discriminative network D, which uses a Markov discriminator and is composed of a series of continuous convolutional modules; Step S3, performing adversarial training between the generative network G and the discriminative network D, and using the trained generative network G to generate a target style-transferred image. This algorithm effectively preserves image structural information and style features during the feature fusion process, significantly improving the cross-scale feature fusion effect and resulting in a generated image with richer details and more natural style transitions.

Description

Technical Field

[0001] The present invention belongs to the field of image processing technology, and in particular relates to an image art style transfer algorithm, a storage medium and a device. Background Art

[0002] Image style transfer, a key research area in computer vision, holds broad application prospects in areas such as artistic creation, visual design, and media entertainment. This technology essentially involves translating images between different visual domains, aiming to transfer the stylistic features of a source image to the target image while preserving the integrity of the content structure. However, this technology faces challenges such as balancing content preservation and style transfer, poor processing quality of local details, and high computational resource requirements.

[0003] For example, existing style transfer generators lack effective feature selection mechanisms during the downsampling phase, making it difficult to accurately capture local salient features. Furthermore, the upsampling phase relies on a simple feature splicing approach, failing to fully utilize multi-scale feature information. Furthermore, during the feature fusion phase, existing technologies, due to these issues, lack sufficient feature information and accuracy. Furthermore, conventional feature fusion methods lack sufficient comprehensiveness and depth to achieve satisfactory results. The transferred images suffer from insufficient local detail and excessive noise, while also failing to identify and highlight certain important features. Therefore, improving the quality and effectiveness of feature fusion while preserving sufficient detail and structural information in the image has become a technical challenge that existing technologies need to address. Summary of the Invention

[0004] The purpose of the present invention is to provide an image art style transfer algorithm to solve the technical problems in the prior art of poor quality and effect of feature fusion and difficulty in repeatedly retaining sufficient details and structural information in the image.

[0005] The image artistic style transfer algorithm includes the following steps.

[0006] Step S1: construct a generative network G. The basic framework of the generative network G includes an encoder and a decoder. The decoder uses a channel-space adaptive fusion module to fuse the upsampled features with the features of the encoder's downsampled module. The channel-space adaptive fusion module includes a channel attention mechanism, a spatial attention mechanism, and an adaptive feature selection stage. The adaptive feature selection stage dynamically integrates and selects the features enhanced by dual attention, and then fuses the features based on the connections between different channel features in the adaptively selected features to finally generate an output result.

[0007] Step S2: construct a discriminant network D. The discriminant network D uses a Markov discriminator and is composed of a series of continuous convolution modules.

[0008] In step S3, the generative network G and the discriminative network D are subjected to adversarial training, and the trained generative network G is used to generate the target style transfer image.

[0009] Preferably, the adaptive feature selection stage accepts the feature map processed by the channel attention mechanism and the spatial attention mechanism as input, establishes associations between channels through 1×1 convolution, and captures the interdependence between features of different channels; then batch normalization is applied to stabilize the training process and accelerate convergence, while reducing internal covariate shift; finally, a nonlinear transformation is introduced through the threshold transfer unit to enhance the model's expressiveness and filter out irrelevant information; the expression for adaptive feature selection is:

[0010]

[0011] in, is the feature after adaptive selection of output channel a, Represents the linear transformation matrix of input mapping to output channel a, b a is the bias term of output channel a, μ a is the channel mean vector of output channel a, β a is the learnable offset parameter of output channel a, is the characteristic variance of output channel a, γ a is a learnable scaling parameter of the output channel a, and ∈ is a numerical stability constant.

[0012] Preferably, the adaptively selected feature F a A linear transformation is performed on the channel dimension through a 1×1 convolution operation to establish the connection between different channel features, and then a batch normalization operation is applied to calculate the mean μ of each channel. B,c and variance The features are normalized to zero mean and unit variance, and then the learned scaling parameter γ is used. c and the offset parameter β c Adjust the distribution to effectively stabilize the training process and accelerate convergence; finally, introduce nonlinear transformation through the threshold transfer unit to enhance the expressive ability of the model and suppress negative responses, filter irrelevant information, and complete the final feature fusion; the linear transformation in the channel dimension is expressed as W f ·F a +b f , where W f is the weight matrix, b f is the bias term; the complete mathematical expression of the final feature fusion is:

[0013]

[0014] Among them, O c,h,w is the value of the final output feature at position (c, h, w), γ c is the scaling parameter of the c-th output channel, W f,c,i is the weight from input channel i to output channel c, F a,i,h,w is the value of the feature at position (i, h, w) after adaptive selection, b f,c is the bias term of output channel c, μ B,c is the statistical mean of the output channel c, is the statistical variance of the output channel c, ∈ is the numerical stability constant, β c is the offset parameter of output channel c.

[0015] Preferably, the processing flow of the channel-space adaptive fusion module includes: first, splicing the upsampled features and the jump connection features in the channel dimension, and then introducing the channel attention mechanism; first, compressing the spatial dimension through adaptive global average pooling to extract channel-level features, and then generating channel attention weights through nonlinear transformations of dimensionality reduction convolution, threshold transfer unit and dimensionality increase convolution; further introducing the spatial attention mechanism, the spatial attention mechanism first calculates the average and maximum values ​​of the feature map in the channel dimension, splices the average and maximum values, and then generates a spatial attention map through convolution and nonlinear activation units, that is, the feature map of the input adaptive feature selection stage.

[0016] Preferably, the encoder adds a parameter-free self-attention module after each downsampling module to enhance feature representation; the process of the parameter-free self-attention module includes: first calculating the feature Figure X The spatial mean ω of , and then calculate the square difference (X-ω) from the mean for each feature position 2 , then calculate the normalized variance ρ of the entire feature map 2 , and further calculate the preliminary attention score based on the square difference of each position, the calculation formula is:

[0017] Where a is the scaling factor, ε is a small constant used to prevent division by zero errors, and b is the offset constant. Finally, the initial attention score y is converted into a weight coefficient between 0 and 1 through the nonlinear activation function ρ(y), and these weights are applied to the original features. The calculation formula is: out =X×ρ(y), where X out is the weighted output feature map, and × is element-wise multiplication.

[0018] Preferably, the encoder is input through an initial module, which is sequentially connected to a reflection filling layer, a convolution layer, a normalization layer and a threshold transfer unit; then two downsampling modules are connected, each downsampling module consists of a convolution layer, a normalization layer and a threshold transfer unit, and a parameter-free self-attention module is added after each downsampling module to enhance the feature representation to form an encoder; the decoder includes two upsampling modules, each upsampling module includes a transposed convolution layer, a normalization layer and a threshold transfer unit connected in sequence; then the channel-space adaptive fusion module is used to fuse the upsampled features with the features of the downsampling module of the encoder; finally, a reflection filling layer, a convolution layer and a hyperbolic tangent nonlinear layer are connected to form a decoder.

[0019] Preferably, in the convolution module of the discriminant network D, the first convolution module includes a convolution layer and a ramp-type rectified linear unit; each intermediate convolution module includes a convolution layer, an instance normalization layer and a ramp-type rectified linear unit, and extracts multi-scale feature representations of the input image through layer-by-layer convolution and downsampling operations; the last convolution module only includes convolution operations to generate the final discriminant feature map.

[0020] The present invention also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the above-mentioned image artistic style transfer algorithm.

[0021] The present invention also provides a computer device, comprising a memory, a processor, and a computer program stored in the memory and capable of running on the processor, wherein when the processor executes the computer program, the steps of the above-mentioned image art style transfer algorithm are implemented.

[0022] The present invention has the following advantages:

[0023] 1. The present invention designs a CSAFM channel-space adaptive fusion module and integrates it into the upsampling stage of the generator, which solves the limitations of simple feature splicing in traditional style transfer. The module adopts the dual mechanisms of channel attention and spatial attention to enhance feature expression from the channel and spatial dimensions respectively, and realizes the intelligent fusion of multi-scale features through an adaptive feature selector. Compared with the existing AFF (adaptive feature fusion) technology that adopts a recursive fusion method, this method integrates CSAFM in the upsampling stage of the generator in terms of architectural design, focusing on the effective fusion of upsampled features and downsampled features, and first performs adaptive selection in the fusion stage, that is, multiple attention mechanisms and feature selection processing, and then improves the final fusion method to achieve a more complex processing flow, enhance the comprehensiveness and depth of feature fusion, and is particularly suitable for detail preservation and structure reconstruction in image generation tasks.

[0024] 2. This paper also improves upon this approach by introducing the SimAM parameter-free attention module. This module generates attention weights during the generator downsampling phase by calculating the local self-similarity of feature maps, without the need for additional parameters. Compared to traditional channel-wise attention mechanisms, SimAM operates directly in the raw feature space, more effectively capturing the spatial correlation of local features and significantly enhancing the network's ability to perceive locally salient features.

[0025] 3. This paper constructs a multi-level perceptual loss based on the VGG16 network, which overcomes the shortcomings of the original loss function that relies solely on adversarial loss and cycle consistency loss. This multi-level perceptual loss mechanism achieves hierarchical supervision from low-level visual features to high-level semantics by matching shallow, mid-level, and deep-level features to constrain texture details, local structure, and high-level semantic information respectively. The loss function design provides the generator with richer gradient information, enabling the model to simultaneously focus on local details and global semantics, effectively improving the quality and naturalness of style transfer, and achieving a more subtle style transfer effect while maintaining the integrity of the image structure. BRIEF DESCRIPTION OF THE DRAWINGS

[0026] Figure 1 This is the basic flow chart of the image art style transfer algorithm in the present invention.

[0027] Figure 2 It is a structural diagram of the overall network in the present invention.

[0028] Figure 3 This is a structural diagram of the generation network G in the present invention.

[0029] Figure 4 This is a structural diagram of the CSAFM module in the present invention.

[0030] Figure 5 This is the structural diagram of the discriminant network D in the present invention.

[0031] Figure 6 This is a comparison chart of the defect areas marked in the results generated by the present invention and the existing style transfer model.

[0032] Figure 7 A comparison chart of the results generated by the present invention and the existing style transfer model.

[0033] Figures 8-12 The following are the effect diagrams of the generation results of the present invention and the existing style transfer model on the five indicators of FID, SSIM, LPIPS, PSNR and MSE. DETAILED DESCRIPTION

[0034] The following is a further detailed description of the specific implementation methods of the present invention through the description of the embodiments with reference to the accompanying drawings, so as to help those skilled in the art to have a more complete, accurate and in-depth understanding of the inventive concept and technical solution of the present invention.

[0035] Example 1.

[0036] like Figure 1-Figure 5 As shown, the present invention provides an image art style transfer algorithm, which includes the following steps.

[0037] Step S1: construct a generative network G. The basic framework of the generative network G includes an encoder and a decoder.

[0038] The basic framework primarily consists of functional modules such as a standard convolutional block, a downsampling module, a residual module, and an upsampling module. Specifically, the encoder is fed through an initialization module, which in turn is connected to a reflection padding layer, a 7×7 convolutional layer, a normalization layer, and a threshold transfer unit. This initialization module is then connected to two downsampling modules, each consisting of a 3×3 convolutional layer, a normalization layer, and a threshold transfer unit. Compared to existing technologies, the present invention adds a parameter-free self-attention module (SimAM) after each downsampling module to enhance feature representation, forming the encoder.

[0039] The decoder mainly includes two upsampling modules, each of which is sequentially connected to a transposed convolution layer, a normalization layer and a threshold transfer unit; compared with the existing technology, the present invention utilizes a channel-space adaptive fusion module (CSAFM) to fuse the upsampled features with the downsampled features of the encoder, and finally connects a reflection padding layer, a 7×7 convolution layer and a hyperbolic tangent nonlinear layer to form the decoder.

[0040] 1) The process of SimAM processing includes: first calculating the feature map The spatial mean of :

[0041] Where B is the batch size, which indicates the number of samples in a batch of data; C is the number of channels, which indicates the depth of the feature map; H is the feature map height, which indicates the number of pixels in the vertical direction; W is the feature map width, which indicates the number of pixels in the horizontal direction; X is the number of channels in the feature map; i,j is the eigenvalue at position (i, j).

[0042] Then calculate the squared difference from the mean for each feature position (X-ω) 2 , then calculate the normalized variance of the entire feature map:

[0043]

[0044] Where, X i,j-ω is the square difference between the eigenvalue and the mean at position (i, j), and H×W-1 is the degree of freedom, which is used for statistical unbiased variance calculation.

[0045] Furthermore, the squared difference at each position is divided by four times the normalized variance plus a small constant ε, and then added with 0.5 to obtain the preliminary attention score, which is calculated as follows:

[0046]

[0047] Wherein, a is a scaling factor, which is taken as 4 in the embodiment to control the impact of variance differences, ε is a small constant used for numerical stability to prevent division by zero errors, and b is an offset constant, which is taken as 0.5 in the embodiment to ensure the baseline value of the attention score.

[0048] Finally, the initial attention score y is converted into a weight coefficient between 0 and 1 through the nonlinear activation function ρ(y), and these weights are applied to the original features. The calculation formula is: out =X×ρ(y), where X out is the weighted output feature map, and × is element-wise multiplication.

[0049] This mechanism does not require additional training parameters. By adaptively adjusting feature weights, it effectively enhances feature discrimination and receptive field, improves the network's ability to capture key information, and solves the problem of information loss in the traditional downsampling process.

[0050] 2) CSAFM includes channel attention mechanism, spatial attention mechanism and adaptive feature selection stage. The processing flow of CSAFM includes: first, concatenating the upsampled features and the skip connection features in the channel dimension to obtain the feature concatenation result F concat , the calculation formula is: Where, F up is the upsampled feature, F down is the downsampled feature, C = C1 + C2 is the total number of channels after splicing, H is the feature map height, which indicates the number of pixels in the vertical direction, W is the feature map width, which indicates the number of pixels in the horizontal direction, and [·,·] is the splicing operation on the channel dimension.

[0051] The channel attention mechanism is then introduced. The core calculation process of the channel attention mechanism is divided into two key steps: first, the spatial dimension is compressed through adaptive global average pooling to extract channel-level features, and then the channel attention weights are generated through nonlinear transformations such as dimensionality reduction convolution, threshold transfer unit, and dimensionality increase convolution. This mechanism can significantly reduce the number of parameters while capturing the importance relationship between channels, achieving efficient channel feature enhancement. The features after channel attention enhancement are calculated using the following formula:

[0052]

[0053] Where, F c ∈R C×H×W is the feature after channel attention enhancement, ⊙ is the Hadamard product (i.e., element-wise multiplication), σ is the Sigmoid function; and is the dimension reduction transformation matrix, r is the dimension reduction ratio, is the dimension-raising transformation matrix, max(0,x) is the ReLU function, which returns when x>0, otherwise it returns 0. is the global average pooling operation.

[0054] Building on the enhanced channel attention feature, CSAFM further introduces a spatial attention mechanism to learn the spatial importance distribution of feature maps. Specifically, the spatial attention mechanism first calculates the average and maximum values ​​of feature maps along the channel dimension, concatenates them, and generates a spatial attention map through a 7×7 convolution and nonlinear activation unit.

[0055] The features after spatial attention enhancement are calculated using the following formula:

[0056]

[0057] Where, F s ∈R C×H×W is the feature after spatial attention enhancement, f 7×7 ∈R 1×2×7×7 It is a convolution transform that receives 2 channels of input and outputs 1 channel. is to calculate the average value along the channel dimension, It calculates the maximum value along the channel dimension, and (;) represents the feature concatenation operation between two features.

[0058] The adaptive feature selection stage dynamically integrates and selects the features enhanced by dual attention. This stage receives the feature map processed by channel-space dual attention as input, establishes associations between channels through 1×1 convolution, and captures the interdependence between features of different channels. Batch normalization is then applied to stabilize the training process and accelerate convergence while reducing internal covariate shift. Finally, a nonlinear transformation is introduced through the threshold transfer unit to enhance the model's expressiveness and filter out irrelevant information. The expression for adaptive feature selection is:

[0059]

[0060] in, is the feature after adaptive selection of output channel a, Represents the linear transformation matrix of input mapping to output channel a, b a is the bias term of output channel a, μ ais the channel mean vector of output channel a, β a is the learnable offset parameter of output channel a, is the characteristic variance of output channel a, γ a is a learnable scaling parameter of the output channel a, and ∈ is a numerical stability constant.

[0061] To receive the feature map F processed by the adaptive feature selection stage a As input, a linear transformation is performed on the channel dimension through a 1×1 convolution operation to establish a connection between different channel features. This transformation can be expressed as W f ·F a +b f , where W f is the weight matrix, b f is the bias term; the batch normalization operation is then applied to calculate the mean μ of each channel B,c and variance The features are normalized to zero mean and unit variance, and then the learned scaling parameter γ is used. c and the offset parameter β c Adjust the distribution to effectively stabilize the training process and accelerate convergence; finally, introduce nonlinear transformation through the threshold transfer unit to enhance the model's expressiveness and suppress negative responses, filter out irrelevant information, and complete the final feature fusion. The complete mathematical expression of the final feature fusion is:

[0062]

[0063] Among them, O c,h,w is the value of the final output feature at position (c, h, w), γ c is the scaling parameter of the c-th output channel, W f,c,i is the weight from input channel i to output channel c, F a,i,h,w is the value of the feature at position (i, h, w) after adaptive selection, b f,c is the bias term of output channel c, μ B,c is the statistical mean of the output channel c, is the statistical variance of the output channel c, ∈ is the numerical stability constant, β c is the offset parameter of output channel c.

[0064] The generative network described above utilizes a multi-stage process: feature concatenation → channel attention → spatial attention → adaptive selection → final fusion. In comparison, existing AFF (Adaptive Feature Fusion) techniques employ a recursive fusion approach. The generative network employed in this method, however, features an architectural design that integrates CSAFM in the generator's upsampling phase, focusing on the effective fusion of upsampled and downsampled features. Adaptive selection is then performed in the fusion phase, utilizing multiple attention mechanisms and feature selection techniques before improving the final fusion approach. This results in a more complex process flow, enhancing the comprehensiveness and depth of feature fusion, making it particularly suitable for detail preservation and structural reconstruction in image generation tasks.

[0065] Step S2: construct a discriminant network D. The discriminant network D adopts a Markov discriminator and is composed of a series of continuous convolution modules.

[0066] In this step, the discriminator network D provides an accurate adversarial training signal for the generator by distinguishing between real images and images synthesized by the generator. The first module contains a convolutional layer and a ramp-type rectified linear unit; the middle module contains a convolutional layer, an instance normalization layer, and a ramp-type rectified linear unit, extracting multi-scale feature representations of the input image through layer-by-layer convolution and downsampling operations; the last module contains only convolution operations to generate the final discriminant feature map.

[0067] The discriminator is designed as a fully convolutional network. Each point in the final output feature map corresponds to a receptive field of approximately 70×70 pixels in the original image, enabling independent discrimination of local image regions. This patch-based discrimination strategy not only overcomes the limitations of traditional discriminators that evaluate the entire image as a whole, reducing the number of parameters and computational complexity, but also enables more accurate perception and assessment of detailed features and texture information in local regions, helping the generator learn better local structure and texture details.

[0068] In step S3, the generative network G and the discriminative network D are subjected to adversarial training, and the trained generative network G is used to generate the target style transfer image.

[0069] In this step, the original style transfer model mainly relies on adversarial loss and cycle consistency loss to achieve image style conversion, but this design has limitations in feature expression and semantic understanding: the adversarial loss only provides supervision in the pixel space and it is difficult to ensure the authenticity of high-level features; the cycle consistency loss is based on a simple L1 distance metric and cannot effectively express hierarchical semantic information.

[0070] To address these issues, this paper introduces a multi-level perceptual loss mechanism based on pre-trained VGG16 to build a feature matching framework from low-level to high-level. The details are as follows:

[0071] (1) Shallow feature matching is responsible for capturing local visual features such as texture and edges to ensure the authenticity of the generated image at the detail level;

[0072] (2) Mid-level feature matching focuses on the medium-scale structure and local semantic information of the image to improve the coherence of the content;

[0073] (3) High-level feature matching extracts high-level semantic representations to ensure the consistency of the generated image at the overall semantic level.

[0074] Multi-level perceptual loss provides the generator with rich gradient information, enabling it to focus on both local details and global semantics, effectively making up for the deficiency of relying solely on pixel-level loss to maintain the consistency of deep features, thereby significantly improving the quality and naturalness of style transfer.

[0075] The loss function of the present invention consists of four parts: adversarial loss, cycle consistency loss, identity loss, and perceptual loss.

[0076] The first is the adversarial loss, which is used to ensure that the generated image looks real and conforms to the feature distribution of the target domain. The adversarial loss can be expressed as:

[0077]

[0078] In order to ensure the bidirectional consistency of image conversion and prevent mode collapse, the cycle consistency loss (Cycle-Consistency Loss) is introduced:

[0079]

[0080] The identity loss is used to maintain the color consistency of the source domain image:

[0081]

[0082] Perceptual Loss is obtained by calculating the weighted accumulation of mean square error between the feature maps of three different levels of the VGG16 network. It is used to measure the difference between the generated image and the real image in high-level semantic features. Its formula is expressed as:

[0083]

[0084] Among them, Φ i (x) represents the feature extraction function, which extracts image features from different abstraction levels. x represents the real image, G(y) represents the image generated by the generator, and λ is the weight coefficient of each feature level. represents the L2 norm (mean square error).

[0085] Therefore, the improved total loss function can be expressed as:

[0086]

[0087] Among them, λ GAN ,λ cyc ,λ identity ,λ perceptual In order The combined design of multiple loss functions improves the overall performance of the model by optimizing complementary objective functions, ensuring that the generated images not only retain realistic texture details but also maintain consistency and rationality in semantic features, providing a more comprehensive quality assurance mechanism for image generation tasks.

[0088] The following describes an image style transfer algorithm, storage medium, and device process using specific experiments.

[0089] like Figure 6 As shown in the figure, from left to right are the input image, CycleGAN algorithm, DualGAN algorithm, DiscoGAN algorithm, CUT algorithm and the algorithm of the present invention (hereinafter referred to as AFST-GAN).

[0090] Without the adaptive module, the traditional feature fusion method has the following defects:

[0091] 1. Loss of local details: As shown in the red box, other methods such as CycleGAN, DualGAN, and DiscoGAN cannot well preserve important details in the converted image.

[0092] 2. Noise and Artifacts: Contrastive methods introduce unwanted noise and visual artifacts during processing, which are particularly noticeable in texture-rich areas such as forest areas and rock surfaces.

[0093] 3. Insufficient feature information recognition: Traditional methods have difficulty in identifying and highlighting important feature information, resulting in the lack of visual key elements in the converted image.

[0094] like Figure 7-12 As shown in the figure, the description experiment of the semantic descriptor in similar scenarios is verified. In order to verify the performance of the proposed AFST-GAN model, objective and subjective evaluation schemes are designed.

[0095] In the objective evaluation phase, four representative benchmark models were selected for comparison: CycleGAN, DualGAN, DiscoGAN, and CUT. The images generated by each model were evaluated using five metrics: SSIM (Structural similarity index measure), PSNR (Peak signal-to-noise ratio), MSE (Mean squared error), LPIPS (Learned perceptual image patch similarity), and FID (Frechet inception distance). Figures 8-12 The results show that the proposed method outperforms the comparison methods in terms of overall performance. In terms of quantitative analysis of objective evaluation indicators, AFST-GAN achieved leading performance on the summer2winter style transfer dataset: the FID value was as low as 51.63, which was 21.32%, 45.30%, 40.32% and 9.37% lower than CycleGAN, DualGAN, DiscoGAN and CUT respectively, indicating that the quality and authenticity of the generated images were significantly improved; the SSIM value reached 0.90, which was 7.14%, 20.00%, 28.57% and 36.36% higher than the other four methods respectively, proving that The structural similarity is significantly improved; the PNSR value is as high as 25.03, which is 5.43%, 47.58%, 74.18% and 29.35% better than the comparison methods, respectively, reflecting the improvement of image reconstruction quality; the LPIPS value is only 0.12, and the lower perceptual distance is reduced by 25.00%, 64.71%, 52.00% and 57.14%, respectively, indicating that the generated image has a higher perceptual consistency with the target style; the MSE value is reduced to 69.02, which is reduced by 8.51%, 22.18%, 32.39% and 22.14%, respectively, proving the effective reduction of pixel-level error.

[0096] Based on this, AFST-GAN achieved optimal performance across all five evaluation metrics, with particularly significant advantages in FID and LPIPS, key metrics for perceptual quality. These objective data fully validate the proposed AFST-GAN algorithm's ability to not only maintain the image's structural integrity and detailed information, but also achieve a more natural and harmonious style transfer effect, comprehensively demonstrating the effectiveness and advancement of the improved method from multiple dimensions.

[0097] In subjective qualitative analysis, Figure 7The results show that CycleGAN suffers from significant texture distortion and detail loss in landscape image transfer, especially when dealing with complex terrain, where the generated images often appear blurry and artifact-laden. Furthermore, CycleGAN exhibits instability in preserving the original scene structure, sometimes producing unnatural color transitions.

[0098] DualGAN exhibits serious color issues during image style transfer, with some images exhibiting noticeable purple and gold tonal shifts. This unnatural color shift significantly reduces the image's realism. Furthermore, the ability to preserve detail in shadow areas is insufficient, resulting in a weakened sense of depth in the image.

[0099] While DiscoGAN can achieve basic style transfer, it has significant shortcomings in maintaining the integrity of image structure. This is particularly true when processing complex scenes, where structural distortions are a common problem. Furthermore, the model performs poorly when processing local image details, with the generated images often exhibiting overly smoothed features, resulting in a significant loss of detail and reduced visual quality.

[0100] The winter images generated by CUT retain a good deal of their original structure and exhibit a unique style in lighting processing. However, some images appear overly bright, and the winter characteristics are less distinct. It's worth noting that CUT performs well in the first and fourth columns of images, successfully transforming summer landscapes into realistic winter scenes, but its overall consistency is less stable than that of other models.

[0101] AFST-GAN demonstrates better overall performance: it excels in maintaining the structural integrity of the original scene, especially when dealing with complex terrain, and is able to better preserve the terrain features; it also better preserves image texture details; its color transfer is more natural and harmonious, avoiding the color distortion problems common in other algorithms; and it shows greater adaptability when dealing with lighting changes, and can better balance the details in light and dark areas, making the image closer to the real winter scenery.

[0102] Example 2.

[0103] Corresponding to the first embodiment of the present invention, the second embodiment of the present invention provides a computer-readable storage medium having a computer program stored thereon. When the program is executed by a processor, the following steps are implemented according to the method of the first embodiment.

[0104] Step S1: construct a generative network G. The basic framework of the generative network G includes an encoder and a decoder.

[0105] Step S2: construct a discriminant network D. The discriminant network D adopts a Markov discriminator and is composed of a series of continuous convolution modules.

[0106] In step S3, the generative network G and the discriminative network D are subjected to adversarial training, and the trained generative network G is used to generate the target style transfer image.

[0107] The above-mentioned storage media include: U disk, mobile hard disk, read-only memory (ROM), random access memory (RAM), optical disk and other media that can store program codes.

[0108] The specific limitations on the steps implemented after the program in the computer-readable storage medium is executed can be found in Example 1, and will not be described in detail here.

[0109] Example 3.

[0110] Corresponding to the first embodiment of the present invention, the third embodiment of the present invention provides a computer device, including a memory, a processor, and a computer program stored in the memory and capable of running on the processor. When the processor executes the computer program, the following steps are implemented according to the method of the first embodiment.

[0111] Step S1: construct a generative network G. The basic framework of the generative network G includes an encoder and a decoder.

[0112] Step S2: construct a discriminant network D. The discriminant network D adopts a Markov discriminator and is composed of a series of continuous convolution modules.

[0113] In step S3, the generative network G and the discriminative network D are subjected to adversarial training, and the trained generative network G is used to generate the target style transfer image.

[0114] The specific limitations on the above steps of implementing the computer device can be found in Example 1 and will not be described in detail here.

[0115] It should be noted that each box in the block diagrams and / or flow charts in the accompanying drawings of the present invention, as well as the combination of boxes in the block diagrams and / or flow charts, can be implemented by a dedicated hardware-based system that performs the specified functions or actions, or can be implemented by a combination of dedicated hardware and machine instructions.

[0116] The present invention is described above by way of example in conjunction with the accompanying drawings. It is obvious that the specific implementation of the present invention is not limited to the above-mentioned method. As long as various non-substantial improvements are made using the inventive concept and technical solution of the present invention, or the inventive concept and technical solution are directly applied to other occasions without improvement, they are all within the scope of protection of the present invention.

Claims

1. An image artistic style transfer algorithm, characterized by: The following steps are involved: Step S1: construct a generative network G. The basic framework of the generative network G includes an encoder and a decoder. The decoder uses a channel-space adaptive fusion module to fuse the upsampled features with the features of the encoder's downsampled module. The channel-space adaptive fusion module includes a channel attention mechanism, a spatial attention mechanism, and an adaptive feature selection stage. The adaptive feature selection stage dynamically integrates and selects the features enhanced by dual attention, then fuses them based on the connections between different channel features in the adaptively selected features, and finally generates the output results; Step S2: construct a discriminant network D. The discriminant network D uses a Markov discriminator and is composed of a series of continuous convolution modules. Step S3: Perform adversarial training on the generative network G and the discriminative network D. The trained generative network G is used to generate the target style transfer image. The adaptive feature selection stage accepts feature maps processed by the channel-wise and spatial-attention mechanisms as input, establishes associations between channels through 1×1 convolution, and captures the interdependencies between features from different channels. Batch normalization is then applied to stabilize the training process and accelerate convergence, while reducing internal covariate shift. Finally, a threshold transfer unit introduces nonlinear transformations to enhance the model's expressiveness and filter out irrelevant information. The expression of adaptive feature selection is: Among them, F a ∈R C×H×W is the feature after adaptive selection of output channel a, W a ∈R C×C Represents the linear transformation matrix of input mapping to output channel a, b a is the bias term of output channel a, μ a is the channel mean vector of output channel a, β a is the learnable offset parameter of output channel a, is the characteristic variance of output channel a, γ a is a learnable scaling parameter of the output channel a, ò is a numerical stability constant; F s It is the feature output after the up-sampled features and down-sampled features are concatenated and fused, and processed by the channel attention mechanism and the spatial attention mechanism in sequence; Feature F after adaptive selection a A linear transformation is performed on the channel dimension through a 1×1 convolution operation to establish the connection between different channel features, and then a batch normalization operation is applied to calculate the mean μ of each channel. B,c and variance The features are normalized to zero mean and unit variance, and then the learned scaling parameter γ is used. c and the offset parameter β c Adjust the distribution to effectively stabilize the training process and accelerate convergence; finally, introduce nonlinear transformation through the threshold transfer unit to enhance the expressive ability of the model and suppress negative responses, filter irrelevant information, and complete the final feature fusion; the linear transformation in the channel dimension is expressed as W f ·F a +b f , where W f is the weight matrix, b f is the bias term; the complete mathematical expression of the final feature fusion is: Among them, O c,h,w is the value of the final output feature at position (c, h, w), γ c is the scaling parameter of the c-th output channel, W f,c,i is the weight from input channel i to output channel c, F a,i,h,w is the value of the feature at position (i, h, w) after adaptive selection, b f,c is the bias term of output channel c, μ B,c is the statistical mean of the output channel c, is the statistical variance of the output channel c, is the numerical stability constant, β c is the offset parameter of output channel c.

2. The image artistic style transfer algorithm according to claim 1, characterized in that: The processing flow of the channel-space adaptive fusion module includes: first, the upsampled features and jump connection features are spliced ​​in the channel dimension, and then the channel attention mechanism is introduced; first, the spatial dimension is compressed through adaptive global average pooling to extract channel-level features, and then the channel attention weights are generated through nonlinear transformations of dimensionality reduction convolution, threshold transfer unit and dimensionality increase convolution; further, the spatial attention mechanism is introduced. The spatial attention mechanism first calculates the average and maximum values ​​of the feature map in the channel dimension, splices the average and maximum values, and then generates a spatial attention map through convolution and nonlinear activation units, which is the feature map input into the adaptive feature selection stage.

3. The image artistic style transfer algorithm according to claim 1, characterized in that: The encoder adds a parameter-free self-attention module after each downsampling module to enhance feature representation; The process of the parameter-free self-attention module includes: first calculating the spatial mean ω of the feature map X, and then calculating the squared difference (X-ω) from the mean for each feature position 2 , then calculate the normalized variance ρ of the entire feature map 2 , and further calculate the preliminary attention score based on the square difference of each position, the calculation formula is: Where a is the scaling factor, ε is a small constant used to prevent division by zero errors, and b is the offset constant. Finally, the initial attention score y is converted into a weight coefficient between 0 and 1 through the nonlinear activation function ρ(y), and these weights are applied to the original features. The calculation formula is: out =X×ρ(y), where X out is the weighted output feature map, and × is element-wise multiplication.

4. The image artistic style transfer algorithm according to claim 1, characterized in that: The encoder is input through an initial module, which is sequentially connected to a reflection padding layer, a convolution layer, a normalization layer and a threshold transfer unit; then two downsampling modules are connected, each downsampling module consists of a convolution layer, a normalization layer and a threshold transfer unit, and a parameter-free self-attention module is added after each downsampling module to enhance the feature representation to form an encoder; the decoder includes two upsampling modules, each upsampling module includes a transposed convolution layer, a normalization layer and a threshold transfer unit connected in sequence; then the channel-space adaptive fusion module is used to fuse the upsampled features with the features of the encoder's downsampling module; finally, a reflection padding layer, a convolution layer and a hyperbolic tangent nonlinear layer are connected to form a decoder.

5. The image artistic style transfer algorithm according to claim 1, characterized in that: In the convolutional module of the discriminant network D, the first convolutional module contains a convolutional layer and a ramp-type rectified linear unit; the intermediate convolutional modules contain convolutional layers, instance normalization layers, and ramp-type rectified linear units, which extract multi-scale feature representations of the input image through layer-by-layer convolution and downsampling operations; the last convolutional module only contains convolution operations to generate the final discriminant feature map.

6. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the steps of an image artistic style transfer algorithm as described in any one of claims 1 to 5 are implemented.

7. A computer device comprising a memory, a processor, and a computer program stored in the memory and capable of running on the processor, characterized in that: When the processor executes the computer program, the steps of the image artistic style transfer algorithm as described in any one of claims 1 to 5 are implemented.

Images