Dual-coding adversarial learning and multi-scale expansion fusion attention image cartoonalization method

Abstract

The invention discloses a dual-coding adversarial learning and multi-scale expansion fusion attention image cartoonalization method, and relates to the technical field of image processing, and the method comprises the steps: constructing a generator comprising a dual-encoder structure, and enabling a content feature encoder to extract the structural semantic features of an input real image, a style feature encoder extracts texture style features of the target cartoon image; the style features are injected into the content features through adaptive instance normalization; processing fusion features by adopting a multi-scale expansion fusion attention mechanism, and adaptively enhancing a key channel and a space region; the decoder reconstructs and generates a cartoon image; meanwhile, a double-discriminator framework composed of a global discriminator and a local discriminator is adopted, the overall style consistency and the local detail authenticity of the image are evaluated respectively, and the generation process is optimized through adversarial learning. According to the method, the high-quality cartoon graph can be generated, the color texture is natural, the edge is clear, and details are rich.

Description

Dual-encoder adversarial learning and multi-scale dilation fusion attention image cartoonization method Technical Field

[0001] This invention relates to the field of image processing technology, and in particular to a method for cartoonizing attention-based images using dual-encoding adversarial learning and multi-scale dilation fusion. Background Technology

[0002] Image cartoonization aims to transform real-world images into cartoon-style images. Its core lies in modifying the color and texture features of the original image while preserving the main content. Automating image cartoonization using computers significantly improves efficiency compared to manual processing, making it particularly suitable for large-scale image processing and batch cartoonization applications.

[0003] Currently, style transfer methods based on model iterative optimization are the mainstream technical approach in the field of image cartoonization. However, these methods still face several significant technical challenges in practical applications: First, the lack of a mechanism for identifying important features. Existing methods generally lack an effective mechanism to accurately determine the importance level of different regions or features in an image. This leads to the situation where, during the conversion process, non-critical fine textures or background noise within the image are easily misjudged as important information and excessively preserved or enhanced, resulting in visual confusion in the generated image; at the same time, truly crucial image landmarks or structural information may be improperly ignored or weakened, resulting in the loss of expression of key visual information.

[0004] Secondly, improving edge sharpness is difficult. Cartoon-style images typically require clear and sharp edge contours. However, in training frameworks based on adversarial loss, the importance of edge features is easily masked by more prominent features such as color and texture, leading to insufficient attention to edges during the optimization process. Furthermore, existing edge enhancement techniques are generally sensitive to image noise, easily misclassifying real edges that should be preserved as noise and suppressing or smoothing them, or failing to effectively detect complete and coherent edge structures in complex scenes. Ultimately, this results in generated cartoon images with blurry edges and insufficient detail.

[0005] Third, pixel-level loss functions have perceptual limitations. Traditional methods often use loss functions that directly compare the pixel values ​​of the generated and target images. Such pixel-level matching cannot fully simulate the differences in human perception of image content, structure, and style. Its limitation is that excessive pursuit of pixel-level similarity may lead to an overly smooth overall generated image, losing necessary details, textures, and layers, and failing to meet the specific visual expressiveness requirements of cartoon styles.

[0006] To address these issues, it is urgent to innovate at the algorithmic level and explore new methods that can accurately determine the importance of features, effectively enhance edge structures, and conform to human perception characteristics. Summary of the Invention

[0007] In view of this, the present invention proposes a dual-encoding adversarial learning and multi-scale dilation fusion attention image cartoonization method to solve the problems of indistinct cartoonization style and blurred edges in the existing technology.

[0008] The specific technical solution of this invention is as follows: a dual-encoder adversarial learning and multi-scale dilatational fusion attention image cartoonization method, comprising: constructing a generator including a dual-encoder structure, wherein the content feature encoder extracts the structural semantic features of the input real image, and the style feature encoder extracts the texture style features of the target cartoon image; injecting style features into content features through adaptive instance normalization; processing fusion features using a multi-scale dilatational fusion attention mechanism to adaptively enhance key channels and spatial regions; reconstructing and generating cartoon images using a decoder; and simultaneously employing a dual-discriminator architecture consisting of a global discriminator and a local discriminator to evaluate the overall style consistency and local detail realism of the image, respectively, and optimizing the generation process through adversarial learning.

[0009] Specifically, both the content feature encoder and the style feature encoder include a convolutional layer module, a residual block stacking module, and a multi-scale dilation fusion attention module in sequence. The residual block stacking module learns deep features by cascading residual blocks, and the multi-scale dilation fusion attention module adaptively weights the feature map through channel attention mechanism and multi-scale spatial attention mechanism.

[0010] Specifically, the mean and variance statistics of style features are extracted, and the affine transformation parameters of content features are dynamically adjusted to align the content feature distribution with the target style.

[0011] Specifically, the generator's loss function is composed of a weighted sum of adversarial loss, content loss, perceptual color loss, and edge loss. The adversarial loss is jointly calculated by the global discriminator loss and the local discriminator loss. The content loss is calculated by extracting high-level semantic feature similarity through a pre-trained network. The perceptual color loss constrains the consistency of color distribution in the feature space. The edge loss is calculated based on the multi-scale edge residuals of the Laplacian pyramid.

[0012] Specifically, the edge loss calculation process is as follows: construct Laplacian pyramids for the generated image and the target image respectively, extract high-frequency edge residual maps layer by layer, and sum the difference values ​​of the residuals of each layer by weight to enhance the contour sharpness of the generated image.

[0013] Specifically, the global discriminator uses a convolutional neural network to determine the degree of matching between the overall style of the generated image and the target cartoon domain; the local discriminator evaluates the realism of details, textures and edge structures by cropping local regions of the image, and the loss functions of the two discriminators work together to optimize the generator.

[0014] Specifically, the channel attention mechanism execution process includes: performing global average pooling and max pooling on the input feature map in the channel dimension respectively, learning channel weights through a multilayer perceptron, and highlighting key feature channels in a weighted manner.

[0015] Specifically, the multi-scale spatial attention mechanism execution process includes: performing average pooling and max pooling on the input feature map along the channel dimension, connecting multiple dilated convolutional layers with different dilation rates in parallel to extract multi-scale spatial features, fusing them to generate a spatial attention weight map, and enhancing the feature response at key spatial locations.

[0016] Specifically, the dilation rate combination of the hollow convolutional layer covers the local to mesoscale receptive field, which is used to balance the extraction intensity of detailed features and structural features.

[0017] Specifically, the Laplacian pyramid edge extraction process is as follows: the image is downsampled at multiple scales, the high-frequency residual components of each scale layer are calculated, and the multi-scale edge feature map is obtained by weighted superposition to avoid edge blurring or noise interference at a single scale.

[0018] The beneficial effects of this invention are as follows: (1) The content feature encoder and the style feature encoder process the structural and style information respectively, reducing the interference of feature mixing, thereby preserving the semantic contour and contour details of the input image and improving the accuracy of the cartoonization result; (2) Combining channel attention and spatial attention, the key regions and channels are adaptively focused, and the dilated convolution expands the receptive field coverage, reducing the redundancy of irrelevant information, improving the texture richness and color naturalness of the generated image, and accelerating the model convergence; (3) The Laplacian pyramid extracts the edge residual and calculates the edge loss, solving the problems of smoothing and blurring of the generated image and strengthening the sharpness of the boundary lines; (4) The adversarial loss, content loss, perceptual color loss and edge loss are weighted and coordinated to ensure the overall style consistency, local detail authenticity, content structure similarity and color stability of the image, and improve the visual coordination of the cartoon image; (5) The global discriminator maintains the style uniformity, and the local discriminator strengthens the detail authenticity. The two work together to optimize the generator output and achieve stable generation of high-quality cartoon images. Attached Figure Description

[0019] To more clearly illustrate the technical solutions in the embodiments of this application, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0020] Figure 1 is a comparison of different cartoon styles of the present invention; Figure 2 is an overall architecture diagram of the MSDFA-GAN model of the present invention; Figure 3 is a schematic diagram of conventional convolution and dilated convolution of the present invention; Figure 4 is a model diagram of the MSDFA attention mechanism of the present invention; Figure 5 is a comparison of different models of the present invention; Figure 6 is a comparison of ablation experiments of the present invention; Figure 7 is a comparison of images with different dilation rates of the present invention. Detailed Implementation

[0021] To make the technical problems to be solved, the technical solutions, and the beneficial effects of the present invention clearer, the present invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the present invention and are not intended to limit the present invention.

[0022] This invention proposes a dual-encoding adversarial learning and multi-scale dilation fusion attention image cartoonization method. The core of this method lies in constructing a generative adversarial network model called MSDFA-GAN (Multi-Scale Dilated Fusion Attention Generative Adversarial Network). The core objective of this model is to learn a mapping from the real image domain to the target cartoon image domain without requiring paired data, thereby generating high-quality cartoon-style images with clear structure and rich, natural textures and colors.

[0023] The image generation process is as follows: Given a real image P={P1,P2,P3...Pn} and a cartoon image C={C1,C2,C3...Cn}, the images are first resized and cropped, then normalized. The second step involves inputting the resized images into a dual encoder to extract content and style features. Style vectors are injected into the content features to couple style and content. These are then input into the decoder to reconstruct the cartoon-style image S={S1,S2...Sn} layer by layer. The model employs a dual discriminator architecture, including a global discriminator (D...). global ) and local discriminator (D local The global discriminator ensures overall stylistic consistency, focusing on global texture and color matching; the local discriminator focuses on small-scale details, such as textures and edges. Through the joint optimization of the two discriminators, the generator can produce high-quality cartoon images that are both consistent with the overall style and possess detail. Figure 1 shows a comparison of different styles. The two images in the left column are real images, and the three columns on the right are images in three anime styles: *The Wind Rises*, *The Garden of Words*, and *Dragon Ball*, as well as images generated by cartoonizing real images.

[0024] Figure 2 shows the overall architecture of the MSDFA-GAN model of this application. The MSDFA-GAN model mainly consists of a generator and a discriminator, with the discriminator employing a dual-discriminator architecture. The generator is responsible for converting the input real image into the target cartoon-style image, while the discriminator is responsible for distinguishing between the generated cartoon image and the real cartoon training image, and guides the improvement of the generator through an adversarial optimization process.

[0025] The generator structure described is the key innovation of this invention, employing a dual-encoder structure consisting of a content feature encoder and a style feature encoder. Unlike a single-encoder structure, the dual encoder can independently extract the content features (mainly including semantic information such as the image's structure and contours) and style features (mainly including style information such as the image's color and texture) of the input image, effectively avoiding the problem of content loss or distortion during style transfer caused by feature mixing. The content feature encoder and the style feature encoder have identical network structure designs, each consisting of three main parts. The first part is a convolutional layer module, which contains several layers of two-dimensional convolutional operations (the specific number of layers and the size of the convolutional kernel can be set according to the actual image size and feature dimension requirements; the initial layer usually uses a larger convolutional kernel, such as 7x7, for downsampling and preliminary feature extraction, while subsequent layers use a smaller 3x3 convolutional kernel). Each convolutional layer is typically followed by instance normalization and activation functions for processing. The second part is the residual block stacking module, which consists of multiple cascaded residual blocks (e.g., including 6 or more standard residual blocks). Each residual block typically contains two convolution, normalization, and activation operations, and achieves identity mapping through shortcut connections. This is used to learn deep image features and alleviate the gradient vanishing problem. Figure 3 shows a schematic diagram of regular convolution and dilated convolution. The third part is the Multi-Scale Dilated Fusion Attention Module (MSDFA Module). Figure 4 shows the MSDFA attention mechanism model. This module is placed after the residual block stacking module and is used to refine the extracted deep features, adaptively emphasizing important regions and key feature channels in the image while suppressing redundant or interfering information.

[0026] The multi-scale dilatational fusion attention module consists of two sequentially executed sub-modules: a channel attention module and a multi-scale spatial attention module. The channel attention module learns the importance weights of different feature channels, enabling the model to focus on channels with richer information.

[0027] The specific calculation process is as follows: Weights are calculated in the channel attention module using global average pooling and max pooling on the input feature map using formulas (1) and (2). ; In order to simulate the nonlinear feature combination and the relationship between channels, channel features are extracted using an MLP (Multilayer Perceptron) network through formulas (3) and (4).

[0028] ; In the formula, Let C represent the feature vector of the i,j spatial position in all channels, with dimension C. It consists of two fully connected layers, where σ represents the Sigmoid activation function.

[0029] The role of the multi-scale spatial attention module is to learn the importance weights of different regions at spatial locations, paying particular attention to key local details and structural information (such as edges and contour lines) in the image. Its calculation process is as follows: Formula (5) performs average pooling on all channels, and the result retains the average intensity of the channel at each spatial location. Formula (6) performs max pooling on all channels, and the result represents the most significant activation intensity at each spatial location.

[0030] ; Formulas (7)(8)(9) in the spatial attention module use dilated convolutions with three different dilation rates of d=1, 2 and 4 to extract multi-scale spatial features.

[0031] ; ; Equation (10) adds and fuses the features from the three scales above, and then activates them with Sigmoid to obtain the spatial attention map. Equation (11) uses the spatial attention map as a weight map and multiplies it element-wise with the original feature map to achieve spatial enhancement.

[0032] ; Formula (12) is the complete process to obtain the final output. The final output is: the original feature map is successively processed by Channel Attention (CA) and Spatial Attention (MSA) and finally fused to output.

[0033] This multi-scale dilation fusion attention module combines channel attention and multi-scale spatial attention, and utilizes dilated convolution to expand the receptive field. It can effectively focus on regions (such as shadows and contour lines) and feature channels (such as color channels) in the image that are crucial to the cartoonization effect. This allows the generated cartoon images to retain more detail in important areas, with richer and more natural colors and textures, while reducing redundant information and improving training efficiency and effectiveness.

[0034] The third core component of the generator is the decoder. The decoder's function is to reconstruct the target cartoon-style image based on the content features extracted by the content encoder and the style features extracted by the style encoder. The fusion of content features and style features is a key step in achieving effective style transfer. In this invention, the fusion is mainly achieved using Adaptive Instance Normalization (AdaIN) technology. AdaIN receives content features as input and uses style features (usually the mean and variance statistics of style features) to dynamically adjust the affine transformation parameters (scaling factor and shift factor) of the content features, thereby injecting style features and aligning the statistical distribution of the content features with the target style while preserving the original structural information. The decoder's network structure is typically the inverse of the encoder (downsampling process), including upsampling operations to restore image resolution. Specifically, the decoder of this invention first includes several residual blocks (e.g., four residual blocks, structurally similar to residual blocks in the encoder) for further processing of the fused features. Then, two upsampling operations are performed (e.g., using deconvolution / transposed convolution or interpolation methods such as bilinear interpolation, nearest neighbor interpolation, etc.), each typically doubling the spatial resolution (height and width) of the feature map. Convolutional layers are usually included between or after the upsampling layers for feature adjustment. Finally, a 7x7 convolutional layer is used to output the image. During decoding, instance layer normalization (ILN) is preferably used for normalization, and the Tanh activation function is used in the final output layer to ensure that the output pixel values ​​are in the range of [-1, 1] (corresponding to the real image normalized to this range), ensuring the numerical stability and stylistic consistency of the output. In summary, the image generation process can be described as follows: First, the input set of real images P={P1, P2, P3...Pn} and the target set of cartoon images C={C1, C2, C3...Cn} are preprocessed, including adjusting image size (e.g., uniformly scaling to 256x256 pixels), random cropping (if necessary), and pixel value normalization (e.g., normalizing pixel values ​​from [0,255] to the range [-1,1]). Then, the preprocessed real images are input into a content feature encoder to extract content feature vectors; simultaneously, the preprocessed cartoon images are input into a style feature encoder to extract style feature vectors. Next, using the AdaIN instance normalization technique, the style feature vectors are injected into the content feature vectors, achieving coupling between content and style features. The coupled feature vectors are input into a decoder, undergo a series of residual block processing and upsampling operations, and finally reconstruct the cartoon style image set S={S1, S2...Sn} through a 7x7 convolutional layer and the Tanh activation function.

[0035] The discriminator in the model employs a dual-discriminator architecture, including a global discriminator (D). global ) and local discriminator (D local The global discriminator focuses on the entire generated image, determining whether its overall style, texture, and color are consistent with the overall style of the target cartoon image. It focuses on the global consistency and harmony of the image. The local discriminator focuses on local patches within the image, determining whether the details of these local patches (such as the fineness of texture, the sharpness of edges, and small structural features) conform to the characteristics of a cartoon image. It focuses on the local realism and detail quality of the image. This dual discriminator mechanism, combining global and local discriminators, allows for a more comprehensive evaluation of the quality of the generated image, guiding the generator to produce high-quality cartoon images that are both stylistically consistent and rich in local detail. The basic structure of the two discriminators is typically based on a convolutional neural network (CNN) design, containing multiple layers of convolution, downsampling, normalization, and activation layers, ultimately outputting a scalar value representing the probability that the input image (or image patch) is a realistic cartoon image.

[0036] During training, edges in real-world images may be mistaken for noise, resulting in less sharp edges in generated images compared to cartoon images. Therefore, an edge detection module is used to address the issues of over-smoothing and unclear contours in the generated images. The Laplacian pyramid can capture both high-frequency and low-frequency components of an image. In this case, image details are related to the high-frequency components, while color is represented in the low-frequency components. The pixel-level features contained in the high-frequency components are crucial for more precise edge segmentation during edge detection. Specifically, the image is represented using a Laplacian pyramid at multiple scales, and the high-frequency edge information of each layer is extracted. This information is then weighted to calculate the edge loss. Compared to directly reducing the image size, this method more effectively preserves structural edges across multiple scales, avoiding misjudgments caused by blurring in low-resolution images.

[0037] Formula (13) is the edge residual loss based on the pyramid layer. Representing an image In the pyramids High-frequency detail extraction (Laplace residual map) is performed on the layer. This refers to the total number of pyramid levels. Refers to generating images. The target image.

[0038] To effectively constrain and optimize the generator, this invention designs a comprehensive loss function consisting of four weighted parts: adversarial loss. Content loss Perceived color loss and edge loss Total loss function Represented as: ;in: To combat the losses, To counteract the loss of weight, For content loss, Weight loss due to content To perceive color loss, To perceive color loss weights, For edge loss, These are the edge loss weights.

[0039] In combating losses In the computation, a dual discriminator approach is used to calculate the adversarial loss, requiring separate calculations for the global discriminator and the local discriminator. The global discriminator captures the overall style of the cartoon image through a global image-level branch, comparing the generated image with the real cartoon image. The global discriminator loss formula is expressed as follows: ;In formula (16), This refers to the target cartoon image. This represents the actual image input. This represents the generated cartoon image. The adjusted image is input into a local discriminator to obtain a local loss, which improves the local cartoonization effect of the generated image. The local discriminator loss is as follows: ; For local patching of the target cartoon image, This is a local patch used to generate a cartoon image.

[0040] The adversarial loss of the overall discriminator combines the losses of the global discriminator and the local discriminator, as shown in the following formula: To ensure that the content features of the generated image are similar to those of the input image and to prevent deviations in content structure, the model uses VGG19.

[19] As a perceptual network, it is used to extract high-level semantic features of images. The content loss formula is as follows: ;In formula (20), Cartoon images generated by the generator. The original image is the input. For layer feature mapping of VGG19, The loss is calculated by measuring the similarity between the generated image and the original image in the VGG19 feature space.

[0041] The model uses perceptual color loss primarily to preserve the color information of the generated image, preventing color shift or color distortion during style transfer. This is in contrast to ordinary... For pixel-level color loss, this application uses a perceptual color loss method combined with a VGG19 pre-trained network, calculating the loss in the feature space rather than the pixel space, resulting in more perceptually consistent colors. The formula for calculating the perceptual color loss is as follows: ;In formula (21), Indicates VGG19 Feature extraction of layers, For the weights of each layer, the same method is used. Loss calculation of color feature differences.

[0042] The MSDFA-GAN model proposed in this application is implemented based on PyTorch. Experimental data includes real images and cartoon images, with real images used for testing. All images are resized to 256*256 during training. The cartoon image dataset includes three different styles: *The Wind Rises*, *The Garden of Words*, and *Dragon Ball*, with 1000 cartoon images in each dataset, used to train the three models respectively. The real image dataset consists of 600 natural scenery images. The model runs on a single NVIDIA RTX4090 GPU. The MSDFA-GAN model undergoes 140 training iterations, including 20 pre-training iterations and 120 formal training iterations. The learning rates for the generator and discriminator are initially fixed at 0.0002. To improve the model's generalization ability, the learning rates for the generator and discriminator are reduced by 0.000002 per training iteration after 60 training iterations. The training batch size is set to n=8, and the Adam optimizer is used for optimization, with the loss weights set as follows: =5, =2, =0.4, =1.

[0043] Using the image "The Wind Rises" as the target image for training, the images generated by our model, CTSS, CartoonGAN, Whitebox, and AnimeGANv2 are shown in Figure 5. It can be seen that CTSS exhibits unstable cartoonization effects, easily producing artifacts, loss of detail, or oversimplification. CartoonGAN generates cartoon images with blurred outlines and edges, and insufficient color clarity. AnimeGANv2 sometimes over-preserves detail, making the image too realistic and losing its cartoonish effect. Whitebox generates images with unclear outlines and indistinct segmentation. Our model, however, generates cartoon images with uniform color, clear outlines, and preserves important content features, achieving a superior cartoonization effect.

[0044] This application uses FID values ​​for analysis, employs the Inceptionv3 model to extract image representations from two different domains, and calculates the distance between them to return a score. A lower score indicates a more similar distribution, suggesting a better cartoonization effect.

[0045] Table 1 lists the FID values ​​of CartoonGAN, Whitebox, AnimeGANv2, CTSS, and the method of this application for three different style datasets. Comparing the FID values, it can be seen that the model of this application has the smallest FID value, indicating that the image generated by the model of this application is more similar to the target image and has a better cartoonization effect.

[0046] Table 1 FID scores for different models

[0047] In the ablation experiment, using "The Wind Rises" as the cartoon dataset, as shown in Figure 6, after removing the multi-scale fusion attention module, module B's generated image showed little cartoonization and could not correctly determine colors. After removing the edge detection module, module C's generated image edges became unclear, with both edges and colors showing some loss, resulting in color mixing. After removing the perceptual color loss module, module D's generated image colors differed significantly from the original image, and overexposure occurred. The FID values ​​of the images generated by the above models were obtained. It can be seen that the FID values ​​of the images generated by the models lacking modules were all greater than those of the complete model. This indicates that the complete model of this application performs better on images with important content and complex edges and colors.

[0048] Table 2 FID values ​​from ablation experiments

[0049] Meanwhile, to verify the impact of the dilation rate 'd' in the Multi-Scale Dilated Attention (MSDFA) mechanism on the cartoonization effect, this application designed four different dilation rate combinations for comparative experiments using *The Garden of Words* as the target image: {1,2,3}, {1,2,4}, {1,3,5}, and {2,4,8}. Using the {1,2,3} combination results in a relatively dense receptive field, allowing the model to focus more on details and textures. Using {1,2,4}, the model used in this application, allows for localized and medium-scale receptive field coverage, balancing image detail and structural expression. Using {1,3,5} results in uneven receptive field coverage, thus assessing the impact of irregular receptive field spans. Using the {2,4,8} combination provides a large receptive field coverage, which can enhance the overall style; this study examines whether it weakens details.

[0050] As shown in Figure 7, the cartoon image generated by the {1,2,4} combination exhibits good edge and color representation while preserving original details. The {1,2,3} combination retains details, but the cartoonization effect is not obvious. The image generated by the {1,3,5} combination has unclear edges and unstable style. The {2,4,8} combination, on the other hand, over-stylizes the image, resulting in the loss of the original image structure and color. Furthermore, the {1,2,4} combination shows a lower FID value in the generated image, demonstrating the effectiveness of the chosen dilation rate.

[0051] Table 3. FID values ​​of images with different dilation rates

[0052] This application proposes an image cartoonization method using dual-encoder adversarial learning and a multi-scale dilatational fusion attention mechanism. The multi-scale dilatational fusion attention mechanism is incorporated into the generator, enabling the dual encoders to focus on key regions and retain more content details. An edge detection module is added to make the edges of the generated image clearer during the cartoonization process. Perceptual color loss is used in the loss function to make the colors of the generated image more similar to the original image. Extensive experiments verify the effectiveness and superiority of the proposed model. Future applications of this model can also be made in other fields, such as the stylization of game characters, scenes, and textures, enabling the conversion of realistic 3D images into a cartoon style and reducing art costs.

[0053] In summary, the beneficial effects of this invention are as follows: (1) The content feature encoder and style feature encoder process structural and style information respectively, reducing feature mixing interference, thereby preserving the semantic contour and contour details of the input image and improving the accuracy of the cartoonization result; (2) Combining channel attention and spatial attention, the key regions and channels are adaptively focused, and the dilated convolution expands the receptive field coverage, reducing irrelevant information redundancy, improving the texture richness and color naturalness of the generated image, and accelerating model convergence; (3) The Laplacian pyramid extracts the edge residual and calculates the edge loss, solving the problems of smoothing and blurring of the generated image and enhancing the sharpness of the boundary lines; (4) The adversarial loss, content loss, perceptual color loss and edge loss are weighted and coordinated to ensure the overall style consistency, local detail authenticity, content structure similarity and color stability of the image, and improve the visual coordination of the cartoon image; (5) The global discriminator maintains style uniformity, and the local discriminator enhances detail authenticity. The two work together to optimize the generator output and achieve stable generation of high-quality cartoon images.

[0054] The above are merely preferred embodiments of the present invention and are not intended to limit the present invention. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A cartoonization method for attention-based images using dual-encoding adversarial learning and multi-scale dilation fusion, characterized in that... include: A generator with a dual encoder structure is constructed, in which the content feature encoder extracts the structural semantic features of the input real image, and the style feature encoder extracts the texture style features of the target cartoon image; the style features are injected into the content features through adaptive instance normalization; a multi-scale dilatation fusion attention mechanism is used to process the fused features and adaptively enhance key channels and spatial regions. The decoder reconstructs and generates cartoon images; at the same time, a dual discriminator architecture consisting of a global discriminator and a local discriminator is adopted to evaluate the overall style consistency and local detail realism of the image respectively, and the generation process is optimized through adversarial learning.

2. The dual-encoding adversarial learning and multi-scale dilation fusion attention image cartoonization method as described in claim 1, characterized in that, Both the content feature encoder and the style feature encoder include, in sequence, a convolutional layer module, a residual block stacking module, and a multi-scale dilation fusion attention module. The residual block stacking module learns deep features by cascading residual blocks, and the multi-scale dilatation fusion attention module adaptively weights the feature map through channel attention mechanism and multi-scale spatial attention mechanism.

3. The dual-encoding adversarial learning and multi-scale dilation fusion attention image cartoonization method as described in claim 1, characterized in that, Extract the mean and variance statistics of style features, and dynamically adjust the affine transformation parameters of content features to align the content feature distribution with the target style.

4. The dual-encoding adversarial learning and multi-scale dilation fusion attention image cartoonization method as described in claim 1, characterized in that, The generator's loss function is composed of a weighted sum of adversarial loss, content loss, perceptual color loss, and edge loss. The adversarial loss is jointly calculated by the global discriminator loss and the local discriminator loss. The content loss is calculated by extracting high-level semantic feature similarity through a pre-trained network. The perceptual color loss constrains the consistency of color distribution in the feature space. The edge loss is calculated based on the multi-scale edge residuals of the Laplacian pyramid.

5. The dual-encoding adversarial learning and multi-scale dilation fusion attention image cartoonization method as described in claim 4, characterized in that, The edge loss calculation process is as follows: construct Laplacian pyramids for the generated image and the target image respectively, extract high-frequency edge residual maps layer by layer, sum the difference values ​​of the residuals of each layer by weight, and enhance the contour sharpness of the generated image.

6. The dual-encoding adversarial learning and multi-scale dilation fusion attention image cartoonization method as described in claim 1, characterized in that, The global discriminator uses a convolutional neural network to determine the matching degree between the overall style of the generated image and the target cartoon domain; the local discriminator evaluates the authenticity of details, textures and edge structures by cropping local regions of the image, and the loss functions of the two discriminators are jointly optimized to optimize the generator.

7. The dual-encoding adversarial learning and multi-scale dilation fusion attention image cartoonization method as described in claim 2, characterized in that, The channel attention mechanism execution process includes: performing global average pooling and max pooling on the input feature map in the channel dimension respectively, learning channel weights through a multilayer perceptron, and highlighting key feature channels in a weighted manner.

8. The dual-encoding adversarial learning and multi-scale dilation fusion attention image cartoonization method as described in claim 2, characterized in that, The execution process of the multi-scale spatial attention mechanism includes: performing average pooling and max pooling on the input feature map along the channel dimension, connecting multiple dilated convolutional layers with different dilation rates in parallel to extract multi-scale spatial features, fusing them to generate a spatial attention weight map, and enhancing the feature response at key spatial locations.

9. The dual-encoding adversarial learning and multi-scale dilation fusion attention image cartoonization method as described in claim 8, characterized in that, The dilation rate combination of the hollow convolutional layer covers the local to mesoscale receptive field, which is used to balance the extraction intensity of detailed features and structural features.

10. The dual-encoding adversarial learning and multi-scale dilation fusion attention image cartoonization method as described in claim 5, characterized in that, The Laplacian pyramid edge extraction process is as follows: multi-scale downsampling of the image, calculation of high-frequency residual components of each scale layer, and weighted superposition to obtain multi-scale edge feature maps, avoiding edge blurring or noise interference at a single scale.

Images