A method, system, device, and storage medium for extracting line art from anime images.

Abstract

This invention provides a method, system, device, and storage medium for extracting line art from anime images, belonging to the field of image processing technology. The method includes: first, performing preprocessing operations on the input artistic image; then deploying a multi-branch feature extraction network, utilizing parallel convolutional paths and inter-layer skip connections to collaboratively capture multi-scale edge response features, constructing a hierarchical feature map with spatial awareness. Next, a dynamic feature fusion module is designed to optimize cross-layer features in both local receptive field feature aggregation and pixel-level semantic alignment, completing the fusion process with learnable attention weights. Finally, a joint supervision mechanism is established, fusing an improved cross-entropy loss function and background artifact suppression constraints, improving line generation quality and suppressing background noise through pixel-level supervision. This method can achieve end-to-end conversion from complex artistic images to structured line art, effectively eliminating artifact interference while maintaining line continuity, and outputting editable line art that meets professional creative needs.

Description

Technical Field

[0001] This invention belongs to the field of image processing technology, specifically relating to a method, system, device, and storage medium for extracting line art from animation images. Background Technology

[0002] Line art extraction, an interdisciplinary field of computer vision and digital art creation, aims to automatically generate structured and semantically continuous line outlines from original images. This process preserves edge information crucial to visual structure while removing redundant textures and complex backgrounds. In art creation, line art is widely used in animation, illustration, digital coloring, style transfer, and composition assistance, serving as a highly expressive and instructive intermediate form of visual art. High-quality line art not only helps clarify the structure and layers of an image but also provides a clear and controllable reference framework for subsequent creative processing. For a long time, line art has relied on manual drawing, a time-consuming and labor-intensive process requiring exceptional drawing skills and structural understanding. This not only raises the barrier to entry for artistic creation but also significantly limits creative efficiency and content output speed. Therefore, recent research has focused on achieving high-quality line art extraction through automated algorithms, particularly deep learning-based methods that have made progress in image edge detection and semantic modeling. However, existing technologies still have many shortcomings. For example, when processing images with complex backgrounds, strong light and shadow interference, or rich material textures, the extraction results often exhibit problems such as broken outlines, blurred edges, or structural disorder. These problems, to some extent, limit the practical application of line drawing extraction technology in professional artistic creation. There is an urgent need to further optimize the model's semantic understanding and structural modeling capabilities to better serve creative generation and artistic expression.

[0003] For example, Chinese invention patent CN119479044A discloses an intelligent comic line art generation system that preserves facial features. This system includes a backbone network U-NET module and a control network U-NET module to generate an initial comic line art draft, and a post-processing line art adjustment module to fine-tune the initial draft, resulting in the final line art draft. Another example is Chinese invention patent CN118968241A, which discloses a mural line art extraction method based on multi-scale extraction and intra-layer learnable fusion. This method establishes a line art extraction model, including a multi-scale feature extraction module and an intra-layer learnable fusion module, and uses this model to generate image line art.

[0004] However, current line art extraction methods have significant limitations in specific fields such as anime-style images, and are insufficient in processing complex textures or high-density edge scenes, easily resulting in artifacts such as edge adhesion and background artifacts. Summary of the Invention

[0005] To address the problem that shallow features are easily over-abstracted or suffer from information decay in deep networks, this invention provides a method for extracting line art from animation images, aiming to generate structured line art that artists can directly use for secondary creation.

[0006] To achieve the above objectives, the present invention provides the following technical solution:

[0007] A method for extracting line art from anime images, comprising:

[0008] Obtain the anime-style image to be processed;

[0009] The high-order features in the hierarchical feature map are deconvolutionally reconstructed, and the reconstructed high-order features are concatenated with adjacent low-order features. The concatenated features are then fused at the pixel level across hierarchical responses through point convolution. At the same time, the edge distortion of the high-order features is corrected by the geometric constraints of the low-order features, resulting in a complementary and enhanced feature representation.

[0010] By dynamically fusing complementary and enhanced feature representations, multi-scale and multi-semantic hierarchical feature responses that integrate pixel-level information are obtained;

[0011] Editable line art is output by integrating multi-scale and multi-semantic level feature responses with pixel-level information.

[0012] Preferably, the anime-style image to be processed is input into the improved U... 2 -Net network, outputting multi-scale and multi-semantic level feature responses that integrate pixel-level information; the improved U 2 -Net network with U 2 -Based on the nested U-shaped structure of -Net, deconvolution is added at the input of each parallel convolutional layer of the symmetric encoder-decoder after the input convolutional layer, and SE attention modules are embedded at the output of each parallel convolutional layer.

[0013] Preferably, the improved U 2 - The parallel convolutional layers of the -Net network set convolutional paths with different receptive fields. By using convolutional kernels of different sizes or different strides, convolutional operations are performed in parallel on the anime-style images to be processed, extracting features from the images at different scales, generating feature responses containing multi-scale information, and constructing hierarchical feature maps with spatial awareness.

[0014] Preferably, the improved U 2 The skip connection layer of the -Net network bypasses the intermediate multi-layer network and directly transmits the low-order features extracted by the shallow network to the deep network. The shallow network is used to capture the low-order visual features of the image, and the deep network is used to extract the high-order semantic features of the image.

[0015] Preferably, the improved U2 The -Net network applies the SE attention module to perform global average pooling on the hierarchical feature maps in each parallel convolutional layer of the symmetric encoder-decoder, compressing global spatial information to generate channel description vectors. The channel description vectors are then processed by two fully connected layers and a sigmoid function to generate attention vectors containing channel importance weights. A 3×3 deep convolutional layer is used to independently process the spatial information of each channel of the attention vector, capturing local neighborhood feature interactions. Then, a 1×1 convolutional layer is used to fuse cross-channel information point by point through the local neighborhood feature interaction results, integrating multi-scale and multi-semantic level feature responses to obtain multi-scale and multi-semantic level feature responses that have completed pixel-level information integration.

[0016] Preferably, the step of inputting the anime-style image to be processed into the improved U... 2 Prior to the -Net network, the network was also trained using an inhibitory loss function based on response distribution analysis and an improved cross-entropy loss function.

[0017] Preferably, the step of inputting the anime-style image to be processed into the improved U... 2 Before the -Net network, preprocessing is performed on the anime-style images to be processed, including illumination balancing and feature enhancement. The preprocessed images are then resized to obtain standardized input data, which is then input into the improved U... 2 -Net network.

[0018] This invention also proposes a line art extraction system for anime-style images, comprising:

[0019] The image acquisition module is used to acquire anime-style images to be processed;

[0020] The feature extraction module is used to extract multi-scale edge response features of anime-style images through parallel convolutional paths, generating hierarchical feature maps with spatial awareness. The multi-scale edge response features are used to represent low-order and high-order features of the image from different receptive fields. The low-order features include the position, direction, and connectivity information of the contour lines, and the high-order features include the semantic information of the background and the character.

[0021] The feature enhancement module is used to perform deconvolution reconstruction on high-order features in the hierarchical feature map, and then concatenate the reconstructed high-order features with adjacent low-order features. The concatenated features are then fused at the pixel level across the hierarchical response through point convolution. At the same time, the edge distortion of the high-order features is corrected by the geometric constraints of the low-order features to obtain complementary enhanced feature representation.

[0022] The feature fusion module is used to dynamically fuse complementary enhanced feature representations to obtain multi-scale and multi-semantic level feature responses that integrate pixel-level information.

[0023] The line art extraction module is used to output editable line art by integrating multi-scale and multi-semantic level feature responses with pixel-level information.

[0024] The present invention also provides a computer device, including a memory, a processor, and a computer program stored in the memory, wherein the processor executes the computer program to implement any of the steps in the animation image line drawing extraction method.

[0025] The present invention also provides a computer-readable storage medium storing a computer program, which, when loaded by a processor, is capable of executing any of the steps in the animation image line drawing extraction method.

[0026] The method for extracting line art from anime images provided by this invention has the following beneficial effects:

[0027] This invention extracts multi-scale edge response features through parallel convolutional paths and, combined with the construction of hierarchical feature maps, directly captures low-order and high-order features, avoiding excessive abstraction of shallow features by deep networks. Simultaneously, it utilizes the geometric constraints of low-order features to correct edge distortion in high-order features, mitigating the attenuation of feature information during transmission. After deconvolutional reconstruction of high-order features, they are concatenated with adjacent low-order feature channels and fused at the pixel level through point convolution. The detailed information of low-order features supplements the lost details of high-order features, balancing the differences between high noise in low-order features and insufficient detail in high-order features, achieving complementary enhancement. Through dynamic fusion of complementary enhanced feature expressions, pixel-level information integration is achieved. Combining the different receptive fields of multi-scale edge response features improves the fusion effect of local details and global structure, avoiding feature blurring or structural disorder after fusion. Precise cross-level feature fusion and geometric constraint correction reduce the interference of background semantic information in high-order features on edges, suppress the impact of background artifacts on line drawing extraction, and improve the accuracy and precision of line drawing extraction. Attached Figure Description

[0028] To more clearly illustrate the embodiments and design schemes of the present invention, the accompanying drawings required for this embodiment will be briefly described below. The drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0029] Figure 1 This is a flowchart of the animation image line drawing extraction method according to Embodiment 1 of the present invention;

[0030] Figure 2 The improved U proposed in this invention 2 -Net model structure diagram;

[0031] Figure 3 For detailed comparison of the extracted line art results of the model;

[0032] Figure 4 This is a complete flowchart of the animation image line drawing extraction method of the present invention. Detailed Implementation

[0033] To enable those skilled in the art to better understand and implement the technical solutions of the present invention, the present invention will be described in detail below with reference to the accompanying drawings and specific embodiments. The following embodiments are only used to more clearly illustrate the technical solutions of the present invention and should not be construed as limiting the scope of protection of the present invention.

[0034] Example 1

[0035] This invention provides a method for extracting line art from anime images, specifically as follows: Figure 1 and Figure 4 As shown, it includes the following steps:

[0036] Step 1: Collaborate with a professional animation team to build the first high-precision hand-drawn line art dataset, ArtLine-2K. This dataset contains 2,000 pairs of rendered images and line art, which are expanded to 10,000 pairs after data augmentation, covering a variety of animation styles to ensure the diversity and generalization ability of model training.

[0037] Step 2: Perform preprocessing on the artistic images (including illumination balancing and feature enhancement) to improve image quality and enhance feature recognizability, and standardize data size to provide standardized input for subsequent feature extraction and model training.

[0038] Step 3: Construct a multi-branch feature extraction network based on U 2 -Net nested U-shaped structure improvement: Parallel convolutional layers and skip connections are used to extract multi-scale edge response features, generating hierarchical feature maps, and deconvolution is introduced to improve the accuracy of hierarchical feature fusion. This invention leverages a U-shaped nested U-shaped structure. 2-Net, an improved multi-branch feature extraction network with a nested U-shaped structure, extracts multi-scale edge response features from images through parallel convolutional layers and skip connections, generating hierarchical feature maps with spatial awareness. Deconvolution is also introduced to improve the accuracy of hierarchical feature fusion. Specifically, multi-scale edge response features of anime-style images are extracted through parallel convolutional paths, generating hierarchical feature maps with spatial awareness. These multi-scale edge response features are used to represent low-order and high-order features of the image from different receptive fields. Low-order features include the position, direction, and connectivity information of contour lines, while high-order features include semantic information about the background and characters. Capturing semantic information aims to improve the accuracy and structure of line art extraction, specifically in the following ways: Optimizing edge discrimination capability: By capturing semantic information in high-order features (such as the limb structure and clothing outlines of characters in anime images), combined with geometric details of low-order features (such as line direction and local edges), edge distortion of high-order features can be corrected, avoiding false edge responses caused by relying solely on low-order features (such as false lines generated by complex texture interference), and enhancing the robustness of valid line discrimination. Ensuring semantic continuity of lines: Line art must possess structure and semantic coherence (such as the integrity of character outlines and the logicality of object boundaries). Capturing semantic information allows the model to understand the semantic relationships between different elements in the image (such as the connection between hair and face, and the matching of clothing folds with body movements), thereby maintaining the overall structural coherence of lines when extracting line art and avoiding problems such as broken outlines and structural inconsistencies. Enhancing the ability to handle complex scenes: In the complex backgrounds of anime-style images (such as environmental shadows and rich textures), semantic information can help the model distinguish between "target lines" (such as character edges) and "background interference" (such as irrelevant textures and lighting noise), reducing the generation of background artifacts and ensuring that the extracted line art focuses on edge information that is crucial to visual structure, meeting the needs of professional creation for high-fidelity line art.

[0039] To improve U 2 -Net's multi-branch feature extraction network forms the backbone, and its feature extraction module can generate responses for each side, represented as shown in the equation. Given that line art belongs to low-order visual features (such as edge contours, line direction, and local texture), it is necessary to avoid excessively deep layers that lead to over-abstraction or information decay of shallow features. Experiments show that when the network reaches the fourth layer, the response of deep convolution kernels decays significantly, but removing the fourth layer weakens the feature extraction ability due to insufficient depth. To address this, a dynamic fusion strategy for adjacent features is proposed: First, deconvolution is performed on high-order features (introducing learnable upsampling kernel parameters to enhance reconstruction controllability and reducing edge errors by increasing nonlinearity). The reconstructed high-order features are then concatenated with adjacent low-order features passed through skip connections. Finally, pixel-level fusion of cross-level responses is achieved through point convolution. While retaining the advantages of multi-scale operation, low-order geometric constraints are used to correct high-order edge distortion, forming a complementary and enhanced feature representation.

[0040] In the improved model, deconvolution and skip connections are added to U 2 -The multi-branch feature extraction network part of -Net is specifically distributed among the various layers of its nested U-shaped structure.

[0041] U 2- In Net, parallel convolutional layers are distributed across different levels. These layers perform convolution operations on the input image in parallel by setting convolutional paths with different receptive fields, such as using convolutional kernels of different sizes or strides. For example, in lower-level parallel convolutional layers, convolutional paths with small receptive fields capture local details of the image, such as the bends and endpoints of lines; while in higher-level layers, convolutional paths with large receptive fields focus on the global structural features of the image, such as the overall outline of objects and the general direction of lines. These parallel convolutional paths work simultaneously, extracting features from the image at different scales, generating feature responses containing rich multi-scale information, and thus constructing hierarchical feature maps with multi-scale perception capabilities.

[0042] Jump connections also exist in U 2 -Nets are nested within a U-shaped structure. Their main function is to directly transfer low-level features extracted from shallow networks (containing rich original spatial details, such as basic edge contours and the initial direction of lines) to deeper networks. For example, during the process of transferring features from shallow to deep layers, skip connections directly transmit information about the basic structure of the image extracted from shallow networks to deeper networks, bypassing intermediate layers. This allows for complex feature extraction and abstraction in deep networks while simultaneously incorporating basic features from shallow layers. This effectively avoids information loss or attenuation caused by the continuous abstraction of shallow features in deep networks, resulting in feature maps that contain both deep semantic information and retain shallow spatial details, providing a more comprehensive and accurate information foundation for subsequent feature fusion and processing.

[0043] In the improved model architecture, parallel convolutional layers and skip connections work together to acquire and integrate image features from different scales to generate high-quality hierarchical feature maps, providing rich information for subsequent line drawing extraction tasks. Their specific collaborative mechanism is as follows:

[0044] Parallel convolutional layers are responsible for multi-scale feature extraction: These layers employ convolutional paths with different receptive fields, using kernels of varying sizes or strides to perform convolutional operations on the input image in parallel. At lower levels, convolutional paths with smaller receptive fields focus on capturing local details, such as subtle bends and endpoints of lines. At higher levels, convolutional paths with larger receptive fields focus on global structural features, such as the overall outline of objects and the general direction of lines. In this way, parallel convolutional layers can extract features from images at different scales, generating feature responses rich in multi-scale information and constructing hierarchical feature maps with multi-scale perception capabilities.

[0045] Skip connections transfer shallow features: The role of skip connections is to directly transfer low-level features extracted by shallow networks (containing rich original spatial details, such as basic edge contours and the initial direction of lines) to deep networks. In the process of transferring features from shallow to deep layers, skip connections bypass intermediate layers, passing basic feature information from the shallow layers to the deep networks. This effectively avoids the information loss or attenuation problem caused by the continuous abstraction of shallow features in deep networks, allowing deep networks to combine basic shallow features while performing complex feature extraction and abstraction.

[0046] Collaborative optimization of cross-level features: Multi-scale features extracted by parallel convolutional layers generate corresponding hierarchical feature maps across network layers. Meanwhile, shallow, low-order features passed through skip connections interact with high-order features (containing more abstract semantic information) extracted by parallel convolutional layers in deeper networks. High-order features can leverage the geometric constraints of low-order features to correct their own edge distortion, for example, making line edges more accurate when generating line art. Low-order features can also enhance their ability to focus on key edges under the semantic guidance of high-order features, such as more accurately identifying object contour edges. Ultimately, through this collaboration, complementary enhancement of cross-level features is achieved, providing more comprehensive and accurate multi-scale feature input for subsequent dynamic feature fusion modules, thereby improving the accuracy and quality of line art extraction.

[0047] Multi-scale processing is achieved through parallel convolutional paths. Different convolutional paths have different receptive fields (e.g., through different kernel sizes or stride settings), allowing them to focus on regions of different sizes within the image. For example, a convolutional path with a small receptive field can capture subtle transitions and endpoints of lines, while a path with a large receptive field can focus on global structural edge features such as the overall outline of an object and the overall direction of lines. Edge response features refer to the feature representation of edge information in an image (such as object outlines and line boundaries). These features reflect key information such as the existence, location, direction, and connectivity of edges. Hierarchical feature maps with spatial awareness lay the foundation for subsequent cross-level feature optimization and fusion, ultimately contributing to the generation of high-quality line art.

[0048] In this embodiment, deconvolution is introduced to improve the traditional upsampling operation. Traditional upsampling is a linear operation, while deconvolution introduces non-linear operations to achieve a better mapping from low-resolution feature maps to high-resolution feature maps. Based on the generation of hierarchical feature maps by extracting multi-scale edge response features using parallel convolutional layers and skip connections, deconvolution upsamples these feature maps, restoring their spatial dimensions and enabling them to contain more spatial detail. For example, if a feature map at a certain level is shrunk due to convolution operations, deconvolution can enlarge its size, expanding the previously compressed spatial information and thus supporting subsequent accurate perception of the spatial location and edge orientation of objects in the image.

[0049] Feature fusion optimization involves fusing the upsampled feature maps generated by deconvolution with feature maps from other layers (such as those passed through skip connections). Feature maps at different layers contain information of varying levels of abstraction and scale. During fusion, the upsampled spatial details of the deconvolution-generated feature maps complement the information from other feature maps. For example, higher-level feature maps contain more semantic information, while lower-level feature maps retain more original spatial details. The participation of the deconvolution-generated feature maps in fusion allows higher-level features to maintain semantic understanding while better incorporating lower-level spatial details, thus optimizing the accuracy of hierarchical feature fusion. Through this fusion optimization, the final generated hierarchical feature maps more comprehensively reflect the spatial layout of objects in the image, the spatial continuity of edges, and other information, thereby possessing stronger spatial perception capabilities.

[0050] Step 4: Generate hierarchical feature maps with spatial awareness using a multi-branch feature extraction network. Optimize cross-level features through scale-adaptive response, local receptive field feature aggregation, and pixel-level semantic alignment. Combine the geometric constraints of low-order features to correct edge distortion of high-order features, achieving complementary and enhanced feature representation. Through scale-adaptive response, local receptive field feature aggregation, and pixel-level semantic alignment, perform deconvolution on high-order features. Concatenate the reconstructed high-order features with adjacent low-order features, and then perform point convolution to achieve pixel-level fusion of cross-level responses. Combine the geometric constraints of low-order features to correct edge distortion of high-order features, forming complementary and enhanced feature representation. This addresses the issue of differences in responses across different levels by using the geometric constraints of low-order features to correct edge distortion of high-order features.

[0051] Step 5: Apply the dynamic feature fusion module to optimize and integrate cross-level responses through learnable attention weights: embed SE attention modules in each side fusion path, and use parallel two-layer depthwise separable convolutions - the first layer 3×3 convolution realizes local feature interaction, and the second layer 1×1 pointwise convolution establishes cross-channel association, thus completing the pixel-level information integration of multi-scale features.

[0052] Specifically, to address the representational discrepancies between low-level features and high-level semantic features, such as Figure 2 As shown, the improved U 2 -Net introduces a dynamic fusion mechanism that achieves more effective feature fusion through adaptive feature guidance and multi-scale interaction. Specifically, the network first introduces an attention mechanism based on the SE module in the side paths. This module compresses the input features through global average pooling and applies attention weights to the channel dimension of the compressed features, thereby enhancing the distinction between the importance of channels. After generating the fusion-guided features, the system further concatenates the feature maps of each branch along the channel dimension to form the initial dynamic fusion features. The constructed fusion features are adjusted through a dual-branch convolutional path. The two branches employ different depthwise separable convolutional kernels, 3×3 and 1×1 respectively, to achieve local perception and cross-channel interaction. The calculation method is shown below: , Where σ represents the activation function, DWConv n×nThis represents a depthwise separable convolution operation. This structural design allows the network to simultaneously focus on local details and global contextual structure during the fusion process, achieving multi-scale, cross-dimensional semantic compensation and edge fidelity. Based on this, the fused features are input into subsequent network structures for end-to-end training, effectively improving the model's ability to construct discriminative features across different layers. The proposed dynamic fusion strategy not only enhances the discriminativity of feature representations but also improves the network's understanding of multi-dimensional content and structural modeling capabilities within images.

[0053] Step 6: Design a joint supervision mechanism: To address the background artifact problem, an inhibitory loss function based on response distribution analysis is proposed, which applies a penalty to each pixel based on the deviation of the false response; a jointly improved cross-entropy loss function optimizes line quality, thereby improving the model's convergence speed, accuracy, and robustness.

[0054] Specifically, when training the model using cross-entropy as the basic supervision signal, two key problems are exposed: first, false edge fragments unrelated to the target structure are easily generated in the background region; second, the edge lines are affected by factors such as color unevenness, resulting in unstable prediction quality. To address the former, this invention proposes a background suppression loss function based on the predicted response distribution. By constructing a pixel-level false detection mechanism, the network is guided to suppress invalid responses. This mechanism statistically analyzes the predicted output of each pixel to identify the degree of false response and assigns different weights based on the magnitude of its deviation from the true label, thereby achieving a differentiated penalty strategy. The loss term is defined as follows: The pixel loss weight function is defined as follows: , The global expectation of the background pseudo-label is used to balance the contribution of each pixel's loss under different background complexities. To address the blurring and unevenness issues in line prediction, a collaborative optimization strategy combining temperature scaling and label smoothing is proposed. The specific form is: , , This method first converts the original labels y∈{0,1} into soft labels. ,in A small constant is used to control the smoothness of the label distribution, which can effectively improve the model's support for extreme value prediction and improve the calibration effect of probability output. Simultaneously, a temperature factor τ is introduced at the output to scale the model output, thereby widening the differences in the probabilities of each class in the predicted distribution and enhancing the model's discrimination ability in the high-confidence region. To balance the model's stability in the early training phase with refined convergence in the later phase, this method designs a gradual cooling strategy: a high temperature is set in the initial training phase. Explore the space by expanding parameters, then linearly decay the temperature every 10 epochs. This enhances the model's ability to learn in extreme prediction regions. Finally, the overall training process is optimized using the following joint loss function: ,in, , The weight parameters are adjustable and It is used to dynamically control the optimal balance between background suppression and edge accuracy.

[0055] Step 7: Train the model using the ArtLine-2K dataset to obtain a high-fidelity line art extraction model. Apply the trained model to a real-world line art extraction task to achieve automated high-fidelity line art output. During training, the model automatically adjusts its network parameters by learning the rendered image-line art pair to minimize the loss function.

[0056] This invention avoids excessively deep hierarchical structures by controlling network depth and employs U... 2 -Net's encoder-decoder architecture. This architecture utilizes its nested U-shaped hierarchy and skip connection mechanism to directly capture shallow features (such as edge contours and line directions) from each stage of the encoder, thereby bypassing the traditional backbone network, reducing computational overhead and mitigating feature loss.

[0057] To address the differences in response at different levels (such as high noise in low-order responses and loss of detail in high-order responses), a dynamic fusion strategy for adjacent-level features is proposed: First, deconvolution is performed on high-order features (introducing learnable upsampling kernel parameters to enhance controllability and nonlinearity, and reduce edge errors). Then, the reconstructed high-order features are concatenated with the low-order features passed from adjacent skip connections. Finally, pixel-level fusion of cross-level responses is achieved through point convolution. The geometric constraints of low-order features are used to correct the edge distortion of high-order features, achieving complementary and enhanced feature representation.

[0058] To address the issue of pseudo-edge responses caused by interference from similar textures in low-order features, a channel attention mechanism (SE module) is embedded in the side aggregation module. The mechanism's execution flow is as follows: Global Average Pooling (GAP) is applied to the input feature map to compress global spatial information and generate a compact channel description vector; this vector is then input into a bottleneck structure consisting of two fully connected layers (FC) (with a non-linear activation function such as ReLU performing information transformation between them); finally, an attention vector containing channel importance weights is generated through the Sigmoid function. This vector is applied to the original input feature map to perform adaptive feature recalibration based on the overall semantic content of the image, significantly suppressing invalid texture responses and enhancing the focusing ability of discriminative edge cues, thereby improving the edge discrimination robustness of low-order features against complex texture backgrounds.

[0059] To address the issue of insufficient quality in multi-scale feature fusion, a two-stream depthwise separable convolution process is implemented on the stitched composite feature map: the first layer employs a 3×3 depthwise convolutional layer, independently processing spatial information from each channel, focusing on capturing local neighborhood feature interactions (such as edge orientation and connectivity), maximizing the preservation of original spatial details while ensuring computational efficiency; the second layer applies a pointwise convolutional layer, fusing cross-channel information from the depthwise convolution output, integrating multi-scale and multi-semantic level feature responses, and enhancing the ability to model the overall edge contour and global structure. This two-stream design synergistically optimizes the key requirements of local detail clarity and global structural coherence in edge detection tasks.

[0060] To address the issue of discrete point noise and short line segment clusters causing pseudo-edge fragments (background artifacts) in complex background regions (such as clothing folds or environmental shadows), an inhibitory loss function based on response distribution analysis is proposed. First, background pixels are labeled using an indicator function I_bg. Then, the cross-entropy loss L_ce of the background pixels is calculated. Finally, a pixel-level dynamic penalty factor α (calculated based on the deviation |P_i-Y_i| between the predicted value P_i and the true value Y_i) is introduced, and the loss contribution is weighted by an adjustable penalty coefficient β. This function imposes a severe penalty on pixels with high deviation from the background, significantly suppressing background artifacts and reducing the model's sensitivity to interference such as water stains / shadows.

[0061] To address the issues of low tolerance for sub-pixel positional deviations and uneven line color caused by gradient vanishing resulting from label binarization (0 / 1) in traditional cross-entropy loss, a combined approach of temperature scaling and label smoothing is adopted: First, hard labels are converted to soft labels Y_smooth=(1-ε)×Y+ε / K (ε is the smoothing intensity parameter), improving probability calibration and enhancing support for correct extreme predictions; Second, a temperature coefficient T is introduced at the prediction end to scale the output logic P_temp=softmax(logits / T), amplifying the difference in probability distribution to improve the model's confidence in clear lines;

[0062] To address the trade-off between training stability and convergence speed in the collaborative optimization of temperature scaling and label smoothing, a phased heating strategy was designed: a high temperature T was set at the beginning of training to broaden the parameter search space, and then T was linearly reduced to the target value every 10 epochs. This strategy allows the model to freely explore the parameter space in the early stages, and then focuses on enhancing the confidence of clear lines in the later stages, ultimately achieving efficient convergence in about 30 minutes.

[0063] Through the systematic improvements in feature extraction, fusion mechanism, and loss design, the model ultimately achieves high-fidelity line drawing extraction for complex anime-style artistic images. The generated line drawings, after evaluation by professional artists, can be directly used for secondary creation, providing high-quality structured input for subsequent artistic creation tasks such as anime coloring, image style transfer, and digital illustration generation.

[0064] Figure 3 This is the final rendering. "Image" is the original anime-style image; "label" is the line art hand-drawn by the artist based on the original anime-style image (a 2048x2048 line art requires a full day to draw by hand); "ours" is the line art result generated by the model of this invention based on the original anime-style image. The part highlighted in red indicates a comparison between the artist's hand-drawn line art and the line art result generated by the model of this invention, magnified 10 times. The comparison results show only minor differences and can be directly provided to the artist for further processing.

[0065] This invention uses U 2 The -Net architecture and skip connections directly capture shallow features, reducing computational overhead and feature loss, and providing more complete basic features for subsequent processing. A dynamic fusion strategy for adjacent features is proposed, achieving cross-level pixel-level fusion through deconvolution, channel concatenation, and point convolution. Low-order features are used to correct edge distortion of high-order features, improving feature complementarity. An SE attention module is embedded, suppressing invalid textures through channel weight adjustment, enhancing edge cue focusing ability, and improving edge discrimination robustness in complex backgrounds. Two-stream depthwise separable convolution is employed to capture local feature interactions and cross-channel information separately, optimizing the balance between local details and global structure, and strengthening edge modeling capabilities. Finally, end-to-end conversion from complex animation images to structured line art is achieved, generating high-fidelity line art with continuous lines and no artifacts, meeting the needs of professional secondary creation.

[0066] Based on the same inventive concept, this invention also proposes a line drawing extraction system for anime-style images, comprising:

[0067] The image acquisition module is used to acquire anime-style images to be processed.

[0068] The feature extraction module is used to extract multi-scale edge response features of anime-style images through parallel convolutional paths, generating hierarchical feature maps with spatial awareness. The multi-scale edge response features are used to represent low-order and high-order features of the image from different receptive fields. Low-order features include the position, direction, and connectivity information of the contour lines, while high-order features include the semantic information of the background and the characters.

[0069] The feature enhancement module is used to perform deconvolution reconstruction on high-order features in the hierarchical feature map, and then concatenate the reconstructed high-order features with adjacent low-order features. The concatenated features are then fused at the pixel level across hierarchical responses through point convolution. At the same time, the geometric constraints of the low-order features are used to correct the edge distortion of the high-order features, resulting in a complementary enhanced feature representation.

[0070] The feature fusion module is used to dynamically fuse complementary enhanced feature representations to obtain multi-scale and multi-semantic level feature responses that integrate pixel-level information.

[0071] The line art extraction module is used to output editable line art by integrating multi-scale and multi-semantic level feature responses with pixel-level information.

[0072] The modules in the aforementioned line art extraction system for anime-style images can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in or independent of the processor in a computer device, or stored in the computer device's memory as software, so that the processor can call and execute the corresponding operations of each module.

[0073] The present invention also provides a computer device, including a memory, a processor, and a computer program stored in the memory. The processor executes the computer program to implement the steps in the embodiment of the animation image line drawing extraction method. Specific implementation methods can be found in the method embodiments, and will not be repeated here.

[0074] Furthermore, the present invention also provides a non-transitory computer-readable storage medium containing instructions, on which a computer program is stored. For example, a memory containing instructions that can be executed by a processor of a computer device to perform the above-described method. For example, the non-transitory computer-readable storage medium may be a ROM, random access memory (RAM), CD-ROM, magnetic tape, floppy disk, and optical data storage device, etc. When the computer program is executed by the processor, it can implement the steps in the embodiments of the animation image line drawing extraction method. Specific implementation methods can be found in the method embodiments, which will not be repeated here.

[0075] Those skilled in the art will understand that embodiments of the present invention can provide methods, systems, or computer program products. Therefore, the present invention can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention can take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0076] This invention is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, as well as combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart. Figure 1 One or more processes and / or boxes Figure 1A device that provides the functions specified in one or more boxes.

[0077] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0078] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0079] It should be noted that the specific embodiments described above enable those skilled in the art to more fully understand the present invention, but do not limit the present invention in any way. Therefore, although the present invention has been described in detail in this specification and embodiments, those skilled in the art should understand that modifications or equivalent substitutions can still be made to the present invention; and all technical solutions and improvements that do not depart from the spirit and scope of the present invention are covered within the protection scope of the present invention patent. No reference numerals in the claims should be construed as limiting the scope of the claims. Any simple variations or equivalent substitutions of technical solutions that can be readily obtained by those skilled in the art within the scope of the technology disclosed in the present invention are within the protection scope of the present invention.

Claims

1. A method for extracting line art from anime images, characterized in that, include: Obtain the anime-style image to be processed; Input the anime-style image to be processed into the improved U... 2 -Net network, outputting multi-scale and multi-semantic level feature responses that integrate pixel-level information; the improved U 2 -Net network with U 2 -Based on the nested U-shaped structure of Net, deconvolution is added at the input of each parallel convolutional layer of the symmetric encoder-decoder after the input convolutional layer, and an SE attention module is embedded at the output of each parallel convolutional layer; the SE attention module compresses the input features through global average pooling and applies attention weights on the compressed channel dimension. After generating the fusion guiding feature, the feature maps of each branch are concatenated and stitched together along the channel dimension to form the initial dynamic fusion feature. The dynamic fusion feature is then adjusted for feature mapping through a two-branch convolutional path. Through improved U 2 The Net network processes anime-style images by: extracting multi-scale edge response features of anime-style images through parallel convolutional paths to generate hierarchical feature maps with spatial awareness; the multi-scale edge response features are used to represent low-order and high-order features of the image from different receptive fields, the low-order features include the position, direction and connectivity information of the contour lines, and the high-order features include the semantic information of the background and the character. The high-order features in the hierarchical feature map are deconvolutionally reconstructed, and the reconstructed high-order features are concatenated with adjacent low-order features. The concatenated features are then fused at the pixel level across hierarchical responses through point convolution. At the same time, the edge distortion of the high-order features is corrected by the geometric constraints of the low-order features, resulting in a complementary and enhanced feature representation. By dynamically fusing complementary and enhanced feature representations, multi-scale and multi-semantic hierarchical feature responses that integrate pixel-level information are obtained; Editable line art is output by integrating multi-scale and multi-semantic level feature responses with pixel-level information.

2. The method for extracting line art from anime images according to claim 1, characterized in that, The improved U 2 - The parallel convolutional layers of the -Net network set convolutional paths with different receptive fields. By using convolutional kernels of different sizes or different strides, convolutional operations are performed in parallel on the anime-style images to be processed, extracting features from the images at different scales, generating feature responses containing multi-scale information, and constructing hierarchical feature maps with spatial awareness.

3. The method for extracting line art from anime images according to claim 1, characterized in that, The improved U 2 The skip connection layer of the -Net network bypasses the intermediate multi-layer network and directly transmits the low-order features extracted by the shallow network to the deep network. The shallow network is used to capture the low-order visual features of the image, and the deep network is used to extract the high-order semantic features of the image.

4. The method for extracting line art from anime images according to claim 2, characterized in that, The improved U 2 The -Net network applies the SE attention module to perform global average pooling on the hierarchical feature maps in each parallel convolutional layer of the symmetric encoder-decoder, compressing global spatial information to generate channel description vectors; the channel description vectors are then processed by two fully connected layers and a sigmoid function to generate attention vectors containing channel importance weights; a 3×3 deep convolutional layer is used to independently process the channel spatial information of the attention vectors to capture local neighborhood feature interactions; Then, by using a 1×1 pointwise convolutional layer to fuse cross-channel information through the interaction results of local neighborhood features, multi-scale and multi-semantic level feature responses are integrated to obtain multi-scale and multi-semantic level feature responses that have completed pixel-level information integration.

5. The method for extracting line art from animation images according to claim 4, characterized in that, Input the anime-style image to be processed into the improved U... 2 Prior to the -Net network, the network was also trained using an inhibitory loss function based on response distribution analysis and an improved cross-entropy loss function.

6. The method for extracting line art from animation images according to claim 1, characterized in that, Input the anime-style image to be processed into the improved U... 2 Before the -Net network, preprocessing is performed on the anime-style images to be processed, including illumination balancing and feature enhancement. The preprocessed images are then resized to obtain standardized input data, which is then input into the improved U... 2 -Net network.

7. A line art extraction system for anime-style images, characterized in that, include: The image acquisition module is used to acquire anime-style images to be processed; Improved U 2 -Net network, used to process anime-style images, outputting multi-scale and multi-semantic level feature responses with pixel-level information integration; the improved U 2 -Net network with U 2 -Based on the nested U-shaped structure of Net, deconvolution is added at the input of each parallel convolutional layer of the symmetric encoder-decoder after the input convolutional layer, and an SE attention module is embedded at the output of each parallel convolutional layer; the SE attention module compresses the input features through global average pooling and applies attention weights on the compressed channel dimension. After generating the fusion guiding feature, the feature maps of each branch are concatenated and stitched together along the channel dimension to form the initial dynamic fusion feature. The dynamic fusion feature is then adjusted for feature mapping through a two-branch convolutional path. The improved U 2 The -Net network includes a feature extraction module, a feature enhancement module, and a feature fusion module; The feature extraction module is used to extract multi-scale edge response features of anime-style images through parallel convolutional paths, generating hierarchical feature maps with spatial awareness. The multi-scale edge response features are used to represent low-order and high-order features of the image from different receptive fields. The low-order features include the position, direction, and connectivity information of the contour lines, and the high-order features include the semantic information of the background and the character. The feature enhancement module is used to perform deconvolution reconstruction on high-order features in the hierarchical feature map, and then concatenate the reconstructed high-order features with adjacent low-order features. The concatenated features are fused at the pixel level across layers using point convolution, while the edge distortion of high-order features is corrected by the geometric constraints of low-order features, resulting in complementary enhanced feature representations. The feature fusion module is used to dynamically fuse complementary enhanced feature representations to obtain multi-scale and multi-semantic level feature responses that integrate pixel-level information. The line art extraction module is used to output editable line art by integrating multi-scale and multi-semantic level feature responses with pixel-level information.

8. A computer device, comprising a memory, a processor, and a computer program stored in the memory, characterized in that, The processor executes the computer program to implement the steps of the method according to any one of claims 1 to 6.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is loaded by the processor, it is able to perform the steps of the method according to any one of claims 1 to 6.

Images