Chinese landscape painting style migration method giving consideration to structure retention and stroke details

Abstract

The invention relates to the field of image processing and artificial intelligence, discloses a Chinese landscape painting style migration method considering structure retention and stroke details, and aims to solve the problem that content structure retention and fine artistic style migration are difficult to consider in an existing method. The method comprises the following steps: respectively capturing global long-distance structural dependence and local detail features of an input image in a parallel mode by adopting a double-branch encoder consisting of a bidirectional state space modeling branch and a self-attention branch; self-adaptive information compensation and enhancement are performed on a feature missing region caused by sparse strokes or large-area blank leaving through a multi-scale context sensing module; a dual-path decoder formed by a spatial hierarchical state space modeling branch and a trans-attention branch is utilized, the spatial hierarchical state space modeling branch is responsible for faithful reconstruction of a content structure under the guidance of style features, and the trans-attention branch is responsible for accurate injection of style strokes and textures; and finally, fusing the output of the dual-path decoder, and generating a stylized image through a convolution decoder.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of image processing and artificial intelligence, in particular to a Chinese landscape painting style transfer method considering structure reservation and brush stroke details. BACKGROUND

[0002] Image style transfer is an important research direction in the field of computer vision and artificial intelligence, and its core goal is to combine the semantic structure of a content image with the artistic features (such as color, brush stroke, texture, etc.) of another style image, thereby generating a new artistic image that integrates the characteristics of both. This technology has broad application prospects in artistic creation, digital entertainment, personalized design, augmented reality and many other scenarios. With the rapid development of deep learning, especially convolutional neural networks, image style transfer technology has made great progress. The current mainstream method, especially arbitrary image style transfer, usually decouples and recombines content and style information in the deep feature space. The core idea of this method is to extract feature maps of content and style images through a pre-trained deep network, then align the statistical properties (such as mean, variance, etc.) of the content feature map to those of the style feature map through a specific feature transformation operation, and finally reconstruct the transformed feature map into the final stylized image through a decoder.

[0003] However, while pursuing universality and efficiency, existing technologies also expose their inherent limitations. When performing feature transformation, these methods often use a global or semi-global statistical alignment strategy. This strategy can effectively capture the overall tone and general texture of the style image, but in essence it treats the style as a homogeneous, globally applicable "texture paste". When faced with artistic styles with rich and delicate local details, clear brush stroke directions, and complex compositions (such as Chinese landscape paintings, impressionist paintings, etc.), it is difficult to capture and reproduce those non-homogeneous local brush strokes and texture details that define the soul of the artwork, resulting in generated images that lack the essence and charm of the original work. More critically, this relatively "rough" statistical remodeling of content information in the feature space inevitably interferes with and destroys the original spatial structure information in the feature map. In order to forcibly inject style, the model often sacrifices the structural integrity of the content, resulting in the final generated image appearing distorted, blurred, or semantically distorted, making the original object outlines in the content image unclear and the structural relationships chaotic. This inherent contradiction between content structure preservation and fine style injection is a core challenge faced by existing technologies.

[0004] Furthermore, existing methods often misinterpret these areas for art styles that feature "blank space" or sparse brushstrokes as important aesthetic characteristics (such as ink painting). Models may interpret large areas of blank space as useless information, or produce artifacts due to insufficient contextual information when dealing with sparse but crucial brushstrokes. Summary of the Invention

[0005] To address the shortcomings of existing technologies, this invention provides a method for transferring the style of Chinese landscape painting that balances structural preservation and brushstroke details. The technical problem this invention aims to solve is that existing image style transfer methods struggle to maintain both the faithful preservation of content structure and the accurate reproduction of style details when transferring art styles with complex brushstrokes and textures, often resulting in distorted content or coarse style in the generated images.

[0006] To achieve the above objectives, this invention provides a method for style transfer in Chinese landscape painting that balances structural preservation and brushstroke details. This method first employs a dual-branch encoder to process the input content image features and style image features in parallel. The dual-branch encoder includes a bidirectional state-space modeling branch and a self-attention branch. The former captures the global layout and long-range dependencies of the image, while the latter focuses on extracting local texture and brushstroke details. The output features of the two branches are fused to obtain an encoded representation that balances global structure and local details.

[0007] Next, the method employs a dual-path decoder to process the aforementioned encoded representation. This dual-path decoder comprises a spatial hierarchical state-space modeling branch and a cross-attention branch. The spatial hierarchical state-space modeling branch is used to maintain the spatial consistency and structural stability of the image during decoding, while the cross-attention branch is responsible for accurately injecting the details and color information from the style image into the content structure. The output features of the two paths are fused to obtain the final decoded representation.

[0008] Finally, the decoded representation is reconstructed into a final stylized image that retains the original structure of the content image while incorporating the artistic features of the stylized image through a convolutional decoder.

[0009] In a preferred embodiment, in the dual-branch encoder, the bidirectional state-space modeling branch propagates gating information and aggregates context along the horizontal and vertical directions through a two-dimensional state scanning mechanism. Its state update process can be described by the following bidirectional state-space equation: ; ; Among them, and

[0010] These are the hidden states for forward and backward propagation, respectively. , Here is the state transition matrix. , For the input matrix, It is a location The input features. In this way, long-range dependencies in images can be effectively modeled, preserving the spatial layout of foreground and background in landscape paintings.

[0011] In a preferred embodiment, the dual-branch encoder models the output features of the bidirectional state space branches. Output features of self-attention branches Through a learnable fusion weight factor Weighted fusion is performed to generate a unified encoding representation. : ; This design achieves a dynamic balance between global structural information and local texture details.

[0012] As a preferred implementation, the encoded representation is first processed by a multi-scale context-aware module before entering the dual-path decoder to compensate for the loss of texture information caused by the sparse brushstrokes and blank areas commonly found in Chinese landscape paintings, and to enhance the hierarchical style expression.

[0013] In a preferred embodiment, the multi-scale context-aware module employs different dilation rates. Parallel processing of convolutional layers to encode representations To extract multi-scale contextual features Subsequently, an adaptive gating mechanism is used to control the original encoded representation and the fused multi-scale features. Weighted fusion is performed. This gating mechanism generates a spatially sensitive weighted graph. Final output The calculation method is as follows: ; ; In a preferred embodiment, in the dual-path decoder, the spatial hierarchy state space modeling branch constructs a spatial structure representation using content features, while its state propagation process is dynamically guided by style features. Specifically, its state update equation is: ; in, This is the current state. The input is the feature vector. The state transition matrix is ​​also included.

[0014] and input matrix It is based on style characteristics It is dynamically generated, thus achieving style adaptation while maintaining global structural consistency.

[0015] In a preferred embodiment, in the dual-path decoder, content features are processed across attention branches. As a query, style features Cross-modal attention computation is performed, acting as both key and value. This allows the model to selectively and precisely inject local details, colors, and brushstroke patterns from stylized images into the corresponding content structure regions.

[0016] In a preferred embodiment, the dual-path decoder models the output features of the spatial hierarchy state space branch. Output features across attention branches Through a learnable fusion factor Weighted fusion is performed to obtain the decoded representation. : ; In a preferred embodiment, the method optimizes the entire network using a joint sensing loss function. This loss function consists of content loss... Style loss and loss of identity , Weighted composition: ; Among them, content loss is used to measure the perceptual similarity between the generated image and the content image; style loss evaluates the consistency of style distribution by matching the statistics of feature maps; and identity loss enhances the stability of training by constraining the model to output the original image when given the same content and style images.

[0017] This invention provides a method for transferring the style of Chinese landscape painting while preserving structural details and brushstroke details. It has the following beneficial effects: 1. By introducing a bidirectional state-space modeling branch into the encoder, this invention can capture and maintain long-distance dependencies of the content image from a global perspective, effectively "understanding" the overall layout and skeleton of the image. Simultaneously, during the decoding stage, a dedicated spatial-level state-space modeling path focuses on faithfully reconstructing the structure, ensuring that key geometric shapes and spatial relationships are not distorted during the feature-to-pixel conversion process. This end-to-end specialized processing of structural information fundamentally avoids content distortion caused by style injection, allowing the final image to retain a clear and discernible original outline and semantics while carrying a new style.

[0018] 2. Unlike traditional methods that rely on macroscopic statistical imitation of style, this invention establishes a fine correspondence between content location and stylistic details through cross-attention branches in its dual-path decoder. It allows the model to "query" the desired style for each pixel of the content area and "retrieve" the most matching brushstrokes, textures, or color patterns from style features for injection. This enables the invention to go beyond simple tone transfer, truly reproducing the subtle, heterogeneous artistic brushstrokes of the original work, resulting in landscape paintings with visible variations in ink density and brushstroke direction, rather than a blurry texture overlay.

[0019] 3. This invention can more intelligently handle art styles with sparse features or large areas of blank space. For art forms like Chinese landscape painting, which often employs the principle of "using white space as black," the multi-scale context-aware module plays a crucial role. It actively detects sparse areas in the feature map through parallelized dilated convolutions and, combined with an adaptive gating mechanism, determines which areas require contextual information compensation. This design allows the model to avoid incorrectly filling large areas of blank space with unnecessary textures while also supplementing sparse but crucial brushstrokes with context, thus accurately reproducing the original work's balanced composition and avoiding artifacts.

[0020] 4. This invention achieves more stable and controllable stylization capabilities by decoupling the structure and style processing paths in the core architecture. From the dual-branch parallel analysis of the encoder to the dual-path separation reconstruction of the decoder, this invention consistently treats "preserving structure" and "injecting style" as two related yet independent objectives. This clear division of responsibilities greatly reduces mutual interference between the two tasks at the feature level, ensuring that style transfer does not come at the expense of content. Therefore, this invention exhibits stronger robustness when dealing with diverse combinations of content and style, and the generated results are cleaner and of higher quality.

[0021] 6. This invention enhances the color fidelity and training stability of the model through a unique identity loss function design. By introducing two identity loss terms, this invention imposes strong constraints on the model during training. It requires the model to output the original image unchanged when receiving input with the same content and style. This mechanism not only effectively suppresses unnecessary image distortion, but more importantly, it forces the model to learn the ability to accurately preserve and reconstruct the color distribution of style images, thereby effectively avoiding the common color cast problem in the final generated image. Attached Figure Description

[0022] Figure 1 This is a system overall block diagram of the present invention; Figure 2This is a schematic diagram of the dual-branch encoder of the present invention; Figure 3 This is a schematic diagram of the structure of the multi-scale context-aware module of the present invention; Figure 4 This is a schematic diagram of the dual-path decoder of the present invention. Detailed Implementation

[0023] The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0024] Please see the appendix Figure 1 To be continued Figure 4 This invention provides a method for style transfer in Chinese landscape painting that balances structural preservation and brushstroke details. The method includes the following main steps: First, a dual-branch encoder is used to process the input content image features and style image features in parallel to generate a fused encoded representation that captures both global structural information and local texture details. Next, a dual-path decoder is used to process the encoded representation, and while maintaining the consistency of the content structure, the artistic features of the style image are precisely injected to obtain a decoded representation. Finally, a convolutional decoder is used to reconstruct the decoded representation into the final stylized image.

[0025] Specifically, in the initial stage of method execution, a content image is... and a style image As input, these two images are first processed by a pre-trained feature extraction network (e.g., the VGG-19 network) to extract multi-level feature maps, yielding content features. and style characteristics These feature maps will serve as input for subsequent core modules.

[0026] One of the core elements of this method is the dual-branch encoder. This encoder receives content features. With style characteristics It processes data in parallel through two internal functional branches. One branch is a bidirectional state-space modeling branch, which focuses on capturing and maintaining the macroscopic, global structural layout of the image by modeling long-range dependencies in the feature maps. This is crucial for maintaining the spatial relationship between distant mountains and nearby water in landscape paintings. The other branch is a self-attention branch, which focuses on extracting fine, local textures and brushstroke patterns from the image, capturing the artistic details of the stylistic image.

[0027] After processing through the two branches described above, a feature map representing the global structure is obtained. and feature maps representing local textures To generate a comprehensive coded representation, these two feature components are weighted and fused in the following manner to produce a unified coded representation. :

[0028] in, It is a fusion weight factor that is automatically learned during network training. It can dynamically adjust the contribution ratio of global structural information and local detail information in the encoding process according to the characteristics of the input image, thereby achieving an adaptive balance between the two.

[0029] Subsequently, the encoded representation generated by the aforementioned steps The data is fed into a dual-path decoder for processing. The design of this decoder is another core element of the invention, and it also contains two cooperating paths. The first path is a spatial hierarchy state space modeling branch, whose core task is to use content features to guide the state propagation process, thereby ensuring that the spatial structure of the generated image is strictly aligned with the structure of the original content image during image decoding and reconstruction, preventing structural distortion or misalignment.

[0030] Meanwhile, the second path in the dual-path decoder is the cross-attention branch. This branch selectively and precisely "draws" key artistic elements unique to the style image, such as brushstroke details, color distribution, and ink density, onto the content structure framework maintained by the first path by establishing cross-modal associations between content features and style features.

[0031] These two paths respectively generated spatial structure preservation features. Injecting features with stylistic details The two components are combined using the following weighted fusion method to form the final decoded representation. :

[0032] in, It is also a learnable fusion factor, responsible for finely balancing the rigidity of structural preservation with the flexibility of style injection during the decoding stage, ensuring the consistency of the final output.

[0033] Finally, the high-dimensional decoded representation output by the dual-path decoder. The data is fed into a standard convolutional decoder. This decoder, through a series of upsampling and convolution operations, gradually restores and reconstructs the abstract feature map into a pixel-level image, ultimately generating a stylized image that retains the core structure of the content image while also imbuing it with the artistic charm of the stylized image. .

[0034] In a specific embodiment of the present invention, before feeding the content features and style features into the dual-branch encoder, these two-dimensional feature maps need to be preprocessed and serialized. The goal of this step is to transform the spatially structured two-dimensional feature data into a one-dimensional vector sequence suitable for subsequent state-space modeling and self-attention mechanism processing.

[0035] This invention employs a convolution-based image patch embedding method to achieve this transformation. Specifically, this method takes the input feature map, whether it contains content features, and embeds it into a patch embedding method.

[0036] or style characteristics Logically divided into a series of non-overlapping elements of size The image is divided into blocks. This partitioning method ensures that the local neighborhood information of the image is completely encapsulated within each image block. This partitioning and embedding process is efficiently implemented through a single convolution operation. This convolutional layer has specific parameter settings: its kernel size and stride are both set to the size of the image block. Equal. This operation will make each size equal. And the number of channels is Image patches, independently linearly mapped to a dimension of The embedding vector. This process can be viewed as extracting features and reducing or increasing the dimensionality of each image patch to obtain a more compact or richer vector representation.

[0037] Let the input feature map be Its spatial dimension is The number of channels is After the above convolution operation, a dimension of is generated. The new feature map. Among them, and These are the height and width of the input feature map, respectively. It is the size of the image patch. This is the number of input channels. This is the dimension of the target embedding space. It is also the basic feature dimension for subsequent encoder processing.

[0038] Finally, to form a standard one-dimensional sequence, the spatial dimensions of this new feature map are flattened. Specifically, it is reshaped from a three-dimensional tensor into a two-dimensional matrix, the final form of which is a matrix of length [missing information]. sequence , where the sequence length This refers to the total number of image patches. Each element in the sequence is a sequence with dimension 1. The feature vector represents a local region of the original image. This sequence This is the final input of the dual-branch encoder, which retains all the information of the original image and is organized into a format suitable for sequence model processing.

[0039] In this embodiment, the feature sequence obtained after image preprocessing and serialization is fed into a dual-branch encoder for deep feature extraction and encoding. The design of this encoder is one of the key innovations of this invention. It uses a parallel dual-branch structure to process and extract information from different levels of the input features to generate a highly balanced encoded representation with rich information dimensions.

[0040] The first branch of this bi-branch encoder is a bidirectional state-space modeling branch. The core purpose of this branch is to model the global structure and long-range dependencies of the image. Specifically, it employs a two-dimensional state scanning mechanism to scan and propagate information across the input feature sequence in multiple directions, including horizontal and vertical. In each scanning direction, it aggregates contextual information through a state-space model. This process can be described by the following set of bidirectional state-space equations:

[0041]

[0042] in, Represents the position in the input sequence eigenvectors; and These represent the positions during forward and backward propagation, respectively. The hidden state; and It is the state transition matrix, which determines how historical information is remembered and transmitted; and The input matrix determines how the current input affects the hidden state. Through this bidirectional propagation and aggregation, this branch can effectively capture macroscopic layout information that requires a large receptive field to perceive, such as mountain ranges and the horizon line, and encode this information into its output features. middle.

[0043] In parallel with the bidirectional state-space modeling branch, the encoder also includes a second branch, a self-attention branch. This branch focuses on capturing local texture and fine brushstroke details of the image. It dynamically aggregates information by calculating the correlation score between any two locations in the sequence. Its core computation process is as follows:

[0044] In this calculation, the input sequence is first linearly projected into three distinct matrices: the query matrix and the input sequence. Key matrix Sum matrix . The dimension representing the key vector is used to scale the dot product result to ensure training stability. This attention mechanism allows the model to focus on the brushstrokes, colors, or texture patterns of other most relevant regions in the style image when processing each image patch, thereby effectively extracting highly discriminative local style elements. This branch ultimately outputs a feature map representing local details. .

[0045] After obtaining the features representing the global structure respectively and features representing local details Subsequently, this invention employs an adaptive fusion mechanism to combine the two. This fusion utilizes a learnable fusion weight factor. To achieve this, the specific fusion operation is defined as follows:

[0046] Here, It is a scalar weight that is optimized along with other parameters during network training. This design allows the model to autonomously determine the global structural information and local detail information in the final encoded representation based on the specific characteristics of the input content and style image. The relative importance of each element. For example, when dealing with complex landscape paintings, the model might assign... Greater weight is given to brushstrokes; however, when dealing with close-ups rich in brushstrokes, more emphasis may be placed on... From this, we obtain It is a high-quality fusion encoding representation that takes into account both macroscopic layout and microscopic texture.

[0047] Furthermore, in another embodiment of the present invention, to further enhance the model's ability to understand the sparse brushstroke layout and large areas of "blank space" unique to Chinese landscape painting, a multi-scale context-aware module can be inserted between the dual-branch encoder and the dual-path decoder. The function of this module is to provide targeted information compensation during the feature encoding stage for areas where information may be insufficient due to sparse brushstrokes.

[0048] This module receives the unified encoded representation generated by the dual-branch encoder. As input, it first employs a set of parallel convolutional layers with different dilation rates to encode the input representation. Processing is performed. Each convolutional layer undergoes dilation due to its dilation rate. Different features have receptive fields of varying sizes, enabling them to capture contextual information from feature maps at different scales and distances. In this way, the model can simultaneously focus on nearby texture details and distant structural relationships. These parallel-extracted multi-scale features are then fused, for example, through feature concatenation followed by a convolutional layer, to obtain a feature map that aggregates rich contextual information. .

[0049] However, not all regions in an image require equal intensity of contextual information supplementation. To achieve selective supplementation of contextual information, rather than a global forced overlay, this invention then employs an adaptive gating fusion mechanism. This mechanism first applies the original encoded representation... Through a single convolution operation and activation function, a spatially sensitive weight map with the same spatial dimensions as the feature map is dynamically generated. Its generation method is as follows:

[0050] in, This represents a convolutional kernel size of convolutional layers, This represents the Sigmoid activation function. After processing with the Sigmoid function, the weight map... The value of each element is constrained to between 0 and 1, allowing it to function as a flexible “switch” or weight.

[0051] Ultimately, the output characteristics of this module The following formula is used to calculate the weighted graph. Original encoding representation and aggregated multi-scale contextual features Perform element-wise weighted summation:

[0052] In this formula, " "" represents element-wise multiplication. The significance of this operation is: for a weighted graph Regions with a median close to 1 indicate that the model needs more contextual information to make judgments about those regions. Therefore, the output features will be more biased towards using features that incorporate multi-scale information. Conversely, for In regions where the median is close to 0, the model will retain more of the original encoded representation. This avoids unnecessary feature perturbations. The output features after processing by this module... It will replace the original This serves as the input for the subsequent dual-path decoder.

[0053] In one embodiment of the present invention, the encoded representation generated by the aforementioned encoding stage is fed into a dual-path decoder for processing. The core function of this decoder is to simultaneously perform two key tasks—structural reconstruction and style injection—in two parallel paths, and to generate a high-quality decoded representation for final image reconstruction by fusing the outputs of both.

[0054] The first path of this decoder is the spatial hierarchical state-space modeling branch. Its core task is to maintain and reconstruct the spatial structural consistency of the original content image. During decoding, this branch receives the encoded representation as input and reconstructs the information through a serialized state-space model. A key innovation is that the core parameters in its state propagation process, such as the state transition matrix, are not fixed but dynamically generated and modulated by style features. This means that style information is used as a high-level guide to influence the way the structure is reconstructed, but fundamentally, the computation process of this branch still prioritizes maintaining the inherent spatial layout of the content image. Thus, this branch can output a structural feature map that is highly faithful to the original content in space. .

[0055] Operating in parallel with the aforementioned path is the second path of the decoder: the cross-attention branch. This branch complements the former, focusing not on structure preservation but on selectively injecting key artistic features from the style image, such as brushstrokes, ink tones, and colors. In this embodiment, it uses content features (or encoded features) as queries and style features as keys and values ​​to perform cross-modal attention computation. Through this mechanism, the model can "query" the feature library of the style image for each local region of the content image and find the most matching style texture and color pattern, then aggregate this style information into the current region. This process ensures that style injection is not random or globally ambiguous, but rather a precise local "drawing" closely related to the content structure. This branch ultimately outputs a feature map rich in style details. .

[0056] Feature maps faithful to the structure were obtained respectively. and stylistic feature maps Then, the present invention fuses the two feature maps. This fusion process is also driven by a learnable fusion factor. The specific calculation method for control is as follows:

[0057] Here, the fusion factor Optimized during network training, its role is to assign dynamic weights to the objectives of "preserving structure" and "injecting style" in the final stage of decoding, achieving a fine dynamic balance between structural rigidity and style flexibility. The final generated decoded representation... The skeleton of the content image was preserved.

[0058] In this embodiment, the decoded representation output by the dual-path decoder It is a high-dimensional feature map containing rich structural and stylistic information, but it does not yet have a visual form as an image. Therefore, a subsequent stylized image reconstruction step is needed to transform this abstract feature representation and generate the final pixel-level image.

[0059] In this invention, this reconstruction task is performed by a convolutional decoder. This convolutional decoder is structurally the inverse of a typical feature encoding network. Its core function is to gradually increase the spatial resolution of the feature map while reducing its channel dimensions through a series of upsampling and convolution operations, ultimately restoring a standard three-channel color image.

[0060] Specifically, decoding means The data is first fed into the first layer of this convolutional decoder. This decoder consists of multiple stacked processing units, each typically containing an upsampling layer and one or more convolutional layers. The upsampling layer is responsible for expanding the height and width of the feature map, for example, through transposed convolution or pixel rearrangement. The subsequent convolutional layers then process and refine the features at a new, larger spatial scale, smoothing out artifacts that upsampling might introduce and beginning to reconstruct the local details of the image.

[0061] This process is repeated step-by-step. At each level, the spatial size of the feature map is further enlarged, and the feature information it contains is gradually transformed into more specific image textures and color representations. This step-by-step reconstruction process from abstract features to specific pixels ensures the richness of detail and visual realism of the final generated image.

[0062] At the end of the reconstruction process, a final convolutional layer is set. The number of kernels in this layer is set to 3, corresponding to the three color channels of a standard RGB color image. Its function is to map the multi-channel feature map output from the previous layer into a three-channel tensor. Subsequently, an activation function (such as the Tanh function) is used to normalize the value of each element in the output tensor to a standard range of -1 to 1 or 0 to 1, so that it can be correctly interpreted and displayed as an image. The final output of this step is the final product of this invention—a stylized image that balances structural preservation and brushstroke detail. .

[0063] In this embodiment, to enable the network model proposed in this invention to learn the accurate mapping relationship from content images and style images to the final stylized image, an optimization process is needed to train its internal learnable parameters. This training process is guided by a carefully designed joint perceptual loss function.

[0064] The joint sensing loss function It is a composite function, consisting of content loss. Style loss and two identity losses , The weighted summation is used to construct the formula. Its overall form is as follows:

[0065] in, , , and These are all preset non-negative weighting coefficients used to balance the contribution of each loss term to the total loss, thereby collaboratively guiding the optimization direction of the model. Content loss The goal is to ensure that the final stylized image is generated. In terms of macrostructure and high-level semantics, it is similar to the original content image. To maintain consistency. This loss is passed through a pre-trained feature extraction network. (For example, the VGG-19 network) to calculate. Specifically, and Enter the network separately In this process, the feature map of a specific convolutional layer is extracted, and then the Euclidean distance or mean square error between the two feature maps is calculated.

[0066] Style loss The purpose is to measure the generated stylized images. In terms of artistic style, it is similar to the target style image. The degree of similarity is determined not by comparing pixel values, but by comparing their statistical distributions in the feature space. In practice, a feature extraction network is also used. extract and Feature maps are generated across multiple convolutional layers of varying depths, and a Gram matrix is ​​calculated for each feature map. The style loss is the sum of the differences between the Gram matrices of the corresponding layers. The Gram matrix effectively captures style information such as texture, color distribution, and brushstroke patterns in an image.

[0067] Loss of identity and It is introduced as a key regularization term, with the main purpose of stabilizing the training process and constraining the model's ability to preserve color and structure.

[0068] Specifically, loss of identity The calculation method is as follows: the content image Simultaneously, these elements serve as both content and style inputs, fed into the network model of this invention for processing. Theoretically, the model should then output the original image. This loss term is the difference between the model's actual output and the original image. The pixel-level differences between them. This constrains the model to avoid unnecessary distortion of the content image when there is no need for style transfer.

[0069] Similarly, loss of identity The calculation method is as follows: the style image Simultaneously, both the content and style inputs are fed into the model. At this point, the model output should match the original image. Maintain consistency. This loss term is the difference between the model's actual output and the original image. The pixel-level differences between them. It is mainly used to constrain the model to completely maintain and reconstruct the color distribution of the style image, and to prevent unwanted color shifts during the stylization process.

[0070] In this embodiment, the implementation and verification of the method of the present invention are carried out in a specific hardware and software environment. The hardware platform uses a server configured with a high-performance graphics processor (GPU, such as NVIDIA Tesla V100 or higher). The software environment is based on the Ubuntu 20.04 operating system, uses Python 3.8 as the programming language, and utilizes the PyTorch 1.10 deep learning framework to build, train, and test the network model proposed in this invention.

[0071] The training dataset in this embodiment consists of two parts. The content image dataset uses the publicly available large-scale image dataset MS-COCO, from which approximately 80,000 images of natural landscapes and architecture with diverse scenes and clear structures are selected. The style image dataset is a specially collected and organized dataset of Chinese landscape paintings, containing approximately 1,500 high-resolution scanned images of landscape paintings by famous artists from different dynasties and schools. Before training, all content images and style images were uniformly resized to 256x256 pixels and normalized.

[0072] During the model training phase, the Adam optimizer is used to iteratively optimize the network parameters, with the learning rate set to a fixed, small value, such as 1e-4, and the momentum parameter... and Set them to 0.9 and 0.999 respectively. The batch size during training is set to 8 based on the GPU memory size.

[0073] In the joint perception loss function, the weight coefficients of each term are set to specific values ​​to balance different optimization objectives. In this embodiment, the content loss weight... Set to 1.0, style loss weights Setting it to 10.0 gives it relatively higher importance to ensure sufficient style learning. Simultaneously, to enhance the model's ability to preserve color and structure, the identity loss weights are adjusted. and The values ​​were set to 50.0 and 1.0 respectively. The entire network model underwent approximately 100,000 iterations of training under the above dataset and parameter configuration until the loss function converged and the model reached stable performance. After training, the model can be used to stylize input images of any size.

[0074] Although embodiments of the invention have been shown and described, it will be understood by those skilled in the art that various changes, modifications, substitutions and alterations can be made to these embodiments without departing from the principles and spirit of the invention, the scope of which is defined by the appended claims and their equivalents.

Claims

1. A Chinese landscape painting style transfer method that balances structure preservation and brushstroke details, characterized by, The method comprises the following steps: The content image features and the style image features are processed by a double-branch encoder to obtain a fused encoding representation; The double-branch encoder comprises a bidirectional state space modeling branch for capturing global structure information and a self-attention branch for extracting local texture features; The encoding representation is processed by a double-path decoder to obtain a decoding representation; The double-path decoder comprises a spatial hierarchical state space modeling branch for maintaining spatial consistency and a cross-attention branch for injecting style details; The decoding representation is reconstructed into a final stylized image by a convolution decoder.

2. The method of claim 1, wherein, The bidirectional state space modeling branch performs gate information propagation and context aggregation along the horizontal and vertical directions through a two-dimensional state scanning mechanism to model long-range dependencies in the image.

3. The method of claim 1, wherein, The step of processing the content image features and the style image features by the double-branch encoder specifically comprises: weighting and fusing the output features of the bidirectional state space modeling branch and the self-attention branch by a learnable fusion weight factor to obtain the encoding representation.

4. The method of claim 1, wherein, Before the double-path decoder is used, the encoding representation is further processed by a multi-scale context perception module to compensate for texture information in sparse areas and enhance hierarchical style performance.

5. The method of claim 4, wherein, The multi-scale context perception module specifically processes in the following manner: parallel processing of the encoding representation by convolution layers with different inflation rates to extract multi-scale context features; and weighting and fusing the original encoding representation and the multi-scale context features by an adaptive gating mechanism.

6. The method of claim 1, wherein, The spatial hierarchical state space modeling branch specifically processes in the following manner: constructing a spatial structure representation using content features and dynamically guiding a state propagation process by style features to maintain global structure consistency while adapting to style.

7. The method of claim 1, wherein, The cross-attention branch specifically processes in the following manner: performing cross-modal attention calculation by taking content features as queries and style features as keys and values to selectively inject local details, colors and stroke patterns of the style image into the content structure.

8. The method of claim 1, wherein, The step of processing the encoding representation by the double-path decoder specifically comprises: weighting and fusing the output features of the spatial hierarchical state space modeling branch and the cross-attention branch to obtain the decoding representation.

9. The method of claim 1, wherein, The method further comprises optimizing the network by a joint perception loss function; the joint perception loss function comprises a content loss, a style loss and an identity loss.

10. The method of claim 9, wherein, The content loss is used to measure the perceptual similarity between the generated image and the content image; the style loss is used to evaluate the feature distribution consistency between the generated image and the style image; and the identity loss is used to constrain the model to output the original image when the same content and style images are input.

Images