Real ink mark driven copy authentic method based on submerged space diffusion

Abstract

The invention discloses a real ink mark driven facsimile authentic method based on submerged space diffusion, which comprises the following steps: firstly, collecting a target facsimile image and a large number of real ink mark sample images, carrying out preprocessing operation on the images, and constructing a standardized training set and a test set; secondly, extracting a contour diagram and a skeleton diagram from the target copy image, and mixing the contour diagram and the skeleton diagram in a linear mode to generate a continuous structural feature map; constructing an ink mark encoder, encoding the authentic mark sample, and extracting the pen and ink characteristics of the authentic ink mark sample; and finally, constructing a reinking network based on a latent space diffusion model, and finally converting the optimized data representation into a pixel image through a decoder to obtain a calligraphy image with natural ink marks and real strokes. The problems that in the prior art, copy images are inflexible in ink color and lack of natural ink color change are solved, meanwhile, stability of a font skeleton and refinement of the stroke of the to-be-restored area are guaranteed, and authentic transformation work is achieved.

Description

Technical Field

[0001] This invention belongs to the field of image processing technology, specifically relating to a method for real ink stain-driven copying based on latent space diffusion. Background Technology

[0002] Calligraphy, a historical treasure of Chinese culture, is not only an important way for ancient literati to express their emotions, but also a vital carrier of traditional Chinese culture. It embodies profound historical accumulation and unique aesthetic charm, and constructs a spiritual coordinate of Eastern art through its distinctive brush and ink language. Some extant masterpieces of calligraphy have been destroyed, leaving only copies made using the tracing and filling-in method. Because the filled-in ink lacks natural variations in ink density, it hinders the study of calligraphy. Examples include Wang Xizhi's Preface to the Orchid Pavilion and many of his letters, as well as some of Yan Zhenqing's calligraphic works. Although the tracing and filling-in method can preserve the original appearance of the work to the greatest extent possible at the level of character structure, its unnatural ink blocks affect both aesthetics and the artistic quality of the natural flow of ink in calligraphy, thus weakening its artistic expression and aesthetic appeal.

[0003] With the development of cultural relic digitization and computer vision technology, the realistic transformation and restoration of ancient calligraphy rubbings has become an important research area for the digital protection and dissemination of cultural heritage. However, in response to the aforementioned problems, traditional image restoration techniques often rely on probabilistic inference based on information surrounding the damaged area, making it difficult to accurately capture the unique artistic expression of calligraphy ink marks. Traditional style transfer methods, when processing calligraphy images, often neglect the constraints of the skeleton and fail to decouple the character structure from the ink style, thus easily leading to character deformation, stroke misalignment, and difficulty in learning the changes in natural ink marks, resulting in stiff transfer effects.

[0004] In recent years, diffusion models have made significant progress in image generation. Based on a probabilistic modeling mechanism of progressive noise addition and denoising, they exhibit higher training stability and better conditional control compared to traditional methods. Based on this, this invention proposes a method for reproducing authentic works of calligraphy driven by real ink marks based on latent space diffusion. This method learns the natural variations in ink color from authentic works of ancient calligraphers, combines this with the stroke features of the work to be restored, and performs style transfer through a diffusion model. This method combines the character structure of the reproduction with the natural ink color variations of the authentic work, resulting in a final image that improves upon the appearance of dead ink, producing a reproduction image with natural ink color and enhancing the aesthetic appeal of the reproduction. Summary of the Invention

[0005] The purpose of this invention is to provide a method for transforming rubbings into authentic works based on real ink stains driven by latent space diffusion. This method solves the problems of rigid ink color and lack of natural ink color variation in existing rubbing images, while ensuring the stability of the character skeleton and the refinement of the brushstrokes in the area to be repaired, thus realizing the transformation into authentic works.

[0006] The technical solution adopted in this invention is a method for authenticating a copy based on real ink stains driven by latent space diffusion, which is implemented according to the following steps: Step 1: Collect target rubbing images and a large number of real ink sample images, perform preprocessing operations on the images, and construct standardized training and testing sets; Step 2: Extract the contour map and skeleton map from the target copy image, and mix them in a linear manner to generate a continuous structural feature map; Step 3: Construct an ink encoder to encode the authentic ink samples and extract the ink features of the authentic ink samples. Step 4: Construct a re-inking network based on the latent space diffusion model. Finally, the optimized data representation is converted into a pixel image by the decoder to obtain a calligraphy image with natural ink marks and realistic brushstrokes.

[0007] The invention is further characterized in that, Step 1 is implemented in the following steps: Step 1.1: Collect images of the rubbings that need to be restored, and at the same time collect images of real ink marks from different calligraphers, including Wang Xizhi and Yan Zhenqing. The selected ink marks cover a variety of brush and ink effects, paper backgrounds and ink density to ensure the richness of the samples. Step 1.2: Preprocess the collected image data, specifically including: converting the image to grayscale, removing discrete noise using median filtering or nonlocal mean, cropping character blocks, and dividing the preprocessed dataset into training and test sets in an 8:2 ratio; Step 1.3: To improve the model's adaptability and generalization, data augmentation is performed on the processed images. During the transformation process, the glyph structure remains unchanged, providing effective data support for subsequent training.

[0008] Step 2 is implemented in the following steps: Based on the preprocessed copy image, key structural information is extracted using traditional image processing algorithms, and the outline of the strokes is identified using edge detection algorithms to generate a contour map. The image is processed using a skeleton extraction algorithm to obtain the centerline information of the strokes, i.e., the skeleton diagram. This represents the direction and general shape of the strokes, which is used to generate structural feature maps that meet the requirements of subsequent input. , skeleton diagram Contour drawing The formula for performing a linear combination and smoothing is as follows:

[0009] in, The standard deviation is The two-dimensional Gaussian kernel represents the processing operations used for noise removal and image smoothing. It is an adjustment parameter between 0 and 1, used to balance the importance of skeleton and contour in structural features.

[0010] Step 3 is implemented in the following steps: Step 3.1: The ink encoder uses a four-stage structure from shallow to deep to extract features layer by layer; Stage 1 captures the local texture and brush marks of Chinese character strokes; Stage 2 integrates multi-scale brush textures based on Stage 1, and the core of Stage 2 is to extract features through multi-scale feature fusion and attention mechanism; Stage 3 further extracts the light and heavy distribution features of strokes; the core of Stage 4 is to focus on the changes in ink density and the dependence of ink color at a distance. Step 3.2: Extract the brushstroke features of real ink samples.

[0011] Step 3.1 is implemented according to the following steps: Phase 1 captures the local texture and brush marks of Chinese character strokes. The input is a single-channel ink sample image, processed through the first layer. After small kernel convolution and group normalized GN processing, it is then passed through... of Hollow convolution further expands the sensory field to After that Convolutional downsampling, combined with group normalized GN and ReLU activation, outputs low-level detailed features. Stage 2 builds upon Stage 1 by integrating multi-scale brush textures. The core of Stage 2 lies in extracting features through multi-scale feature fusion and attention mechanisms. After processing through two residual blocks, each residual block contains two... Convolution, group normalization, and activation functions are applied; subsequently, the number of channels is expanded through channel-wise up-dimensional convolution. In the ASPP (Aperture Priority Process) module with a holed spatial pyramid pooling architecture, a multi-branch parallel structure is employed: branch A uses... Convolution extracts local features; branches B, C, and D use dilation values ​​of 6, 12, and 18, respectively. Dilated convolution captures brush textures at different scales, and branch E is combined after pooling. Convolution and upsampling are performed; the outputs of each branch are concatenated and then processed... Convolutional fusion and normalization, activation function processing; then, the CBAM attention mechanism is introduced, where channel attention is fused with global average pooling (GAP) and global max pooling (GMP) statistics, and channel weights are generated through a multilayer perceptron (MLP) and sigmoid activation. Spatial attention is then achieved through... Convolution processes the channel pooling results to generate spatial weights; then... Convolutional downsampling yields brush texture features Stage 3 involves further extracting the weight distribution features of the strokes, and then applying the features obtained in Stage 2... Through two layers Convolutional processing is performed, with each layer incorporating group normalization and ReLU activation. Then, global average pooling (GAP) is used to generate channel statistics, followed by MLP and sigmoid activation to generate gated vectors. These gated vectors are then broadcast to the spatial dimension for adaptive channel adjustment. Finally, 3×3 convolutional downsampling is used to obtain stroke weight distribution features. The core of Phase 4 lies in focusing on the variations in ink density and their dependence on ink color over long distances, highlighting key characteristics. through Large kernel convolution and normalization are performed, and then the processed features are flattened into a spatial sequence with positional encoding to form a sequence with positional information. The sequence then passes through a 1-2 layer Transformer encoder, each layer containing normalization, multi-head self-attention (MHA), dropout, residual connections, and a feedforward neural network (FFN). Finally, the features output by the Transformer are further processed... Convolution transforms the features back into spatial dimensions, yielding the ink color features. .

[0012] Step 3.2 is implemented according to the following steps: At the end of the encoder, the features from each stage are first... , and Adjusted to a uniform spatial size and number of channels, forming a consistent feature representation: among which, For the adjusted texture features, For the adjusted distribution characteristics of light and heavy strokes, This represents the adjusted ink color features, and then two parallel output branches are set up based on this. The overall ink color style path applies to the three features mentioned above respectively. , , Global average pooling is used to aggregate global information, resulting in three generalized feature vectors: This represents the overall style vector of the brush texture. A global distribution vector representing the weight of strokes. This is the overall feature vector representing the density of ink color; subsequently, these three vectors are concatenated sequentially to obtain the overall ink color style reference vector. As a reference for subsequently injecting the overall style, the relevant formula is as follows:

[0013]

[0014] Spatial feature paths that preserve local details will combine three features , , The features are concatenated along the channel dimension to form a comprehensive feature, and then fused and compressed through a convolution operation with a kernel size of 1 to generate a spatial reference feature containing local brushstroke details. The formula is:

[0015] in, Indicates use Perform a linear mapping on the convolution kernel. This is for splicing operations.

[0016] In step 4, the re-inking network is built based on the diffusion model framework, and the overall architecture is as follows: Figure 4 As shown, it consists of the following five core modules: VAE codec, multi-scale U-Net generation backbone, structure adapter, overall style injection module, and spatial ink adapter. These modules work together as follows: The VAE codec is used to establish a reversible mapping between the pixel domain and the latent space. The encoder converts the input image into a low-dimensional data representation, and the decoder can restore a high-quality image from this low-dimensional data, thereby effectively reducing the computational complexity of the subsequent generation process and preserving key visual information; the multi-scale U-Net architecture serves as the generation backbone, performing progressive denoising and image generation in the latent space; the structure adapter is used to introduce glyph skeletons and contours as structural prior constraints at different scales of U-Net; the overall style injection module is responsible for introducing the overall ink color variation style; and the spatial ink adapter provides local brushstroke details.

[0017] Step 4 is implemented in the following steps: Step 4.1: Obtain the structural feature map from Step 2. The input is processed by a VAE encoder and converted into a low-dimensional data representation, i.e., latent variables. Then Gaussian noise was added to the latent variables to obtain This serves as the input for subsequent denoising; the reference original image is input into the VAE encoder to obtain the latent variables corresponding to the original image. This serves as a style reference region for the original artwork in subsequent resampling strategies; simultaneously, to limit the resampling region, the feature map... Downsampling to the latent space size yields the mask. ; Step 4.2: During the step-by-step generation process, the structural adapter will introduce glyph constraints at different scales. Specifically, this involves: converting the skeleton diagram... and outline Adjust to the current scale, denoted as and Then, the adjusted skeleton diagram and outline The glyphs are concatenated and input into the structure adapter for processing. The structure adapter is a small convolutional mapping network. After processing by the structure adapter, the glyphs are incorporated into the features of the current encoder in an element-wise addition manner and provided to the decoder of the corresponding scale through skip connections, thereby maintaining the stability of the glyph structure at multiple scales. Step 4.3: In the decoder of U-net in the re-inking network, two types of style information are injected in parallel for each layer: one is the overall style injection, and the other is the local detail injection. Overall style injection: Injecting style reference vectors Expand to the same spatial size as the current decoder layer, then expand the vector... By splicing and fusing with the decoder features, the overall ink color style is injected; Local detail injection: injecting spatial-level details Adjust as needed to the current decoder layer size, then obtain the detailed features after adjustment. The input is fed into the ink adapter, which is a shallow convolutional mapping network, to obtain detailed features consistent with the number of channels in the decoding layer. and in the form of residuals Add back decoder features to restore both overall style and brushstroke details at the same scale; Step 4.4: During the iterative generation process, the area to be repaired is optimized by focusing on the latent variables of the original work. The noisy result obtained from upsampling is compared with the prediction result given by the network in the region to be repaired, according to the mask. Perform linear mixing to obtain a new low-dimensional data representation for this time step, then simulate forward diffusion to add noise to the result, and then repeat the above process; Step 4.5: Finally, the optimized latent variables are obtained. That is, the optimized low-dimensional data representation is restored into an image through the VAE decoder, and the obtained image is post-processed with contrast adjustment, edge smoothing and other processes to obtain a target image that combines the original character structure and natural ink effect. Step 4.6: Use a custom joint loss function, specifically:

[0018] in, For the overall joint loss function, To predict the loss for diffused noise, For style statistics loss, Indicates perceived loss. , , These are the corresponding weight hyperparameters; Predicted loss of diffused noise The constraint diffusion model can accurately predict the noise component in the latent variables. It is the core constraint for training the re-inking network, and is achieved by comparing the difference between the noise predicted by the network and the actual noise. The formula is as follows:

[0019] in, As initial latent variables, For noisy latent variables in the diffusion process, It is Gaussian noise. For the prediction noise of the diffusion network, This indicates conditions such as structure and ink marks. It is expressed as the L2 norm, or Euclidean norm, which measures the difference between predicted noise and actual noise in the formula; Style statistical loss By comparing the ink distribution patterns of the modified image and the real ink stains, the formula is as follows:

[0020] in, Let represent the mean and standard deviation of the image pixel distribution, respectively. To generate an image, An image of the original work for reference; Perceived loss Perceptual loss is calculated based on the difference between the generated map and the reference map in the feature space. The formula is:

[0021] in, It is the first pre-trained network such as VGG Layer feature mapping, These are the weighting coefficients for each layer; the above loss functions are combined in a weighted manner to guide the training process of the model, so as to achieve a balance between structural accuracy, visual quality and natural ink color.

[0022] The beneficial effects of this invention are that it uses a method for reproducing authentic ink marks driven by latent space diffusion. This method learns the natural variations in ink color from authentic works of ancient artists and combines this with the stroke features of the work to be restored to perform style transfer. First, the character skeleton and outline are extracted from the authentic image and synthesized into a structural feature map as the character framework. This map is then converted into low-dimensional data and input into the generation model. Simultaneously, feature encoding is performed on authentic samples to learn the natural ink color variation patterns as a style reference. During the generation process, a multi-scale adapter injects character structure constraints and ink style references into the network, guiding the model to generate natural ink mark effects in the area to be restored. In the model training phase, this invention employs multiple loss functions to ensure the effectiveness of structure and ink mark restoration, enabling the generated image to retain the original structural information of the authentic work while possessing a natural ink mark effect, thus enhancing visual realism. Attached Figure Description

[0023] Figure 1 This is a flowchart illustrating the method for real ink stain-driven copying based on latent space diffusion according to the present invention. Figure 2 This is a schematic diagram of the overall network architecture of the real ink-driven copying method based on latent space diffusion of the present invention. Figure 3 This is a schematic diagram of the ink encoder used in this invention; Figure 4 This is a schematic diagram of the reinking network used in this invention; Figure 5 This is a schematic diagram of the resampling strategy used in this invention. Detailed Implementation

[0024] The present invention will now be described in detail with reference to the accompanying drawings and specific embodiments.

[0025] Example 1 This invention is based on a real ink stain driven by latent space diffusion to create a method for authenticating a copy. The flowchart is as follows: Figure 1 As shown in the diagram, the overall network architecture is as follows: Figure 2 As shown, please follow these steps: Step 1: Collect target copy images and a large number of real ink sample images. Perform preprocessing operations such as grayscale conversion, noise reduction, and resolution unification on the images to construct standardized training and testing sets, providing a high-quality data foundation for model training. Step 1 is implemented in the following steps: Step 1.1: Collect images of the rubbings that need to be restored. At the same time, collect a wide range of real ink images of calligraphers with different styles, including Wang Xizhi and Yan Zhenqing. The selected ink images cover a variety of brush and ink effects, paper backgrounds and ink density to ensure the richness of the samples. Step 1.2: Preprocess the collected image data, specifically including: converting the image to grayscale, removing discrete noise using median filtering or nonlocal mean, cropping character blocks, and dividing the preprocessed dataset into training and test sets in an 8:2 ratio; Step 1.3: To improve the model's adaptability and generalization, data augmentation operations are performed on the processed images. These operations include small-angle rotation, scaling, translation, and mirror flipping. During the transformation process, the glyph structure remains unchanged, providing effective data support for subsequent training.

[0026] Step 2: Extract contour maps and skeleton maps from the target copy image using contour extraction and skeleton extraction algorithms, and mix them linearly to generate a continuous structural feature map, which serves as the input to the encoder in the re-inking network. Step 2 is implemented in the following steps: Based on the preprocessed copy image, key structural information is extracted using traditional image processing algorithms, and the outline of the strokes is identified using edge detection algorithms to generate a contour map. The image is processed using a skeleton extraction algorithm to obtain the centerline information of the strokes, i.e., the skeleton diagram. This represents the direction and general shape of the strokes, which is used to generate structural feature maps that meet the requirements of subsequent input. , skeleton diagram Contour drawing The formula for performing a linear combination and smoothing is as follows:

[0027] in, The standard deviation is The two-dimensional Gaussian kernel represents the processing operations used for noise removal and image smoothing. It is an adjustment parameter between 0 and 1, used to balance the importance of skeleton and contour in structural features.

[0028] Step 3: Construct an ink encoder to encode authentic ink samples, analyze the characteristics of ancient ink strokes, learn the natural ink color variation patterns, and extract the ink features of authentic ink samples. The encoder will eventually output two forms of style reference information: one to summarize the overall style of the ink strokes, which is achieved by performing local average pooling and concatenation on the extracted feature information; the other to preserve the local details of the ink strokes, which is generated by concatenating different features and reducing dimensionality. Step 3 is implemented in the following steps: The constructed ink encoder extracts multi-scale features of brush texture, stroke weight variation, and ink density from real ink samples, progressively. This module has two output branches, generating reference information reflecting the overall ink style and spatial reference information preserving brushstroke details, respectively. Details are as follows: Step 3.1: To achieve the above functions, the ink encoder adopts a four-stage structure from shallow to deep to extract features layer by layer; Stage 1 captures the local texture and brush marks of Chinese character strokes; Stage 2 integrates multi-scale brush textures based on Stage 1, and the core of Stage 2 is to extract features through multi-scale feature fusion and attention mechanism; Stage 3 further extracts the light and heavy distribution features of strokes; the core of Stage 4 is to focus on the changes in ink density and the dependence of ink color at a distance. Step 3.1 is implemented according to the following steps: like Figure 3 As shown, stage 1 captures the local texture and brush marks of Chinese character strokes. The input is a single-channel ink sample image, which is processed through the first layer. After small kernel convolution and group normalized GN processing, it is then passed through... of Hollow convolution further expands the sensory field to After that Convolutional downsampling, combined with group normalized GN and ReLU activation, outputs low-level detailed features. Stage 2 builds upon Stage 1 by integrating multi-scale brush textures. The core of Stage 2 lies in extracting features through multi-scale feature fusion and attention mechanisms. After processing through two residual blocks, each residual block contains two... Convolution, group normalization, and activation functions are applied; subsequently, the number of channels is expanded through channel-wise up-dimensional convolution. In the ASPP (Aperture Priority Process) module with a holed spatial pyramid pooling architecture, a multi-branch parallel structure is employed: branch A uses... Convolution extracts local features; branches B, C, and D use dilation values ​​of 6, 12, and 18, respectively. Dilated convolution captures brush textures at different scales, and branch E is combined after pooling. Convolution and upsampling are performed; the outputs of each branch are concatenated and then processed... Convolutional fusion and normalization, activation function processing; then, the CBAM attention mechanism is introduced, where channel attention is fused with global average pooling (GAP) and global max pooling (GMP) statistics, and channel weights are generated through a multilayer perceptron (MLP) and sigmoid activation. Spatial attention is then achieved through... Convolution processes the channel pooling results to generate spatial weights; then... Convolutional downsampling yields brush texture features Stage 3 involves further extracting the weight distribution features of the strokes, and then applying the features obtained in Stage 2... Through two layers Convolutional processing is performed, with each layer incorporating group normalization and ReLU activation. Then, global average pooling (GAP) is used to generate channel statistics, followed by MLP and sigmoid activation to generate gated vectors. These gated vectors are then broadcast to the spatial dimension for adaptive channel adjustment. Finally, 3×3 convolutional downsampling is used to obtain stroke weight distribution features. The core of Phase 4 lies in focusing on the variations in ink density and their dependence on ink color over long distances, highlighting key characteristics. through Large kernel convolution and normalization are performed, and then the processed features are flattened into a spatial sequence with positional encoding to form a sequence with positional information. The sequence then passes through a 1-2 layer Transformer encoder, each layer containing normalization, multi-head self-attention (MHA), dropout, residual connections, and a feedforward neural network (FFN). Finally, the features output by the Transformer are further processed... Convolution transforms the features back into spatial dimensions, yielding the ink color features. .

[0029] Step 3.2: Extract the brushstroke features of real ink samples.

[0030] Step 3.2 is implemented according to the following steps: At the end of the encoder, the features from each stage are first... , and Adjusted to a uniform spatial size and number of channels, forming a consistent feature representation: among which, For the adjusted texture features, For the adjusted distribution characteristics of light and heavy strokes, This represents the adjusted ink color features, and then two parallel output branches are set up based on this. The overall ink color style path applies to the three features mentioned above respectively. , , Global average pooling is used to aggregate global information, resulting in three generalized feature vectors: This represents the overall style vector of the brush texture. A global distribution vector representing the weight of strokes. This is the overall feature vector representing the density of ink color; subsequently, these three vectors are concatenated sequentially to obtain the overall ink color style reference vector. As a reference for subsequently injecting the overall style, the relevant formula is as follows:

[0031]

[0032] Spatial feature paths that preserve local details will combine three features , , The features are concatenated along the channel dimension to form a comprehensive feature, and then fused and compressed through a convolution operation with a kernel size of 1 to generate a spatial reference feature containing local brushstroke details. The formula is:

[0033] in, Indicates use Perform a linear mapping on the convolution kernel. This is for splicing operations.

[0034] Step 4: Using the U-net architecture as the basic framework, construct a re-inking network based on the latent space diffusion model. First, the structural feature map obtained in Step 2 is encoded by the encoder and converted into low-dimensional data, which serves as the initial starting point for the diffusion model. In the subsequent generation process, a multi-scale adapter is used to introduce skeleton and contour information as structural constraints, as well as the ink color features learned in Step 3, to achieve synchronous control of character structure and brushstroke style. A resampling strategy is used to optimize the area to be repaired multiple times. Finally, the optimized data representation is converted into a pixel image by the decoder, and then post-processed to obtain a calligraphy image with natural ink marks and realistic brushstrokes.

[0035] In step 4, the re-inking network is built based on the diffusion model framework, and the overall architecture is as follows: Figure 4 As shown, it consists of the following five core modules: VAE codec, multi-scale U-Net generation backbone, structure adapter, overall style injection module, and spatial ink adapter. These modules work together as follows: The VAE codec is used to establish a reversible mapping between the pixel domain and the latent space. The encoder converts the input image into a low-dimensional data representation, and the decoder can restore a high-quality image from this low-dimensional data, thereby effectively reducing the computational complexity of the subsequent generation process and preserving key visual information; the multi-scale U-Net architecture serves as the generation backbone, performing progressive denoising and image generation in the latent space; the structure adapter is used to introduce glyph skeletons and contours as structural prior constraints at different scales of U-Net; the overall style injection module is responsible for introducing the overall ink color variation style; and the spatial ink adapter provides local brushstroke details.

[0036] Step 4 is implemented in the following steps: Step 4.1: Obtain the structural feature map from Step 2. The input is processed by a VAE encoder and converted into a low-dimensional data representation, i.e., latent variables. Then Gaussian noise was added to the latent variables to obtain This serves as the input for subsequent denoising; the reference original image is input into the VAE encoder to obtain the latent variables corresponding to the original image. This serves as a style reference region for the original artwork in subsequent resampling strategies; simultaneously, to limit the resampling region, the feature map... Downsampling to the latent space size yields the mask. ; Step 4.2: During the step-by-step generation process, the structural adapter will introduce glyph constraints at different scales. Specifically, this involves: converting the skeleton diagram... and outline Adjust to the current scale, denoted as and Then, the adjusted skeleton diagram and outline The glyphs are concatenated and input into the structure adapter for processing. The structure adapter is a small convolutional mapping network. After processing by the structure adapter, the glyphs are incorporated into the features of the current encoder in an element-wise addition manner and provided to the decoder of the corresponding scale through skip connections, thereby maintaining the stability of the glyph structure at multiple scales. Step 4.3: In the decoder of U-net in the re-inking network, two types of style information are injected in parallel for each layer: one is the overall style injection, and the other is the local detail injection.

[0037] Overall style injection: Injecting style reference vectors Expand to the same spatial size as the current decoder layer, then expand the vector... By splicing and fusing with the decoder features, the overall ink color style is injected; Local detail injection: injecting spatial-level details Adjust as needed to the current decoder layer size, then obtain the detailed features after adjustment. The input is fed into the ink adapter, which is a shallow convolutional mapping network, to obtain detailed features consistent with the number of channels in the decoding layer. and in the form of residuals Add back decoder features to restore both overall style and brushstroke details at the same scale; Step 4.4: During the iterative generation process, focus on optimizing the areas to be repaired, such as... Figure 5 As shown, through the latent variables of the original document The noisy result obtained from upsampling is compared with the prediction result given by the network in the region to be repaired, according to the mask. Linear mixing is performed to obtain a new low-dimensional data representation for that time step. Then, forward diffusion noise is added to the result, and the above process is repeated to improve the naturalness of the ink effect in the area to be repaired. Step 4.5: Finally, the optimized latent variables are obtained. That is, the optimized low-dimensional data representation is restored into an image through the VAE decoder, and the obtained image is post-processed with contrast adjustment, edge smoothing and other processes to obtain a target image that combines the original character structure and natural ink effect. Step 4.6: Employ a custom joint loss function to ensure comprehensive optimization of the generated image in terms of structural constraints and visual quality. Specifically:

[0038] in, For the overall joint loss function, To predict the loss for diffused noise, For style statistics loss, Indicates perceived loss. , , These are the corresponding weight hyperparameters; Predicted loss of diffused noise The constraint diffusion model can accurately predict the noise component in the latent variables. It is the core constraint for training the re-inking network, and is achieved by comparing the difference between the noise predicted by the network and the actual noise. The formula is as follows:

[0039] in, As initial latent variables, For noisy latent variables in the diffusion process, It is Gaussian noise. For the prediction noise of the diffusion network, This indicates conditions such as structure and ink marks. It is expressed as the L2 norm, or Euclidean norm, which measures the difference between predicted noise and actual noise in the formula; Style statistical loss By comparing the ink distribution patterns of the modified image with those of the real ink marks, the generated result is guaranteed to conform to the characteristics of real calligraphy in terms of ink density variations, making the overall style close to the original work. The formula is as follows:

[0040] in, Let represent the mean and standard deviation of the image pixel distribution, respectively. To generate an image, An image of the original work for reference; Perceived loss Based on the differences between the generated map and the reference map calculated in the feature space, the generation result is compared with the reference map in terms of detail to ensure that the generated result is consistent with the reference in texture detail, thereby improving the visual realism and reducing perceptual loss. The formula is:

[0041] in, It is the first pre-trained network such as VGG Layer feature mapping, These are the weighting coefficients for each layer; the above loss functions are combined in a weighted manner to guide the training process of the model, so as to achieve a balance between structural accuracy, visual quality and natural ink color.

[0042] Example 2 This invention is based on a real ink stain driven by latent space diffusion to create a method for authenticating a copy. The flowchart is as follows: Figure 1 As shown, please follow these steps: Step 1: Collect target copy images and a large number of real ink sample images. Perform preprocessing operations such as grayscale conversion, noise reduction, and resolution unification on the images to construct standardized training and testing sets, providing a high-quality data foundation for model training. Step 2: Extract contour maps and skeleton maps from the target copy image using contour extraction and skeleton extraction algorithms, and mix them linearly to generate a continuous structural feature map, which serves as the input to the encoder in the re-inking network. Step 3: Construct an ink encoder to encode authentic ink samples, analyze the characteristics of ancient ink strokes, learn the natural ink color variation patterns, and extract the ink features of authentic ink samples. The encoder will eventually output two forms of style reference information: one to summarize the overall style of the ink strokes, which is achieved by performing local average pooling and concatenation on the extracted feature information; the other to preserve the local details of the ink strokes, which is generated by concatenating different features and reducing dimensionality. Step 4: Using the U-net architecture as the basic framework, construct a re-inking network based on the latent space diffusion model. First, the structural feature map obtained in Step 2 is encoded by the encoder and converted into low-dimensional data, which serves as the initial starting point for the diffusion model. In the subsequent generation process, skeleton and contour information are introduced as structural constraints through a multi-scale adapter, along with the ink color features learned in Step 3, to achieve synchronous control of character structure and brushstroke style. A resampling strategy is used to optimize the area to be repaired multiple times. Finally, the optimized data representation is converted into a pixel image by the decoder, and then post-processed to obtain a calligraphy image with natural ink marks and realistic brushstrokes. Example 3 This invention is based on a real ink stain driven by latent space diffusion to create a method for authenticating a copy. The flowchart is as follows: Figure 1 As shown, please follow these steps: Step 1: Collect target copy images and a large number of real ink sample images. Perform preprocessing operations such as grayscale conversion, noise reduction, and resolution unification on the images to construct standardized training and testing sets, providing a high-quality data foundation for model training. Step 1 is implemented in the following steps: Step 1.1: Collect images of the rubbings that need to be restored. At the same time, collect a wide range of real ink images of calligraphers with different styles, including Wang Xizhi and Yan Zhenqing. The selected ink images cover a variety of brush and ink effects, paper backgrounds and ink density to ensure the richness of the samples. Step 1.2: Preprocess the collected image data, specifically including: converting the image to grayscale, removing discrete noise using median filtering or nonlocal mean, cropping character blocks, and dividing the preprocessed dataset into training and test sets in an 8:2 ratio; Step 1.3: To improve the model's adaptability and generalization, data augmentation operations are performed on the processed images. These operations include small-angle rotation, scaling, translation, and mirror flipping. During the transformation process, the glyph structure remains unchanged, providing effective data support for subsequent training.

[0043] Step 2: Extract contour maps and skeleton maps from the target copy image using contour extraction and skeleton extraction algorithms, and mix them linearly to generate a continuous structural feature map, which serves as the input to the encoder in the re-inking network. Step 3: Construct an ink encoder to encode authentic ink samples, analyze the characteristics of ancient ink strokes, learn the natural ink color variation patterns, and extract the ink features of authentic ink samples. The encoder will eventually output two forms of style reference information: one to summarize the overall style of the ink strokes, which is achieved by performing local average pooling and concatenation on the extracted feature information; the other to preserve the local details of the ink strokes, which is generated by concatenating different features and reducing dimensionality. Step 4: Using the U-net architecture as the basic framework, construct a re-inking network based on the latent space diffusion model. First, the structural feature map obtained in Step 2 is encoded by the encoder and converted into low-dimensional data, which serves as the initial starting point for the diffusion model. In the subsequent generation process, a multi-scale adapter is used to introduce skeleton and contour information as structural constraints, as well as the ink color features learned in Step 3, to achieve synchronous control of character structure and brushstroke style. A resampling strategy is used to optimize the area to be repaired multiple times. Finally, the optimized data representation is converted into a pixel image by the decoder, and then post-processed to obtain a calligraphy image with natural ink marks and realistic brushstrokes.

[0044] Example 4 This invention is based on a real ink stain driven by latent space diffusion to create a method for authenticating a copy. The flowchart is as follows: Figure 1 As shown, please follow these steps: Step 1: Collect target copy images and a large number of real ink sample images. Perform preprocessing operations such as grayscale conversion, noise reduction, and resolution unification on the images to construct standardized training and testing sets, providing a high-quality data foundation for model training. Step 1 is implemented in the following steps: Step 1.1: Collect images of the rubbings that need to be restored. At the same time, collect a wide range of real ink images of calligraphers with different styles, including Wang Xizhi and Yan Zhenqing. The selected ink images cover a variety of brush and ink effects, paper backgrounds and ink density to ensure the richness of the samples. Step 1.2: Preprocess the collected image data, specifically including: converting the image to grayscale, removing discrete noise using median filtering or nonlocal mean, cropping character blocks, and dividing the preprocessed dataset into training and test sets in an 8:2 ratio; Step 1.3: To improve the model's adaptability and generalization, data augmentation operations are performed on the processed images. These operations include small-angle rotation, scaling, translation, and mirror flipping. During the transformation process, the glyph structure remains unchanged, providing effective data support for subsequent training.

[0045] Step 2: Extract contour maps and skeleton maps from the target copy image using contour extraction and skeleton extraction algorithms, and mix them linearly to generate a continuous structural feature map, which serves as the input to the encoder in the re-inking network. Step 2 is implemented in the following steps: Based on the preprocessed copy image, key structural information is extracted using traditional image processing algorithms, and the outline of the strokes is identified using edge detection algorithms to generate a contour map. The image is processed using a skeleton extraction algorithm to obtain the centerline information of the strokes, i.e., the skeleton diagram. This represents the direction and general shape of the strokes, which is used to generate structural feature maps that meet the requirements of subsequent input. , skeleton diagram Contour drawing The formula for performing a linear combination and smoothing is as follows:

[0046] in, The standard deviation is The two-dimensional Gaussian kernel represents the processing operations used for noise removal and image smoothing. It is an adjustment parameter between 0 and 1, used to balance the importance of skeleton and contour in structural features.

[0047] Step 3: Construct an ink encoder to encode authentic ink samples, analyze the characteristics of ancient ink strokes, learn the natural ink color variation patterns, and extract the ink features of authentic ink samples. The encoder will eventually output two forms of style reference information: one to summarize the overall style of the ink strokes, which is achieved by performing local average pooling and concatenation on the extracted feature information; the other to preserve the local details of the ink strokes, which is generated by concatenating different features and reducing dimensionality. Step 4: Using the U-net architecture as the basic framework, construct a re-inking network based on the latent space diffusion model. First, the structural feature map obtained in Step 2 is encoded by the encoder and converted into low-dimensional data, which serves as the initial starting point for the diffusion model. In the subsequent generation process, a multi-scale adapter is used to introduce skeleton and contour information as structural constraints, as well as the ink color features learned in Step 3, to achieve synchronous control of character structure and brushstroke style. A resampling strategy is used to optimize the area to be repaired multiple times. Finally, the optimized data representation is converted into a pixel image by the decoder, and then post-processed to obtain a calligraphy image with natural ink marks and realistic brushstrokes.

[0048] Example 5 This invention is based on a real ink stain driven by latent space diffusion to create a method for authenticating a copy. The flowchart is as follows: Figure 1 As shown, please follow these steps: Step 1: Collect target copy images and a large number of real ink sample images. Perform preprocessing operations such as grayscale conversion, noise reduction, and resolution unification on the images to construct standardized training and testing sets, providing a high-quality data foundation for model training. Step 1 is implemented in the following steps: Step 1.1: Collect images of the rubbings that need to be restored. At the same time, collect a wide range of real ink images of calligraphers with different styles, including Wang Xizhi and Yan Zhenqing. The selected ink images cover a variety of brush and ink effects, paper backgrounds and ink density to ensure the richness of the samples. Step 1.2: Preprocess the collected image data, specifically including: converting the image to grayscale, removing discrete noise using median filtering or nonlocal mean, cropping character blocks, and dividing the preprocessed dataset into training and test sets in an 8:2 ratio; Step 1.3: To improve the model's adaptability and generalization, data augmentation operations are performed on the processed images. These operations include small-angle rotation, scaling, translation, and mirror flipping. During the transformation process, the glyph structure remains unchanged, providing effective data support for subsequent training.

[0049] Step 2: Extract contour maps and skeleton maps from the target copy image using contour extraction and skeleton extraction algorithms, and mix them linearly to generate a continuous structural feature map, which serves as the input to the encoder in the re-inking network. Step 2 is implemented in the following steps: Based on the preprocessed copy image, key structural information is extracted using traditional image processing algorithms, and the outline of the strokes is identified using edge detection algorithms to generate a contour map. The image is processed using a skeleton extraction algorithm to obtain the centerline information of the strokes, i.e., the skeleton diagram. This represents the direction and general shape of the strokes, which is used to generate structural feature maps that meet the requirements of subsequent input. , skeleton diagram Contour drawing The formula for performing a linear combination and smoothing is as follows:

[0050] in, The standard deviation is The two-dimensional Gaussian kernel represents the processing operations used for noise removal and image smoothing. It is an adjustment parameter between 0 and 1, used to balance the importance of skeleton and contour in structural features.

[0051] Step 3: Construct an ink encoder to encode authentic ink samples, analyze the characteristics of ancient ink strokes, learn the natural ink color variation patterns, and extract the ink features of authentic ink samples. The encoder will eventually output two forms of style reference information: one to summarize the overall style of the ink strokes, which is achieved by performing local average pooling and concatenation on the extracted feature information; the other to preserve the local details of the ink strokes, which is generated by concatenating different features and reducing dimensionality. Step 3 is implemented in the following steps: The constructed ink encoder extracts multi-scale features of brush texture, stroke weight variation, and ink density from real ink samples, progressively. This module has two output branches, generating reference information reflecting the overall ink style and spatial reference information preserving brushstroke details, respectively. Details are as follows: Step 3.1: To achieve the above functions, the ink encoder adopts a four-stage structure from shallow to deep to extract features layer by layer; Stage 1 captures the local texture and brush marks of Chinese character strokes; Stage 2 integrates multi-scale brush textures based on Stage 1, and the core of Stage 2 is to extract features through multi-scale feature fusion and attention mechanism; Stage 3 further extracts the light and heavy distribution features of strokes; the core of Stage 4 is to focus on the changes in ink density and the dependence of ink color at a distance. Step 3.1 is implemented according to the following steps: like Figure 3 As shown, stage 1 captures the local texture and brush marks of Chinese character strokes. The input is a single-channel ink sample image, which is processed through the first layer. After small kernel convolution and group normalized GN processing, it is then passed through... of Hollow convolution further expands the sensory field to After that Convolutional downsampling, combined with group normalized GN and ReLU activation, outputs low-level detailed features. Stage 2 builds upon Stage 1 by integrating multi-scale brush textures. The core of Stage 2 lies in extracting features through multi-scale feature fusion and attention mechanisms. After processing through two residual blocks, each residual block contains two... Convolution, group normalization, and activation functions are applied; subsequently, the number of channels is expanded through channel-wise up-dimensional convolution. In the ASPP (Aperture Priority Process) module with a holed spatial pyramid pooling architecture, a multi-branch parallel structure is employed: branch A uses... Convolution extracts local features; branches B, C, and D use dilation values ​​of 6, 12, and 18, respectively. Dilated convolution captures brush textures at different scales, and branch E is combined after pooling. Convolution and upsampling are performed; the outputs of each branch are concatenated and then processed... Convolutional fusion and normalization, activation function processing; then, the CBAM attention mechanism is introduced, where channel attention is fused with global average pooling (GAP) and global max pooling (GMP) statistics, and channel weights are generated through a multilayer perceptron (MLP) and sigmoid activation. Spatial attention is then achieved through... Convolution processes the channel pooling results to generate spatial weights; then... Convolutional downsampling yields brush texture features Stage 3 involves further extracting the weight distribution features of the strokes, and then applying the features obtained in Stage 2... Through two layers Convolutional processing is performed, with each layer incorporating group normalization and ReLU activation. Then, global average pooling (GAP) is used to generate channel statistics, followed by MLP and sigmoid activation to generate gated vectors. These gated vectors are then broadcast to the spatial dimension for adaptive channel adjustment. Finally, 3×3 convolutional downsampling is used to obtain stroke weight distribution features. The core of Phase 4 lies in focusing on the variations in ink density and their dependence on ink color over long distances, highlighting key characteristics. through Large kernel convolution and normalization are performed, and then the processed features are flattened into a spatial sequence with positional encoding to form a sequence with positional information. The sequence then passes through a 1-2 layer Transformer encoder, each layer containing normalization, multi-head self-attention (MHA), dropout, residual connections, and a feedforward neural network (FFN). Finally, the features output by the Transformer are further processed... Convolution transforms the features back into spatial dimensions, yielding the ink color features. .

[0052] Step 3.2: Extract the brushstroke features of real ink samples.

[0053] Step 4: Using the U-net architecture as the basic framework, construct a re-inking network based on the latent space diffusion model. First, the structural feature map obtained in Step 2 is encoded by the encoder and converted into low-dimensional data, which serves as the initial starting point for the diffusion model. In the subsequent generation process, a multi-scale adapter is used to introduce skeleton and contour information as structural constraints, as well as the ink color features learned in Step 3, to achieve synchronous control of character structure and brushstroke style. A resampling strategy is used to optimize the area to be repaired multiple times. Finally, the optimized data representation is converted into a pixel image by the decoder, and then post-processed to obtain a calligraphy image with natural ink marks and realistic brushstrokes.

[0054] Example 6 This invention is based on a real ink stain driven by latent space diffusion to create a method for authenticating a copy. The flowchart is as follows: Figure 1 As shown, please follow these steps: Step 1: Collect target copy images and a large number of real ink sample images. Perform preprocessing operations such as grayscale conversion, noise reduction, and resolution unification on the images to construct standardized training and testing sets, providing a high-quality data foundation for model training. Step 1 is implemented in the following steps: Step 1.1: Collect images of the rubbings that need to be restored. At the same time, collect a wide range of real ink images of calligraphers with different styles, including Wang Xizhi and Yan Zhenqing. The selected ink images cover a variety of brush and ink effects, paper backgrounds and ink density to ensure the richness of the samples. Step 1.2: Preprocess the collected image data, specifically including: converting the image to grayscale, removing discrete noise using median filtering or nonlocal mean, cropping character blocks, and dividing the preprocessed dataset into training and test sets in an 8:2 ratio; Step 1.3: To improve the model's adaptability and generalization, data augmentation operations are performed on the processed images. These operations include small-angle rotation, scaling, translation, and mirror flipping. During the transformation process, the glyph structure remains unchanged, providing effective data support for subsequent training.

[0055] Step 2: Extract contour maps and skeleton maps from the target copy image using contour extraction and skeleton extraction algorithms, and mix them linearly to generate a continuous structural feature map, which serves as the input to the encoder in the re-inking network. Step 2 is implemented in the following steps: Based on the preprocessed copy image, key structural information is extracted using traditional image processing algorithms, and the outline of the strokes is identified using edge detection algorithms to generate a contour map. The image is processed using a skeleton extraction algorithm to obtain the centerline information of the strokes, i.e., the skeleton diagram. This represents the direction and general shape of the strokes, which is used to generate structural feature maps that meet the requirements of subsequent input. , skeleton diagram Contour drawing The formula for performing a linear combination and smoothing is as follows:

[0056] in, The standard deviation is The two-dimensional Gaussian kernel represents the processing operations used for noise removal and image smoothing. It is an adjustment parameter between 0 and 1, used to balance the importance of skeleton and contour in structural features.

[0057] Step 3: Construct an ink encoder to encode authentic ink samples, analyze the characteristics of ancient ink strokes, learn the natural ink color variation patterns, and extract the ink features of authentic ink samples. The encoder will eventually output two forms of style reference information: one to summarize the overall style of the ink strokes, which is achieved by performing local average pooling and concatenation on the extracted feature information; the other to preserve the local details of the ink strokes, which is generated by concatenating different features and reducing dimensionality. Step 3 is implemented in the following steps: The constructed ink encoder extracts multi-scale features of brush texture, stroke weight variation, and ink density from real ink samples, progressively. This module has two output branches, generating reference information reflecting the overall ink style and spatial reference information preserving brushstroke details, respectively. Details are as follows: Step 3.1: To achieve the above functions, the ink encoder adopts a four-stage structure from shallow to deep to extract features layer by layer; Stage 1 captures the local texture and brush marks of Chinese character strokes; Stage 2 integrates multi-scale brush textures based on Stage 1, and the core of Stage 2 is to extract features through multi-scale feature fusion and attention mechanism; Stage 3 further extracts the light and heavy distribution features of strokes; the core of Stage 4 is to focus on the changes in ink density and the dependence of ink color at a distance. Step 3.1 is implemented according to the following steps: like Figure 3 As shown, stage 1 captures the local texture and brush marks of Chinese character strokes. The input is a single-channel ink sample image, which is processed through the first layer. After small kernel convolution and group normalized GN processing, it is then passed through... of Hollow convolution further expands the sensory field to After that Convolutional downsampling, combined with group normalized GN and ReLU activation, outputs low-level detailed features. Stage 2 builds upon Stage 1 by integrating multi-scale brush textures. The core of Stage 2 lies in extracting features through multi-scale feature fusion and attention mechanisms. After processing through two residual blocks, each residual block contains two... Convolution, group normalization, and activation functions are applied; subsequently, the number of channels is expanded through channel-wise up-dimensional convolution. In the ASPP (Aperture Priority Process) module with a holed spatial pyramid pooling architecture, a multi-branch parallel structure is employed: branch A uses... Convolution extracts local features; branches B, C, and D use dilation values ​​of 6, 12, and 18, respectively. Dilated convolution captures brush textures at different scales, and branch E is combined after pooling. Convolution and upsampling are performed; the outputs of each branch are concatenated and then processed... Convolutional fusion and normalization, activation function processing; then, the CBAM attention mechanism is introduced, where channel attention is fused with global average pooling (GAP) and global max pooling (GMP) statistics, and channel weights are generated through a multilayer perceptron (MLP) and sigmoid activation. Spatial attention is then achieved through... Convolution processes the channel pooling results to generate spatial weights; then... Convolutional downsampling yields brush texture features Stage 3 involves further extracting the weight distribution features of the strokes, and then applying the features obtained in Stage 2... Through two layers Convolutional processing is performed, with each layer incorporating group normalization and ReLU activation. Then, global average pooling (GAP) is used to generate channel statistics, followed by MLP and sigmoid activation to generate gated vectors. These gated vectors are then broadcast to the spatial dimension for adaptive channel adjustment. Finally, 3×3 convolutional downsampling is used to obtain stroke weight distribution features. The core of Phase 4 lies in focusing on the variations in ink density and their dependence on ink color over long distances, highlighting key characteristics. through Large kernel convolution and normalization are performed, and then the processed features are flattened into a spatial sequence with positional encoding to form a sequence with positional information. The sequence then passes through a 1-2 layer Transformer encoder, each layer containing normalization, multi-head self-attention (MHA), dropout, residual connections, and a feedforward neural network (FFN). Finally, the features output by the Transformer are further processed... Convolution transforms the features back into spatial dimensions, yielding the ink color features. .

[0058] Step 3.2: Extract the brushstroke features of real ink samples.

[0059] Step 3.2 is implemented according to the following steps: At the end of the encoder, the features from each stage are first... , and Adjusted to a uniform spatial size and number of channels, forming a consistent feature representation: among which, For the adjusted texture features, For the adjusted distribution characteristics of light and heavy strokes, This represents the adjusted ink color features, and then two parallel output branches are set up based on this. The overall ink color style path applies to the three features mentioned above respectively. , , Global average pooling is used to aggregate global information, resulting in three generalized feature vectors: This represents the overall style vector of the brush texture. A global distribution vector representing the weight of strokes. This is the overall feature vector representing the density of ink color; subsequently, these three vectors are concatenated sequentially to obtain the overall ink color style reference vector. As a reference for subsequently injecting the overall style, the relevant formula is as follows:

[0060]

[0061] Spatial feature paths that preserve local details will combine three features , , The features are concatenated along the channel dimension to form a comprehensive feature, and then fused and compressed through a convolution operation with a kernel size of 1 to generate a spatial reference feature containing local brushstroke details. The formula is:

[0062] in, Indicates use Perform a linear mapping on the convolution kernel. This is for splicing operations.

[0063] Step 4: Using the U-net architecture as the basic framework, construct a re-inking network based on the latent space diffusion model. First, the structural feature map obtained in Step 2 is encoded by the encoder and converted into low-dimensional data, which serves as the initial starting point for the diffusion model. In the subsequent generation process, a multi-scale adapter is used to introduce skeleton and contour information as structural constraints, as well as the ink color features learned in Step 3, to achieve synchronous control of character structure and brushstroke style. A resampling strategy is used to optimize the area to be repaired multiple times. Finally, the optimized data representation is converted into a pixel image by the decoder, and then post-processed to obtain a calligraphy image with natural ink marks and realistic brushstrokes.

Claims

1. A method for authenticating copies of ink samples based on latent space diffusion-driven real ink marks, characterized in that, The specific steps are as follows: Step 1: Collect target rubbing images and a large number of real ink sample images, perform preprocessing operations on the images, and construct standardized training and testing sets; Step 2: Extract the contour map and skeleton map from the target copy image, and mix them in a linear manner to generate a continuous structural feature map; Step 3: Construct an ink encoder to encode the authentic ink samples and extract the ink features of the authentic ink samples. Step 4: Construct a re-inking network based on the latent space diffusion model. Finally, the optimized data representation is converted into a pixel image by the decoder to obtain a calligraphy image with natural ink marks and realistic brushstrokes.

2. The method for authenticating a copy based on latent space diffusion of real ink marks according to claim 1, characterized in that, Step 1 is implemented in the following steps: Step 1.1: Collect images of the rubbings that need to be restored, and at the same time collect images of real ink marks from different calligraphers, including Wang Xizhi and Yan Zhenqing. The selected ink marks cover a variety of brush and ink effects, paper backgrounds and ink density to ensure the richness of the samples. Step 1.2: Preprocess the collected image data, specifically including: converting the image to grayscale, removing discrete noise using median filtering or nonlocal mean, cropping character blocks, and dividing the preprocessed dataset into training and test sets in an 8:2 ratio; Step 1.3: To improve the model's adaptability and generalization, data augmentation is performed on the processed images. During the transformation process, the glyph structure remains unchanged, providing effective data support for subsequent training.

3. The method for authenticating a copy based on latent space diffusion of real ink marks according to claim 2, characterized in that, Step 2 is implemented in the following steps: Based on the preprocessed copy image, key structural information is extracted using traditional image processing algorithms, and the outline of the strokes is identified using edge detection algorithms to generate a contour map. The image is processed using a skeleton extraction algorithm to obtain the centerline information of the strokes, i.e., the skeleton diagram. This represents the direction and general shape of the strokes, which is used to generate structural feature maps that meet the requirements of subsequent input. , skeleton diagram Contour drawing The formula for performing a linear combination and smoothing is as follows: in, The standard deviation is The two-dimensional Gaussian kernel represents the processing operations used for noise removal and image smoothing. It is an adjustment parameter between 0 and 1, used to balance the importance of skeleton and contour in structural features.

4. The method for authenticating a copy based on latent space diffusion of real ink marks according to claim 3, characterized in that, Step 3 is implemented in the following steps: Step 3.1: The ink encoder uses a four-stage structure from shallow to deep to extract features layer by layer; Stage 1 captures the local texture and brush marks of Chinese character strokes; Stage 2 integrates multi-scale brush textures based on Stage 1. The core of Stage 2 is to extract features through multi-scale feature fusion and attention mechanism. Stage 3 involves further extracting the characteristics of the weight distribution of strokes; the core of Stage 4 lies in focusing on the changes in ink density and the dependence of ink color over long distances. Step 3.2: Extract the brushstroke features of real ink samples.

5. The method for authenticating a copy based on latent space diffusion of real ink marks according to claim 4, characterized in that, Step 3.1 is implemented in the following steps: Phase 1 captures the local texture and brush marks of Chinese character strokes. The input is a single-channel ink sample image, processed through the first layer. After small kernel convolution and group normalized GN processing, it is then passed through... of Hollow convolution further expands the sense field to After that Convolutional downsampling, combined with group normalized GN and ReLU activation, outputs low-level detailed features. ; Phase 2 builds upon Phase 1 by integrating multi-scale brush textures. The core of Phase 2 lies in extracting features through multi-scale feature fusion and attention mechanisms. After processing through two residual blocks, each residual block contains two... Convolution, group normalization, and activation functions are applied; subsequently, the number of channels is expanded through channel-wise up-dimensional convolution. In the ASPP (Aperture Priority Process) module with a holed spatial pyramid pooling architecture, a multi-branch parallel structure is employed: branch A uses... Convolution extracts local features; branches B, C, and D use dilation values ​​of 6, 12, and 18, respectively. Dilated convolution captures brush textures at different scales, and branch E is combined after pooling. Convolution and upsampling are performed; the outputs of each branch are concatenated and then processed... Convolutional fusion and normalization, activation function processing; Then, the CBAM attention mechanism is introduced. Channel attention is fused with global average pooling (GAP) and global max pooling (GMP) statistics, and channel weights are generated through multilayer perceptron (MLP) and sigmoid activation. Spatial attention is then achieved through... Convolution processes the channel pooling results to generate spatial weights; Then after Convolutional downsampling yields brush texture features ; Stage 3 involves further extracting the weight distribution features of the strokes, and then applying the features obtained in Stage 2... Through two layers Convolutional processing is performed, with each layer incorporating group normalization and ReLU activation. Then, global average pooling (GAP) is used to generate channel statistics, followed by MLP and sigmoid activation to generate gated vectors. These gated vectors are then broadcast to the spatial dimension for adaptive channel adjustment. Finally, 3×3 convolutional downsampling is used to obtain stroke weight distribution features. The core of Phase 4 lies in focusing on the variations in ink density and their dependence on ink color over long distances, highlighting key characteristics. through Large kernel convolution and normalization are performed, and then the processed features are flattened into a spatial sequence with positional encoding to form a sequence with positional information. The sequence then passes through a 1-2 layer Transformer encoder, each layer containing normalization, multi-head self-attention (MHA), dropout, residual connections, and a feedforward neural network (FFN). Finally, the features output by the Transformer are further processed... Convolution transforms the features back into spatial dimensions, yielding the ink color features. .

6. The method for authenticating a copy based on latent space diffusion of real ink as described in claim 5, characterized in that, Step 3.2 is implemented in the following steps: At the end of the encoder, the features from each stage are first... , and Adjusted to a uniform spatial size and number of channels, forming a consistent feature representation: among which, For the adjusted texture features, For the adjusted distribution characteristics of light and heavy strokes, This represents the adjusted ink color features, and then two parallel output branches are set up based on this. The overall ink color style path applies to the three features mentioned above respectively. , , Global average pooling is used to aggregate global information, resulting in three generalized feature vectors: This represents the overall style vector of the brush texture. A global distribution vector representing the weight of strokes. This is the overall feature vector representing the density of ink color; subsequently, these three vectors are concatenated sequentially to obtain the overall ink color style reference vector. As a reference for subsequently injecting the overall style, the relevant formula is as follows: Spatial feature paths that preserve local details will combine three features , , The features are concatenated along the channel dimension to form a comprehensive feature, and then fused and compressed through a convolution operation with a kernel size of 1 to generate a spatial reference feature containing local brushstroke details. The formula is: in, Indicates use Perform a linear mapping on the convolution kernel. This is for splicing operations.

7. The method for authenticating a copy based on latent space diffusion of real ink marks according to claim 6, characterized in that, In step 4, the re-inking network is built based on a diffusion model framework. The overall architecture is shown in Figure 4 and consists of five core modules: VAE encoder / decoder, multi-scale U-Net generation backbone, structure adapter, overall style injection module, and spatial ink adapter. These modules work together as follows: The VAE encoder / decoder establishes a reversible mapping between the pixel domain and the latent space. The encoder converts the input image into a low-dimensional data representation, and the decoder can reconstruct a high-quality image from this low-dimensional data, thereby effectively reducing the computational complexity of the subsequent generation process and preserving key visual information. The multi-scale U-Net architecture serves as the generation backbone, performing progressive denoising and image generation in the latent space. The structure adapter introduces glyph skeletons and contours as structural prior constraints at different scales of the U-Net. The overall style injection module is responsible for introducing the overall ink color variation style, and the spatial ink adapter provides local brushstroke details.

8. The method for authenticating a copy based on latent space diffusion of real ink marks according to claim 7, characterized in that, Step 4 is implemented in the following steps: Step 4.1: Obtain the structural feature map from Step 2. The input is processed by a VAE encoder and converted into a low-dimensional data representation, i.e., latent variables. Then Gaussian noise was added to the latent variables to obtain This serves as the input for subsequent denoising; the reference original image is input into the VAE encoder to obtain the latent variables corresponding to the original image. This serves as a style reference region for the original artwork in subsequent resampling strategies; simultaneously, to limit the resampling region, the feature map... Downsampling to the latent space size yields the mask. ; Step 4.2: During the step-by-step generation process, the structural adapter will introduce glyph constraints at different scales. Specifically, this involves: converting the skeleton diagram... and outline Adjust to the current scale, denoted as and Then, the adjusted skeleton diagram and outline The glyphs are concatenated and input into the structure adapter for processing. The structure adapter is a small convolutional mapping network. After processing by the structure adapter, the glyphs are incorporated into the features of the current encoder in an element-wise addition manner and provided to the decoder of the corresponding scale through skip connections, thereby maintaining the stability of the glyph structure at multiple scales. Step 4.3: In the decoder of U-net in the re-inking network, two types of style information are injected in parallel for each layer: one is the overall style injection, and the other is the local detail injection. Overall style injection: Injecting style reference vectors Expand to the same spatial size as the current decoder layer, then expand the vector... By splicing and fusing with the decoder features, the overall ink color style is injected; Local detail injection: injecting spatial-level details Adjust as needed to the current decoder layer size, then obtain the detailed features after adjustment. The input is fed into the ink adapter, which is a shallow convolutional mapping network, to obtain detailed features consistent with the number of channels in the decoding layer. and in the form of residuals Add back decoder features to restore both overall style and brushstroke details at the same scale; Step 4.4: During the iterative generation process, the area to be repaired is optimized by focusing on the latent variables of the original work. The noisy result obtained from upsampling is compared with the prediction result given by the network in the region to be repaired, according to the mask. Perform linear mixing to obtain a new low-dimensional data representation for this time step, then simulate forward diffusion to add noise to the result, and then repeat the above process; Step 4.5: Finally, the optimized latent variables are obtained. That is, the optimized low-dimensional data representation is restored into an image through the VAE decoder, and the obtained image is post-processed with contrast adjustment, edge smoothing and other processes to obtain a target image that combines the original character structure and natural ink effect. Step 4.6: Use a custom joint loss function, specifically: in, For the overall joint loss function, To predict the loss for diffused noise, For style statistics loss, Indicates perceived loss. , , These are the corresponding weight hyperparameters; Predicted loss of diffused noise The constraint diffusion model can accurately predict the noise component in the latent variables. It is the core constraint for training the re-inking network, and is achieved by comparing the difference between the noise predicted by the network and the actual noise. The formula is as follows: in, As initial latent variables, For noisy latent variables in the diffusion process, It is Gaussian noise. For the prediction noise of the diffusion network, This indicates conditions such as structure and ink marks. It is expressed as the L2 norm, or Euclidean norm, which measures the difference between predicted noise and actual noise in the formula; Style statistical loss By comparing the ink distribution patterns of the modified image and the real ink stains, the formula is as follows: in, Let represent the mean and standard deviation of the image pixel distribution, respectively. To generate an image, An image of the original work for reference; Perceived loss Perceptual loss is calculated based on the difference between the generated map and the reference map in the feature space. The formula is: in, It is the first pre-trained network such as VGG Layer feature mapping, These are the weighting coefficients for each layer; the above loss functions are combined in a weighted manner to guide the training process of the model, so as to achieve a balance between structural accuracy, visual quality and natural ink color.

Images