A method, system, device and medium for restoring degraded archival image handwriting

Abstract

The application relates to a degraded archive image handwriting restoration method, system, device and medium. The method comprises the following steps: preprocessing a degraded archive image and calculating a structure tensor matrix; decomposing a matrix eigenvalue to generate an initial diffusion and a plume direction field; extracting paper texture and ink structure features in parallel through a double-branch network; adaptively fusing a multi-modal fusion feature map through channel attention; inputting the fusion feature into a physical information neural network to carry out plume-diffusion reverse evolution decoding with the two types of direction fields as apriori, and outputting a high-definition restoration image. The method can completely retain paper texture, clearly restore blurred degraded handwriting, and improve the authenticity of archive image restoration.

Description

Technical Field

[0001] This invention relates to the field of image processing technology, and in particular to a method, system, device, and medium for restoring handwriting in degraded archival images. Background Technology

[0002] Historical archives, as vital physical carriers of human civilization and social development, possess irreplaceable value in the rescue and protection of cultural heritage through their digital reconstruction and clear extraction of textual information. Over long periods of time and environmental changes, historical paper in museum collections commonly exhibits degradation phenomena such as acidification, yellowing, and mold. Simultaneously, the ink within the paper is prone to non-linear blurring and smudging due to adhesive degradation and microscopic seepage within capillary pores. Image restoration of degraded archival text, as a core method utilizing computer vision and image processing technology to clarify aging and damaged archival images, aims to accurately restore blurred or diffused strokes of text, thus restoring the key historical textual information recorded in the archives.

[0003] However, existing archival image enhancement and denoising techniques still have significant limitations when dealing with complex degradation scenarios. On the one hand, traditional image binarization or general deep learning denoising models typically assume that there is a discrete distribution difference between the foreground text and the background grayscale, and tend to perform smoothing or hard truncation of the image. This easily treats the original natural fiber interweaving and porous texture of the paper as noise and roughly erases it, resulting in a stiff "plastic" feel to the restored archival background, losing the historical material texture attributes that the archive should have. On the other hand, conventional deep learning models rely too much on pure pixel-level mapping relationships, making it difficult to correct the nonlinear diffusion and deformation of ink in porous paper media that follow the laws of physical fluid dynamics. This can easily cause stiff edges, broken strokes, or artifacts in the restored text. Summary of the Invention

[0004] Therefore, there is a need for a method to restore degraded archival images that can faithfully preserve the natural fiber texture of paper and accurately restore the edges of diffused writing.

[0005] Firstly, this application provides a method for restoring handwriting in degraded archival images, including:

[0006] S1. Obtain the degraded archive image, perform size and pixel scaling preprocessing on the degraded archive image to obtain a preprocessed image; perform structure tensor calculation on the preprocessed image to obtain the structure tensor matrix;

[0007] S2. Perform eigenvalue decomposition on the structure tensor matrix to obtain the eigenvector space orientation, and generate the initial diffusion direction field and the initial advection direction field based on the eigenvector space orientation.

[0008] S3. Input the preprocessed image into the dual-branch feature extraction network, and perform parallel feature extraction through the texture preservation branch and the ink structure branch to obtain the paper texture feature map and the ink structure feature map; wherein, the texture preservation branch is constructed based on discrete wavelet transform, and the ink structure branch is constructed based on the enhanced text edge attention mechanism and Swin-Transformer.

[0009] S4. Based on the channel attention-guided fusion mechanism, adaptive gating fusion of paper texture feature map and ink structure feature map is performed through global pooling and multilayer perceptron to output multimodal fusion feature map;

[0010] S5. Input the multimodal fusion feature map into the physical information neural network decoder, and use the initial diffusion direction field and the initial advection direction field as prior guidance to perform inverse physical evolution decoding based on the two-dimensional advection-diffusion equation, and output a high-definition restored image.

[0011] Secondly, this application also provides a degraded archival image handwriting restoration system for implementing the method described in the first aspect, comprising:

[0012] The image preprocessing module is used to acquire degraded archive images, perform size and pixel scaling preprocessing on the degraded archive images to obtain preprocessed images, and perform structure tensor calculation on the preprocessed images to obtain structure tensor matrices.

[0013] The physics initialization module is used to perform eigenvalue decomposition on the structure tensor matrix to obtain the eigenvector space orientation, and generate the initial diffusion direction field and the initial advection direction field based on the eigenvector space orientation.

[0014] The parallel feature extraction module is used to input the preprocessed image into the dual-branch feature extraction network, and perform parallel feature extraction through the texture preservation branch and the ink structure branch to obtain paper texture feature map and ink structure feature map;

[0015] The gated feature fusion module is used to perform adaptive gated fusion of paper texture feature map and ink structure feature map through global pooling and multilayer perceptron based on the channel attention-guided fusion mechanism, and output a multimodal fusion feature map.

[0016] The physical evolution decoding module is used to input the multimodal fusion feature map into the physical information neural network decoder. Guided by the initial diffusion direction field and the initial advection direction field, it performs inverse physical evolution decoding based on the two-dimensional advection-diffusion equation and outputs a high-definition restored image.

[0017] Thirdly, this application also provides a computer device, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement a method for restoring handwriting in a degraded archival image as described in the first aspect.

[0018] Fourthly, this application also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements a method for restoring handwriting in a degraded archival image as described in the first aspect.

[0019] The aforementioned method, system, device, and medium for restoring handwriting in degraded archival images generate an initial physical direction field through structural tensor decomposition, providing prior guidance for inverse physical evolution decoding and improving the convergence efficiency of physical constraints. Parallel extraction of texture branches using discrete wavelet transform and structural branches based on edge attention and Swin-Transformer enhances handwriting edges and global semantics while preserving paper fiber texture, improving stroke coherence. Adaptive weighting of two types of features through a channel attention fusion mechanism adjusts the contributions of texture and structure according to image content, preserving paper texture in the background and highlighting edge clarity in the handwriting area. Introducing advection-diffusion equation residual constraints through a physical information decoder guides the restoration process along a hydrodynamic inverse evolution path, reducing edge artifacts and improving restoration fidelity. Attached Figure Description

[0020] To more clearly illustrate the technical solutions in the embodiments or related technologies of this application, the accompanying drawings used in the description of the embodiments or related technologies will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0021] Figure 1 This is a flowchart illustrating a method for restoring handwriting in degraded archival images according to an embodiment of the present invention;

[0022] Figure 2 This is a schematic diagram of a degraded archival image handwriting restoration system according to an embodiment of the present invention. Detailed Implementation

[0023] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0024] In one embodiment, such as Figure 1As shown, a method for restoring handwriting in degraded archival images is provided. This embodiment illustrates the application of this method to a restoration terminal. It is understood that this method can also be applied to a server, and further to a cluster including both a restoration terminal and a server, and is implemented through interaction between the restoration terminal and the server. In this embodiment, the method may include the following steps:

[0025] S1. Obtain the degraded archive image, perform size and pixel scaling preprocessing on the degraded archive image to obtain a preprocessed image; perform structure tensor calculation on the preprocessed image to obtain the structure tensor matrix.

[0026] Specifically, degraded archival images can be digital images obtained by digitizing paper archives using image acquisition equipment. These digital images include paper fiber texture, ink marks, and deterioration features such as mold and yellowing. The structure tensor matrix can be a second-order symmetric matrix, which can be composed of the gradient vectors of the image within a local region and their outer product. The structure tensor matrix can be used to describe the edge direction and texture orientation of a local region of the image.

[0027] Optionally, the restoration terminal can acquire degraded archival images, which can be obtained by digitizing paper archives using an image acquisition device and then inputting the images into the restoration terminal. The restoration terminal can perform size scaling preprocessing on the degraded archival images, adjusting their spatial dimensions to a preset uniform resolution. The restoration terminal can perform pixel scaling preprocessing on the resized degraded archival images, linearly mapping the pixel values ​​of each pixel in the image from their original numerical range to a preset target numerical range, and outputting a preprocessed image. For each pixel location in the preprocessed image, the restoration terminal can construct a local neighborhood window centered on that pixel location and calculate the spatial gradient vector of each pixel within that local neighborhood window. The restoration terminal can sum and average the gradient cross products of all pixels within the local neighborhood window based on the spatial gradient vectors of each pixel, obtaining the structure tensor matrix at that pixel location. The restoration terminal can traverse all pixel locations in the preprocessed image to obtain the structure tensor matrix corresponding to each pixel location in the preprocessed image.

[0028] S2. Perform eigenvalue decomposition on the structure tensor matrix to obtain the eigenvector space orientation, and generate the initial diffusion direction field and the initial advection direction field based on the eigenvector space orientation.

[0029] Specifically, the spatial orientation of the feature vector can be the pointing direction of the feature vector in the two-dimensional image plane, which can reflect the gradient change direction or texture tangency of a local region of the image. The initial diffusion direction field can assign a diffusion direction vector to each pixel location in the image, and can be used to guide the anisotropic diffusion direction in the subsequent physical evolution process. Similarly, the initial advection direction field can assign an advection direction vector to each pixel location in the image, and can be used to guide the direction of information migration along a specific direction in the subsequent physical evolution process.

[0030] Optionally, the restoration terminal can perform eigenvalue decomposition on the structure tensor matrix at each pixel location of the preprocessed image, extracting the maximum eigenvalue, minimum eigenvalue, the first eigenvector corresponding to the maximum eigenvalue, and the second eigenvector corresponding to the minimum eigenvalue for each structure tensor matrix. The restoration terminal can calculate the local edge anisotropy intensity based on the difference between the maximum and minimum eigenvalues, generating an anisotropic gradient scalar field. The restoration terminal can map the scalar values ​​at each pixel location in the anisotropic gradient scalar field along the direction of the second eigenvector at that pixel location to obtain an initial diffusion direction field. This initial diffusion direction field can indicate the orientation of ink diffusion preferentially along the paper fiber direction. The restoration terminal can also map the scalar values ​​at each pixel location in the anisotropic gradient scalar field along the direction of the first eigenvector at that pixel location to obtain an initial advection direction field. This initial advection direction field can indicate the orientation of the handwriting edges contracting or migrating along the gradient direction.

[0031] S3. Input the preprocessed image into the dual-branch feature extraction network, and perform parallel feature extraction through the texture preservation branch and the ink structure branch to obtain the paper texture feature map and the ink structure feature map.

[0032] Specifically, the dual-branch feature extraction network can be a neural network structure comprising two parallel feature extraction paths. This neural network structure receives a preprocessed image as input and performs feature calculations on the preprocessed image through two independent feature extraction paths. The texture preservation branch can be the first feature extraction path in the dual-branch feature extraction network. This branch performs multi-scale decomposition of the input image based on discrete wavelet transform to extract high-frequency texture information from the image. The ink structure branch can be the second feature extraction path in the dual-branch feature extraction network. This branch extracts features from the input image based on an enhanced text edge attention mechanism and Swin-Transformer. The enhanced text edge attention mechanism can activate the structural contour map of character strokes from shallow features using edge extraction operators, and then use this structural contour map as a spatial attention mask to weight the feature map. Swin-Transformer is a neural network structure based on a shift window self-attention mechanism. This neural network structure achieves long-range dependency modeling between different positions within the feature map through non-local self-attention calculation. The paper texture feature map can be the feature map output by the texture preservation branch. The paper texture feature map includes microscopic texture information such as fiber interweaving and pore distribution on the paper surface. The ink structure feature map can be the feature map output by the ink structure branch. The ink structure feature map includes the edge contour of the characters, the stroke skeleton and the global character structure information.

[0033] Optionally, the restoration terminal can input the preprocessed image into a dual-branch feature extraction network. In the texture preservation branch, the restoration terminal can perform multi-level decomposition of the preprocessed image using two-dimensional discrete wavelet transform. In each level of decomposition, the feature map is decomposed into a low-frequency sub-band and multiple high-frequency sub-bands in various directions. The coefficients in the high-frequency sub-bands can preserve the microstructural information of the paper fiber texture. The restoration terminal can then pass the coefficients of each high-frequency sub-band to subsequent convolutional layers for nonlinear transformation, outputting a paper texture feature map. In the ink structure branch, the restoration terminal can extract shallow features from the preprocessed image through convolutional layers. Subsequently, in the enhanced text edge attention mechanism, edge extraction operators can be used to calculate the structural contour map of the character strokes from the shallow features. This structural contour map is then used as a spatial attention mask and multiplied element-wise with the shallow features, enabling the neural network to focus on the edges of the characters and the stroke regions. The restoration terminal can then input the attention-weighted feature map into a Swin-Transformer, and use a shifted window self-attention mechanism to perform global context modeling on the feature map, inferring the character skeleton obscured by the degradation region, and outputting an ink structure feature map.

[0034] S4. Based on the channel attention-guided fusion mechanism, adaptive gating fusion of paper texture feature map and ink structure feature map is performed through global pooling and multilayer perceptron to output multimodal fusion feature map.

[0035] Specifically, channel attention-guided fusion mechanisms can calculate and allocate attention weights to multiple input feature maps along the channel dimension, and adaptively determine the contribution ratio of different feature maps in the fusion based on the importance of each channel. Global pooling can calculate the global average or global maximum value of feature maps in the spatial dimension, and can compress the spatial information of each channel into a scalar value. A multilayer perceptron is a feedforward neural network structure that includes an input layer, one or more hidden layers, and an output layer. A multilayer perceptron can achieve non-linear mapping of the input vector through fully connected and non-linear activations between layers. Adaptive gating fusion refers to the process of dynamically calculating fusion weights based on the content of the input features and using these weights to perform a weighted combination of multiple feature maps.

[0036] Optionally, the repair terminal can concatenate the paper texture feature map and the ink structure feature map along the channel dimension to obtain a concatenated feature map. The number of channels in this concatenated feature map can be the sum of the number of channels in the paper texture feature map and the number of channels in the ink structure feature map. The repair terminal can perform global average pooling on the concatenated feature map, calculating the global average for each channel in the spatial dimension, compressing the spatial information of each channel into a scalar value, and outputting a global channel compression vector. The repair terminal can input the global channel compression vector into a multilayer perceptron. The multilayer perceptron can perform nonlinear mapping on the global channel compression vector through fully connected layers and nonlinear activation functions to generate channel attention weight vectors. The repair terminal can activate the channel attention weight vectors using the sigmoid function, mapping the values ​​of each element in the channel attention weight vectors to the interval between 0 and 1, generating an adaptive gated activation map. The repair terminal can perform a weighted summation of the feature maps of corresponding channels in the paper texture feature map and the ink structure feature map based on the weight values ​​in the adaptive gating activation map; the feature maps of channels with weight values ​​close to 1 are retained, while the feature maps of channels with weight values ​​close to 0 are suppressed, and a multimodal fusion feature map is output.

[0037] S5. Input the multimodal fusion feature map into the physical information neural network decoder, and use the initial diffusion direction field and the initial advection direction field as prior guidance to perform inverse physical evolution decoding based on the two-dimensional advection-diffusion equation, and output a high-definition restored image.

[0038] Specifically, the physical information neural network decoder can embed physical equations into the network's objective function as soft-constrained residuals during training, ensuring the network's output conforms to the evolution of the physical equations. Prior guidance involves inputting pre-calculated physical field information as initial or boundary conditions into the network before feature decoding, constraining the network's decoding path. The two-dimensional advection-diffusion equation can describe the evolution of a scalar field over time in two-dimensional space. This equation includes diffusion and advection terms; the diffusion term simulates the migration of a scalar from a high-concentration region to a low-concentration region, while the advection term simulates the velocity field migration of a scalar along a specific direction. Reverse physical evolution decoding utilizes the time-reverse evolution process of the two-dimensional advection-diffusion equation to reverse-engineer a degraded image from its current degraded state back to its clear state before degradation. The high-resolution restored image can be the digital image output by the physical information neural network decoder, where the handwriting edges are clear, the strokes are complete, and the paper texture is natural.

[0039] Optionally, the repair terminal can input the multimodal fused feature map into the physical information neural network decoder. The physical information neural network decoder includes a feature reconstruction layer and a physical constraint layer. In the feature reconstruction layer, the repair terminal can perform upsampling and convolutional decoding operations on the multimodal fused feature map to gradually restore the spatial resolution of the image. In the physical constraint layer, the repair terminal can use the initial diffusion direction field and the initial advection direction field as prior guidance, inputting them into the physical parameter prediction subnetwork of the physical information neural network decoder. The physical parameter prediction subnetwork can predict the locally equivalent inverse diffusion tensor field and the inverse advection velocity field based on the initial diffusion direction field and the initial advection direction field. Based on the predicted locally equivalent inverse diffusion tensor field and inverse advection velocity field, the repair terminal can calculate the spatial first-order gradient operator and the second-order Laplacian operator on the initial decoded feature map output by the feature reconstruction layer, and combine the spatial first-order gradient operator and the second-order Laplacian operator to obtain the intermediate features of the physical operators. The restoration terminal can calculate the residuals of the partial differential equations over the time steps of the two-dimensional advection-diffusion equations based on the intermediate features of the physical operators. These residuals are then embedded as constraint terms into the loss function of the physical information neural network decoder, forcing the decoder's output to be reconstructed along the inverse evolution path of hydrodynamics. The restoration terminal can then map the feature map output by the physical information neural network decoder to pixel space, outputting a high-resolution restored image.

[0040] For example, the expression for the two-dimensional advection-diffusion equation can be:

[0041]

[0042] in, Representing spatial coordinates In the evolutionary time Ink image density at that time; Indicates evolution time; Represents the gradient operator; This represents the diffusion tensor field, used to control the diffusion rate of ink in all directions; This represents the advection velocity field, used to guide the ink density to migrate along a specific direction; Indicates a local source term or absorber term; This represents the diffusion term, simulating the spontaneous anisotropic diffusion of ink into surrounding areas with lower concentrations. This represents the advection term, used to guide information to shift in a specific direction.

[0043] In this embodiment, a method for restoring handwriting in degraded archival images is provided. An initial physical direction field is generated through structural tensor decomposition, which provides prior guidance for inverse physical evolution decoding, improving the convergence efficiency of physical constraints. Parallel extraction of texture branches using discrete wavelet transform and structural branches based on edge attention and Swin-Transformer enhances handwriting edges and global semantics while preserving paper fiber texture, improving stroke coherence. Adaptive weighting of two types of features through channel attention fusion adjusts the contributions of texture and structure according to image content, preserving paper texture in the background and highlighting edge clarity in the handwriting area. Introducing advection-diffusion equation residual constraints through a physical information decoder guides the restoration process along a hydrodynamic inverse evolution path, reducing edge artifacts and improving restoration fidelity.

[0044] In one embodiment, the training and optimization steps of the dual-branch feature extraction network and the physical information neural network decoder may include:

[0045] S61. Obtain a real high-definition image. Input the high-definition restored image and the real high-definition image into a global-local dual discriminator. Perform feature extraction and downsampling on the high-definition restored image and the real high-definition image through a convolutional layer. Output the first discrimination score corresponding to the high-definition restored image and the second discrimination score corresponding to the real high-definition image.

[0046] Specifically, the real high-resolution image can be a clear, undegraded image corresponding to a degraded archival image, and can serve as a supervisory signal during training. The global-local dual discriminator can be a neural network structure comprising two discriminative branches. The global discriminative branch takes the entire image as input and judges the overall structure and semantic coherence of the image as real or fake; the local discriminative branch takes randomly cropped local patches from the image as input and judges the texture authenticity of local regions of the image. Convolutional layers can be the basic computational units in the global-local dual discriminator, performing sliding window operations on the input image or feature map through convolutional kernels to extract the spatial features of the image. The first discrimination score can be the numerical value output by the global-local dual discriminator after feature extraction and downsampling of the high-resolution restored image, representing the probability that the high-resolution restored image is judged as a real image. The second discrimination score can also be the numerical value output by the global-local dual discriminator after feature extraction and downsampling of the real high-resolution image, representing the probability that the real high-resolution image is judged as a real image.

[0047] Optionally, the restoration terminal can acquire a real high-resolution image. The restoration terminal can input the high-resolution restored image into a global-local dual discriminator. The global discriminator branch performs layer-by-layer feature extraction and downsampling on the high-resolution restored image through convolutional layers, and outputs the global discriminant score corresponding to the high-resolution restored image at the last layer. The local discriminator branch can randomly crop multiple local patches from the high-resolution restored image, and perform layer-by-layer feature extraction and downsampling on each local patch through convolutional layers, outputting the local discriminant score corresponding to each local patch at the last layer. Alternatively, the restoration terminal can input a real high-resolution image into the global-local dual discriminator. The global discriminator branch performs layer-by-layer feature extraction and downsampling on the real high-resolution image through convolutional layers, outputting the global discriminant score corresponding to the real high-resolution image at the last layer. The local discriminator branch can randomly crop multiple local patches from the real high-resolution image, and perform layer-by-layer feature extraction and downsampling on each local patch through convolutional layers, outputting the local discriminant score corresponding to each local patch at the last layer. The restoration terminal can merge the global discrimination score corresponding to the high-definition restored image and the local discrimination scores corresponding to each local patch into a first discrimination score, and can merge the global discrimination score corresponding to the real high-definition image and the local discrimination scores corresponding to each local patch into a second discrimination score.

[0048] S62. Calculate the first hinge loss based on the first discrimination score and the preset false label, and calculate the second hinge loss based on the second discrimination score and the preset true label; take the sum of the first hinge loss and the second hinge loss as the hinge resistance loss.

[0049] Specifically, the preset fake label can be the target label value corresponding to the high-resolution restored image in adversarial training. The preset fake label can be used to train the discriminator to identify the generated image as fake. The preset real label can be the target label value corresponding to the real high-resolution image in adversarial training. The preset real label can be used to train the discriminator to identify the real image as real. The first hinge loss can be the loss value calculated by the hinge loss function based on the difference between the first discrimination score and the preset fake label. The second hinge loss can be the loss value calculated by the hinge loss function based on the difference between the second discrimination score and the preset real label. The hinge adversarial loss can be used to update the network parameters of the global-local dual discriminator to improve the discrimination ability, and the hinge adversarial loss can be used to update the network parameters of the dual-branch feature extraction network and the physical information neural network decoder, so that the discrimination score of the high-resolution restored image in the discriminator approaches the preset real label.

[0050] S63. Calculate the absolute difference pixel by pixel between the high-definition restored image and the real high-definition image to obtain the L1 pixel reconstruction loss.

[0051] Specifically, the L1 pixel reconstruction loss can be the mean of the pixel-wise absolute error between the high-definition restored image and the real high-definition image. The L1 pixel reconstruction loss can be used to measure the difference between the high-definition restored image and the real high-definition image in pixel space, and constrain the accurate restoration of brightness and color of the restored image.

[0052] Optionally, the restoration terminal can calculate the absolute difference of pixel values ​​at the same pixel location between the high-resolution restored image and the original high-resolution image. The restoration terminal can sum the absolute differences at all pixel locations and divide the sum by the total number of pixels in the image to obtain the L1 pixel reconstruction loss.

[0053] S64. Input the high-definition restored image and the real high-definition image into the pre-trained VGG network, respectively, extract the high-level feature maps corresponding to the high-definition restored image and the real high-definition image, calculate the mean square error between the high-level feature maps, and obtain the VGG perceptual loss.

[0054] Specifically, the pre-trained VGG network can be a pre-trained VGG convolutional neural network, which can extract multi-level feature representations of the image. High-level feature maps can be feature maps output by convolutional layers near the output end of the VGG network, encoding the semantic structure and texture information of the image. The VGG perceptual loss can be the average of the squared element-wise differences between the high-level feature maps of the reconstructed image and the real high-level image. The VGG perceptual loss can be used to constrain the reconstructed image to maintain consistency with the real high-level image in the feature space, improving the semantic recognizability of the reconstruction result.

[0055] Optionally, the restoration terminal can input a high-resolution restored image into a pre-trained VGG network. The VGG network can perform layer-by-layer convolution and pooling operations on the high-resolution restored image. The restoration terminal can extract feature maps from preset layers of the VGG network to obtain high-level feature maps corresponding to the high-resolution restored image. Alternatively, the restoration terminal can input a real high-resolution image into the pre-trained VGG network. The VGG network can perform layer-by-layer convolution and pooling operations on the real high-resolution image. The restoration terminal can extract feature maps from the same preset layer of the VGG network to obtain high-level feature maps corresponding to the real high-resolution image. The restoration terminal can calculate the average of the squared element-wise differences between the high-level feature maps corresponding to the high-resolution restored image and the high-level feature maps corresponding to the real high-resolution image to obtain the VGG perceptual loss.

[0056] S65. Calculate the Gram matrix of the high-level feature map of the high-definition restored image and the high-level feature map of the real high-definition image respectively, and calculate the mean square error of the Gram matrix to obtain the Gram matrix texture loss.

[0057] Specifically, the Gram matrix can be the inner product matrix between channels of high-level feature maps. This inner product matrix measures the second-order statistical properties of the texture by calculating the cross-correlation between the channels of the high-level feature maps. It does not depend on the specific spatial location of pixels and can characterize statistical features such as texture coarseness and density. The Gram matrix texture loss can be the average of the squared differences between the Gram matrix of the high-level feature map of the high-definition restored image and the Gram matrix of the high-level feature map of the real high-definition image. The Gram matrix texture loss can be used to constrain the restored image to maintain consistency with the real high-definition image in terms of texture statistical properties.

[0058] Optionally, the restoration terminal can acquire the high-level feature map corresponding to the high-resolution restored image. The restoration terminal can vectorize the high-level feature map corresponding to the high-resolution restored image along the channel dimension. For any two channels, the restoration terminal can calculate the inner product between the two channel vectors, using the inner product value as the element value at the corresponding position in the Gram matrix to construct the Gram matrix of the high-level feature map corresponding to the high-resolution restored image. Alternatively, the restoration terminal can vectorize the high-level feature map corresponding to the real high-resolution image along the channel dimension. For any two channels, the restoration terminal can calculate the inner product between the two channel vectors to construct the Gram matrix of the high-level feature map corresponding to the real high-resolution image. The restoration terminal can calculate the average of the squared element-wise differences between the Gram matrix of the high-resolution restored image and the Gram matrix of the real high-resolution image to obtain the Gram matrix texture loss.

[0059] S66. Obtain the initial decoded feature map, local equivalent inverse diffusion tensor field, and reverse advection velocity field generated during the forward propagation of the physical information neural network decoder; based on the local equivalent inverse diffusion tensor field and reverse advection velocity field, calculate the mean square error of the partial differential equation residual tensor of the initial decoded feature map on the time step of the two-dimensional advection-diffusion equation, and obtain the partial differential equation constraint loss.

[0060] Specifically, the initial decoded feature map can be the feature map output by the feature reconstruction layer of the physical information neural network decoder. The local equivalent inverse diffusion tensor field can be a tensor field predicted by the physical parameter prediction subnetwork based on the initial diffusion direction field. This tensor field can assign an inverse diffusion tensor to each pixel position in the image, and the inverse diffusion tensor can describe the rate and direction of anisotropic diffusion when evolving backward from the degraded state. The inverse advection velocity field can be a vector field predicted by the physical parameter prediction subnetwork based on the initial advection direction field. This vector field can assign an inverse advection velocity vector to each pixel position in the image, and the inverse advection velocity vector can describe the speed and direction of ink concentration migration along a specific direction when evolving backward from the degraded state. The partial differential equation residual tensor can be a tensor formed by the difference between the left and right sides of the equation after substituting the intermediate features of the physical operator into the two-dimensional advection-diffusion equation. This tensor can reflect the degree to which the initial decoded feature map deviates from the evolution law of the two-dimensional advection-diffusion equation. The partial differential equation constraint loss can be the average of the squares of each element in the partial differential equation residual tensor. The partial differential equation constraint loss can be quantified in scalar form to the degree to which the initial decoded feature map satisfies the two-dimensional advection-diffusion equation.

[0061] Optionally, the repair terminal can acquire the initial decoded feature map, the locally equivalent inverse diffusion tensor field, and the reverse advection velocity field generated during the forward propagation of the physical information neural network decoder. The repair terminal can calculate the first-order partial derivatives of the initial decoded feature map in both the horizontal and vertical spatial coordinate directions to obtain the first-order spatial gradient operator. The repair terminal can calculate the second-order partial derivatives of the initial decoded feature map in both the horizontal and vertical spatial coordinate directions, and add the second-order partial derivatives in the horizontal and vertical spatial coordinate directions to obtain the second-order Laplace operator. The repair terminal can perform tensor contraction operations on the locally equivalent inverse diffusion tensor field and the first-order spatial gradient operator to obtain the advection contribution term. The repair terminal can perform a combined operation on the second-order Laplace operator and the reverse advection velocity field to obtain the diffusion contribution term. The repair terminal can add the advection contribution term and the diffusion contribution term to obtain the intermediate features of the physical operator. The repair terminal can use the intermediate features of the physical operator as the equation residuals on the time step of the two-dimensional advection-diffusion equation to obtain the partial differential equation residual tensor. The repair terminal can square all elements in the residual tensor of the partial differential equation, sum all the squared elements, and divide the sum by the total number of elements in the residual tensor of the partial differential equation to obtain the constraint loss of the partial differential equation.

[0062] S67. The hinge adversarial loss, L1 pixel reconstruction loss, VGG perception loss, Gram matrix texture loss, and partial differential equation constraint loss are weighted and summed according to preset dynamic weight coefficients to obtain the multi-objective joint loss.

[0063] Specifically, the preset dynamic weight coefficients can be numerical weights assigned to each loss term during training. These dynamic weight coefficients can be used to balance the contribution of each loss term to the total loss.

[0064] Optionally, the repair terminal can obtain preset hinge adversarial loss weight coefficients, L1 pixel reconstruction loss weight coefficients, VGG perceptual loss weight coefficients, Gram matrix texture loss weight coefficients, and partial differential equation constraint loss weight coefficients. The repair terminal can multiply the hinge adversarial loss by the hinge adversarial loss weight coefficients to obtain a weighted hinge adversarial loss. The repair terminal can multiply the L1 pixel reconstruction loss by the L1 pixel reconstruction loss weight coefficients to obtain a weighted L1 pixel reconstruction loss. The repair terminal can multiply the VGG perceptual loss by the VGG perceptual loss weight coefficients to obtain a weighted VGG perceptual loss. The repair terminal can multiply the Gram matrix texture loss by the Gram matrix texture loss weight coefficients to obtain a weighted Gram matrix texture loss. The repair terminal can multiply the partial differential equation constraint loss by the partial differential equation constraint loss weight coefficients to obtain a weighted partial differential equation constraint loss. The repair terminal can add the weighted hinge adversarial loss, weighted L1 pixel reconstruction loss, weighted VGG perceptual loss, weighted Gram matrix texture loss, and weighted partial differential equation constraint loss to obtain a multi-objective joint loss.

[0065] S68. Based on the multi-objective joint loss function, perform multi-stage progressive gradient updates on the dual-branch feature extraction network, physical information neural network decoder, and global-local dual discriminator, and output updated network parameters.

[0066] Specifically, multi-stage progressive gradient update can divide the training process into multiple stages, freezing or activating different network components and loss terms at different stages to sequentially perform gradient descent optimization. Updating the network parameters can be the values ​​of the trainable weights of each layer of the network after the gradient update.

[0067] Optionally, the repair terminal can perform multi-stage progressive gradient updates on the bi-branch feature extraction network, the physical information neural network decoder, and the global-local dual discriminator based on a multi-objective joint loss function. In each stage, the repair terminal can calculate the parameter gradients of each network component through backpropagation and update the network parameters using an optimizer based on the calculated gradients. The repair terminal can output the updated network parameters after all stages are completed.

[0068] S69. Repeatedly execute multi-stage progressive gradient updates until the absolute value of the change in the multi-objective joint loss function within consecutive preset rounds is less than a preset threshold; update the dual-branch feature extraction network and the physical information neural network decoder by updating the network parameters.

[0069] Specifically, the absolute value of the change in the multi-objective joint loss function within consecutive preset rounds being less than a preset threshold can mean that the value of the multi-objective joint loss function no longer decreases significantly in consecutive iteration rounds. This condition can be used to determine whether the training has converged.

[0070] Optionally, the repair terminal can repeatedly execute the multi-stage progressive gradient update process in step S68. The repair terminal can perform multi-stage progressive gradient updates in each iteration and record the value of the multi-objective joint loss function after each iteration. The repair terminal can calculate the absolute value of the difference between the current iteration and the previous iteration's multi-objective joint loss function value. When the repair terminal detects that the absolute value of the change in the multi-objective joint loss function value is less than a preset threshold in a consecutive preset number of iterations, the repair terminal can stop repeating the process. The repair terminal can output the network parameters of the current dual-branch feature extraction network and the network parameters of the physical information neural network decoder as the final network parameters, and can use the updated dual-branch feature extraction network and the updated physical information neural network decoder to execute a degraded archival image handwriting restoration method.

[0071] This embodiment provides a method for restoring handwriting in degraded archival images. It extracts features from the entire image and randomly cropped local patches using global and local discriminative branches, respectively, and outputs discriminative scores. This allows for adversarial supervision of the restoration results on both the overall semantic coherence and local texture realism scales, improving the realism of the restored image at both the macroscopic structure and microscopic detail levels. A multi-objective joint loss, composed of hinge adversarial loss, L1 pixel reconstruction loss, VGG perceptual loss, Gram matrix texture loss, and partial differential equation constraint loss, jointly constrains the restoration results from multiple dimensions, including pixel spatial alignment, feature semantic recognition, texture statistical matching, and physical evolution consistency. This improves the overall quality of the restored image in terms of brightness fidelity, semantic recognizability, fiber texture naturalness, and edge evolution rationality. A multi-stage progressive gradient update strategy gradually activates different network components and loss terms at different training stages, mitigating gradient conflicts between multiple competing loss objectives and improving the stability of the training process and the generalization ability of the restoration network.

[0072] In one embodiment, S2 may include:

[0073] S21. Perform eigenvalue decomposition on the structure tensor matrix to extract the maximum eigenvalue, the minimum eigenvalue, the first eigenvector corresponding to the maximum eigenvalue, and the second eigenvector corresponding to the minimum eigenvalue.

[0074] Specifically, the direction of the first feature vector represents the image gradient direction, and the direction of the second feature vector represents the local texture tangency.

[0075] Specifically, the largest eigenvalue is the larger of the two eigenvalues ​​obtained after eigenvalue decomposition of the structure tensor matrix. The magnitude of the largest eigenvalue reflects the intensity of image changes along the gradient direction. The smallest eigenvalue is the smaller of the two eigenvalues ​​obtained after eigenvalue decomposition of the structure tensor matrix. The magnitude of the smallest eigenvalue reflects the intensity of image changes along the local texture tangent direction.

[0076] Optionally, the repair terminal can perform eigenvalue decomposition on the structure tensor matrix at each pixel location of the preprocessed image. For each structure tensor matrix, the repair terminal can solve for the characteristic polynomial of the structure tensor matrix to obtain the maximum and minimum eigenvalues. The repair terminal can substitute the maximum eigenvalue into the equation relating the structure tensor matrix and its eigenvalues ​​to obtain the first eigenvector corresponding to the maximum eigenvalue. The repair terminal can substitute the minimum eigenvalue into the equation relating the structure tensor matrix and its eigenvalues ​​to obtain the second eigenvector corresponding to the minimum eigenvalue.

[0077] S22. Calculate the local edge anisotropy intensity based on the difference between the maximum and minimum eigenvalues, and generate anisotropic gradient scalar field.

[0078] Specifically, the local edge anisotropy intensity can describe the directional selectivity of edges in local image regions: when there are obvious edges in a local region, the difference between the maximum and minimum eigenvalues ​​is large, and the anisotropy intensity is high; when the local region is a flat region or a region with isotropic texture, the difference between the maximum and minimum eigenvalues ​​is small, and the anisotropy intensity is low. An anisotropic gradient scalar field can assign an anisotropic intensity value to each pixel location in the preprocessed image.

[0079] Optionally, for each pixel location in the preprocessed image, the repair terminal can extract the maximum and minimum eigenvalues ​​of the structure tensor matrix at that pixel location, and calculate the difference between the maximum and minimum eigenvalues, using this difference as the local edge anisotropy intensity at that pixel location. The repair terminal can traverse all pixel locations in the preprocessed image, arranging the calculated local edge anisotropy intensities at each pixel location according to the pixel location to form an anisotropic gradient scalar field.

[0080] S23. Map the anisotropic gradient scalar field along the direction of the second eigenvector to obtain the initial diffusion direction field; map the anisotropic gradient scalar field along the direction of the first eigenvector to obtain the initial advection direction field.

[0081] For example, the formula for calculating the initial diffusion direction field can be:

[0082]

[0083] in, Representing spatial coordinates The initial diffusion direction field at the location; The largest eigenvalue, It is the smallest eigenvalue; This represents the diffusion attenuation adjustment coefficient; This represents the eigenvector corresponding to the smallest eigenvalue.

[0084] For example, the formula for calculating the initial advection direction field can be:

[0085]

[0086] in, Representing spatial coordinates The initial advection direction field at the location; This represents the eigenvector corresponding to the largest eigenvalue; This represents the image gradient operator for preprocessed images; This represents the scaling factor for advection velocity.

[0087] Optionally, for each pixel location in the preprocessed image, the repair terminal can obtain the anisotropic gradient scalar value at that pixel location, as well as the maximum eigenvalue, minimum eigenvalue, and second eigenvector of the structure tensor matrix at that pixel location. The repair terminal can calculate the square of the difference between the maximum and minimum eigenvalues, and multiply the diffusion attenuation adjustment coefficient by this squared value, taking the negative of the exponential function to obtain the diffusion attenuation factor. The repair terminal can multiply the diffusion attenuation factor by the outer product of the second eigenvector and the transpose of the second eigenvector to obtain the initial diffusion direction field tensor at that pixel location. The repair terminal can traverse all pixel locations in the preprocessed image, arranging the initial diffusion direction fields at all pixel locations according to their pixel locations to form the initial diffusion direction field. For each pixel location in the preprocessed image, the repair terminal can obtain the first eigenvector of the structure tensor matrix at that pixel location and calculate the image gradient operator of the preprocessed image at that pixel location. The repair terminal can multiply the advection velocity scaling factor by the image gradient operator and the first eigenvector to obtain the initial advection direction field vector at that pixel location. The repair terminal can traverse all pixel positions of the preprocessed image and arrange the initial advection direction field at all pixel positions according to the pixel position to form the initial advection direction field.

[0088] In this embodiment, a method for restoring handwriting in degraded archival images is provided. By performing eigenvalue decomposition on the structural tensor matrix to extract the maximum and minimum eigenvalues, as well as the corresponding first and second eigenvectors, a diffusion direction field and a flow direction field can be constructed based on the directional information of the local structure of the image. This is beneficial for providing physical prior guidance that matches the image content for the subsequent physical information neural network decoder. By calculating the difference between the maximum and minimum eigenvalues ​​to generate an anisotropic gradient scalar field, higher anisotropic intensity can be obtained in edge regions to enhance directional constraints, while lower anisotropic intensity can be obtained in flat regions to avoid over-constraint, which is beneficial for improving the adaptability of the physical prior field in different regions of the image. By mapping the anisotropic gradient scalar field along the direction of the second eigenvector to obtain the initial diffusion direction field, and mapping the anisotropic gradient scalar field along the direction of the first eigenvector to obtain the initial flow direction field, the diffusion process can be preferentially carried out along the local texture tangential direction while the flow process can be preferentially carried out along the gradient direction. This is beneficial for matching the physical evolution path with the actual diffusion mechanism of ink in porous media, thereby improving the accuracy of handwriting edge reconstruction during the restoration process.

[0089] In one embodiment, S4 may include:

[0090] S41. Perform channel-dimensional splicing on the paper texture feature map and the ink structure feature map to obtain the spliced ​​feature map.

[0091] Specifically, channel-dimensional stitching can be an operation that connects two or more feature maps with the same spatial height and spatial width along the channel dimension. The spatial height and spatial width of the connected feature map remain unchanged, and the number of channels is the sum of the number of channels of each feature map involved in the stitching.

[0092] Optionally, the repair terminal can acquire a paper texture feature map and an ink structure feature map. The paper texture feature map has a preset spatial height, spatial width, and a first number of channels; the ink structure feature map has the same spatial height, the same spatial width, and a second number of channels. The repair terminal can connect the paper texture feature map and the ink structure feature map along the channel dimension to generate a spliced ​​feature map. The spatial height of the spliced ​​feature map is equal to the spatial height of the paper texture feature map, the spatial width of the spliced ​​feature map is equal to the spatial width of the paper texture feature map, and the number of channels of the spliced ​​feature map is equal to the sum of the first number of channels and the second number of channels.

[0093] S42. Perform global average pooling on the spliced ​​feature map to obtain the global channel compression vector.

[0094] Specifically, global average pooling can calculate the arithmetic mean of pixel values ​​at all spatial locations for each channel of the feature map in the spatial dimension, and can compress the spatial information of each channel into a scalar value. The length of the global channel compression vector is equal to the number of channels of the input feature map, and each element in the global channel compression vector corresponds to the global average response value of one channel.

[0095] Optionally, the repair terminal can sum all pixel values ​​within each channel of the stitched feature map in both the spatial height and spatial width dimensions, and divide the sum by the product of the spatial height and spatial width to obtain the global average response value of that channel. The repair terminal can then arrange the global average response values ​​of all channels of the stitched feature map in channel order to form a global channel compression vector.

[0096] S43. Input the global channel compression vector into the multilayer perceptron for nonlinear mapping to generate channel attention weight vectors, and activate the channel attention weight vectors through the Sigmoid function to generate an adaptive gating activation map.

[0097] Specifically, the channel attention weight vector can be a vector output by a multilayer perceptron, where each element corresponds to the attention weight value of a channel. The adaptive gating activation map can be a gating signal obtained by activating the channel attention weight vector with a sigmoid function. This gating signal can be used to control the degree of preservation or suppression of each channel feature map during the fusion process.

[0098] Optionally, the repair terminal can input the global channel compression vector into a multilayer perceptron. The multilayer perceptron may include a first fully connected layer, a first activation function, a second fully connected layer, and a second activation function. The repair terminal can perform a linear transformation on the global channel compression vector in the first fully connected layer of the multilayer perceptron, outputting a first intermediate vector. The repair terminal can perform a non-linear transformation on the first intermediate vector using the first activation function, outputting a second intermediate vector. The repair terminal can perform a linear transformation on the second intermediate vector in the second fully connected layer of the multilayer perceptron, outputting a third intermediate vector. The repair terminal can perform a non-linear transformation on the third intermediate vector using the second activation function, outputting a channel attention weight vector. The length of the channel attention weight vector is the same as the number of channels in the concatenated feature map. The repair terminal can input the channel attention weight vector into a sigmoid function. The sigmoid function can perform exponential and normalization operations on each element in the channel attention weight vector, mapping each element value to the interval between 0 and 1, and outputting an adaptive gated activation map.

[0099] S44. Based on the adaptive gating activation map, perform channel weighted summation on the paper texture feature map and the ink structure feature map to obtain the multimodal fusion feature map.

[0100] Specifically, the channel weighted summation can be a process of multiplying the channel weight values ​​in the adaptive gating activation map with the corresponding channel feature maps in the spliced ​​feature map channel by channel to obtain a weighted feature map, and then summing the weighted feature maps of all channels along the channel dimension to output a single feature map.

[0101] Optionally, the repair terminal can multiply the channel weights in the adaptive gated activation map channel by channel in order with the corresponding channel feature maps in the stitched feature map to obtain a weighted feature map. In the weighted feature map, the channels from the paper texture feature map and the channels from the ink structure feature map can be multiplied by their corresponding weights in the adaptive gated activation map. The repair terminal can sum all channels of the weighted feature map along the channel dimension to output a multimodal fusion feature map. In the multimodal fusion feature map, the feature information corresponding to channels with weights close to 1 is preserved, while the feature information corresponding to channels with weights close to 0 is suppressed.

[0102] In this embodiment, a method for restoring handwriting in degraded archival images is provided. By concatenating the paper texture feature map and the ink structure feature map along the channel dimension and then performing global average pooling, the spatial information of each channel can be compressed into a global response value. This is beneficial for capturing the overall activation level of each channel and providing a global statistical basis for subsequent weight allocation. A multilayer perceptron is used to perform nonlinear mapping on the global channel compression vector and generate an adaptive gated activation map through sigmoid activation. The fusion weights can be dynamically calculated based on the global response features of each channel, which is beneficial for adaptive selection and weighted combination of feature channels. By performing channel weighted summation on the paper texture feature map and the ink structure feature map according to the adaptive gated activation map, the texture channel weight can be amplified in the background area to preserve the paper fiber texture, while the structure channel weight can be amplified in the handwriting area to enhance edge sharpness. This facilitates a balanced fusion of paper texture and ink structure in different image regions.

[0103] In one embodiment, S68 may include:

[0104] S681. In the first stage of progressive training, the global-local dual discriminator is frozen. The gradient of the dual-branch feature extraction network and the physical information neural network decoder is updated through L1 pixel reconstruction loss and VGG perception loss to obtain the first intermediate network parameters.

[0105] Specifically, the first stage of progressive training can be the first stage of the entire training process. Freezing the global-local dual discriminator means that the network parameters of the global-local dual discriminator are not updated during training. The global-local dual discriminator calculates the discrimination score normally during forward propagation, but does not calculate the gradient. The network parameters of the global-local dual discriminator remain unchanged during backpropagation. The first intermediate network parameters can be the network parameter values ​​of the dual-branch feature extraction network and the physical information neural network decoder after the first stage of progressive training.

[0106] Optionally, the repair terminal can freeze the global-local dual discriminator at the start of the first stage of progressive training, preventing gradient calculation during forward propagation. The repair terminal can calculate the total loss using L1 pixel reconstruction loss and VGG perceptual loss, without introducing hinge adversarial loss, Gram matrix texture loss, or partial differential equation constraint loss. The repair terminal can calculate the gradients of the network parameters of the dual-branch feature extraction network and the physical information neural network decoder using backpropagation. The repair terminal can then use an optimizer to update the network parameters of the dual-branch feature extraction network and the physical information neural network decoder based on the calculated gradients. The repair terminal can repeat the above process until the preset number of iterations in the first stage is completed, outputting the first intermediate network parameters.

[0107] S682. In the second stage of progressive training, the global-local dual discriminator is unfrozen, the first intermediate network parameters are loaded, and the dual-branch feature extraction network, physical information neural network decoder and global-local dual discriminator are alternately trained through L1 pixel reconstruction loss, VGG perceptual loss, hinge adversarial loss and Gram matrix texture loss to obtain the second intermediate network parameters.

[0108] Specifically, the second-stage progressive training can be the second stage of the entire training process. Unfreezing the global-local dual discriminator refers to restoring the network parameter update capability of the global-local dual discriminator during training. Loading the first intermediate network parameters can be assigning the first intermediate network parameters obtained in the first stage of training to the dual-branch feature extraction network and the physical information neural network decoder as the current initial parameters. The second intermediate network parameters can be the network parameter values ​​of the dual-branch feature extraction network, the physical information neural network decoder, and the global-local dual discriminator after the second-stage progressive training.

[0109] Optionally, the repair terminal can unfreeze the global-local dual discriminator at the start of the second phase of progressive training, restoring its gradient calculation and parameter update capabilities. The repair terminal can load the first intermediate network parameters into the dual-branch feature extraction network and the physical information neural network decoder as initial values. The repair terminal can perform alternating adversarial training in each iteration. In the first sub-step, the repair terminal can fix the parameters of the dual-branch feature extraction network and the physical information neural network decoder, input the high-resolution repaired image and the real high-resolution image into the global-local dual discriminator, calculate the discriminator loss value through hinge adversarial loss, and update the network parameters of the global-local dual discriminator through backpropagation. In the second sub-step, the repair terminal can fix the parameters of the global-local dual discriminator, calculate the generator loss value through L1 pixel reconstruction loss, VGG perceptual loss, hinge adversarial loss, and Gram matrix texture loss, and update the network parameters of the dual-branch feature extraction network and the physical information neural network decoder through backpropagation. The repair terminal can repeat the above alternating adversarial training process until the preset number of iterations in the second phase is completed, outputting the second intermediate network parameters.

[0110] S683. In the third stage of progressive training, the second intermediate network parameters are loaded, and the dual-branch feature extraction network, physical information neural network decoder and global-local dual discriminator are jointly fine-tuned through multi-objective joint loss, and the updated network parameters are output.

[0111] Specifically, the third-stage progressive training can be the third stage of the entire training process. Loading the second intermediate network parameters refers to assigning the second intermediate network parameters obtained from the second-stage training to the dual-branch feature extraction network, the physical information neural network decoder, and the global-local dual discriminator as the current initial parameters. Updating the network parameters can be the final network parameter values ​​of all network components after the third-stage progressive training.

[0112] Optionally, at the start of the third-stage progressive training, the repair terminal can load the second intermediate network parameters into the bi-branch feature extraction network, the physical information neural network decoder, and the global-local dual discriminator. In each iteration, the repair terminal can calculate the hinge adversarial loss, L1 pixel reconstruction loss, VGG perceptual loss, Gram matrix texture loss, and partial differential equation constraint loss. It can then weight and sum these five losses according to preset dynamic weight coefficients to obtain the multi-objective joint loss value. This partial differential equation constraint loss was not included in the training in the second stage but is introduced for the first time in the third stage. Based on the multi-objective joint loss value, the repair terminal can simultaneously calculate the network parameter gradients of the bi-branch feature extraction network, the physical information neural network decoder, and the global-local dual discriminator through backpropagation. The repair terminal can use an optimizer to simultaneously update the network parameters of the bi-branch feature extraction network, the physical information neural network decoder, and the global-local dual discriminator based on the calculated network parameter gradients. The repair terminal can repeat the above joint fine-tuning process until the preset number of iterations in the third stage is completed, and then output the updated network parameters.

[0113] In this embodiment, a method for restoring handwriting in degraded archival images is provided. In the first stage, the global-local dual discriminator is frozen, and gradient updates are performed using only L1 pixel reconstruction loss and VGG perceptual loss. This allows for the early establishment of basic brightness alignment and semantic mapping from the degraded image to the clear image in the generative network, providing stable initial parameters for the introduction of subsequent complex loss terms. In the second stage, the global-local dual discriminator is unfrozen, and alternating adversarial training is performed using hinge adversarial loss and Gram matrix texture loss. This allows for the gradual introduction of adversarial game theory and texture statistical constraints while maintaining the stability of the basic mapping, which helps refine the microstructure of paper fibers and improve the overall realism of the restored image. In the third stage, second intermediate network parameters are loaded, and joint fine-tuning is performed using a multi-objective joint loss, including partial differential equation constraint loss. This introduces physical evolution constraints based on the mapping and texture established in the first two stages, facilitating fluid-mechanically consistent inverse reconstruction of ink edges and improving the edge accuracy of the restored result.

[0114] It should be understood that although the steps in the flowcharts of the embodiments described above are shown sequentially according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the flowcharts of the embodiments described above may include multiple steps or multiple stages. These steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these steps or stages is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the steps or stages of other steps.

[0115] Based on the same inventive concept, this application also provides a system for implementing the aforementioned method for restoring handwriting in degraded archival images. The solution provided by this system is similar to the solution described in the above method; therefore, the specific limitations of one or more embodiments of a degraded archival image handwriting restoration system provided below can be found in the above-described limitations of the method for restoring handwriting in degraded archival images, and will not be repeated here.

[0116] In one exemplary embodiment, such as Figure 2 As shown, a degraded archival image handwriting restoration system 200 is provided to implement the methods in the above-described method embodiments. This system may include:

[0117] Image preprocessing module 201 is used to acquire degraded archive images, perform size and pixel scaling preprocessing on the degraded archive images to obtain preprocessed images, and perform structure tensor calculation on the preprocessed images to obtain structure tensor matrices.

[0118] The physics field initialization module 202 is used to perform eigenvalue decomposition on the structure tensor matrix to obtain the eigenvector space orientation, and generate the initial diffusion direction field and the initial advection direction field based on the eigenvector space orientation.

[0119] Parallel feature extraction module 203 is used to input the preprocessed image into the dual-branch feature extraction network, and perform parallel feature extraction through the texture preservation branch and the ink structure branch to obtain paper texture feature map and ink structure feature map;

[0120] The gated feature fusion module 204 is used to perform adaptive gated fusion of paper texture feature map and ink structure feature map through global pooling and multilayer perceptron based on the channel attention-guided fusion mechanism, and output a multimodal fusion feature map.

[0121] The physical evolution decoding module 205 is used to input the multimodal fusion feature map into the physical information neural network decoder, and perform inverse physical evolution decoding based on the two-dimensional advection-diffusion equation with the initial diffusion direction field and the initial advection direction field as prior guidance, and output a high-definition restored image.

[0122] In one embodiment, a degraded archival image handwriting restoration system 200 further includes a network training and optimization module, which can be used to:

[0123] A real high-definition image is acquired, and the high-definition restored image and the real high-definition image are respectively input into a global-local dual discriminator. The high-definition restored image and the real high-definition image are respectively subjected to feature extraction and downsampling through convolutional layers, and the first discrimination score corresponding to the high-definition restored image and the second discrimination score corresponding to the real high-definition image are output.

[0124] The first hinge loss is calculated based on the first discrimination score and the preset false label, and the second hinge loss is calculated based on the second discrimination score and the preset true label; the sum of the first hinge loss and the second hinge loss is taken as the hinge resistance loss.

[0125] The absolute difference between the high-resolution restored image and the original high-resolution image is calculated pixel by pixel to obtain the L1 pixel reconstruction loss.

[0126] The high-resolution restored image and the real high-resolution image are respectively input into the pre-trained VGG network. High-level feature maps corresponding to the high-resolution restored image and the real high-resolution image are extracted. The mean square error between the high-level feature maps is calculated to obtain the VGG perceptual loss.

[0127] The Gram matrix of the high-level feature map of the high-resolution restored image and the high-level feature map of the real high-resolution image are calculated respectively, and the mean square error of the Gram matrix is ​​calculated to obtain the Gram matrix texture loss.

[0128] The initial decoded feature map, local equivalent inverse diffusion tensor field, and reverse advection velocity field generated during the forward propagation of the physical information neural network decoder are obtained. Based on the local equivalent inverse diffusion tensor field and reverse advection velocity field, the mean square error of the partial differential equation residual tensor of the initial decoded feature map on the time step of the two-dimensional advection-diffusion equation is calculated, and the constraint loss of the partial differential equation is obtained.

[0129] The hinge adversarial loss, L1 pixel reconstruction loss, VGG perception loss, Gram matrix texture loss, and partial differential equation constraint loss are weighted and summed according to preset dynamic weight coefficients to obtain the multi-objective joint loss.

[0130] Based on the multi-objective joint loss function, a multi-stage progressive gradient update is performed on the dual-branch feature extraction network, the physical information neural network decoder, and the global-local dual discriminator, and the updated network parameters are output.

[0131] Repeatedly perform multi-stage progressive gradient updates until the absolute value of the change in the multi-objective joint loss function within consecutive preset rounds is less than a preset threshold; update the dual-branch feature extraction network and the physical information neural network decoder by updating the network parameters.

[0132] In one embodiment, the physics initialization module 202 can also be used for:

[0133] Eigenvalue decomposition is performed on the structure tensor matrix to extract the maximum eigenvalue, the minimum eigenvalue, the first eigenvector corresponding to the maximum eigenvalue, and the second eigenvector corresponding to the minimum eigenvalue.

[0134] Based on the difference between the maximum and minimum eigenvalues, the anisotropic intensity of the local edge is calculated, and an anisotropic gradient scalar field is generated.

[0135] The anisotropic gradient scalar field is mapped along the direction of the second eigenvector to obtain the initial diffusion direction field; the anisotropic gradient scalar field is mapped along the direction of the first eigenvector to obtain the initial advection direction field.

[0136] In one embodiment, the gated feature fusion module 204 can also be used for:

[0137] The paper texture feature map and the ink structure feature map are spliced ​​together along the channel dimension to obtain the spliced ​​feature map.

[0138] Global average pooling is performed on the spliced ​​feature map to obtain a global channel compression vector.

[0139] The global channel compression vector is input into the multilayer perceptron for nonlinear mapping to generate channel attention weight vectors. The channel attention weight vectors are then activated by the sigmoid function to generate an adaptive gating activation map.

[0140] Based on the adaptive gating activation map, the paper texture feature map and the ink structure feature map are summed by channel weights to obtain a multimodal fusion feature map.

[0141] In one embodiment, the network training optimization module can also be used for:

[0142] In the first stage of progressive training, the global-local dual discriminator is frozen, and the gradient of the dual-branch feature extraction network and the physical information neural network decoder is updated through L1 pixel reconstruction loss and VGG perceptual loss to obtain the first intermediate network parameters.

[0143] In the second stage of progressive training, the global-local dual discriminator is unfrozen, the first intermediate network parameters are loaded, and the dual-branch feature extraction network, physical information neural network decoder and global-local dual discriminator are alternately trained through L1 pixel reconstruction loss, VGG perceptual loss, hinge adversarial loss and Gram matrix texture loss to obtain the second intermediate network parameters.

[0144] In the third stage of progressive training, the second intermediate network parameters are loaded, and the dual-branch feature extraction network, physical information neural network decoder, and global-local dual discriminator are jointly fine-tuned through multi-objective joint loss, and the network parameters are updated by output.

[0145] In one embodiment, a computer device is provided, including a memory and a processor, the memory storing a computer program, the processor executing the computer program to implement a degraded archival image handwriting restoration method as described above.

[0146] In one embodiment, a computer-readable storage medium is provided having a computer program stored thereon, which, when executed by a processor, implements the steps in the above method embodiments.

[0147] For the device embodiments, since they basically correspond to the method embodiments, the relevant parts can be referred to in the description of the method embodiments. The device embodiments described above are merely illustrative. The components described as separate parts may or may not be physically separate, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this disclosure according to actual needs. Those skilled in the art can understand and implement this without creative effort.

[0148] The above-described embodiments are merely illustrative of several implementation methods of the embodiments of this application, and their descriptions are relatively specific and detailed. However, they should not be construed as limiting the scope of the patent application. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of the embodiments of this application, and these modifications and improvements all fall within the protection scope of the embodiments of this application.

Claims

1. A method for restoring handwriting in degraded archival images, characterized in that, The method includes: S1. Obtain the degraded archive image, perform size and pixel scaling preprocessing on the degraded archive image to obtain a preprocessed image; perform structure tensor calculation on the preprocessed image to obtain a structure tensor matrix; S2. Perform eigenvalue decomposition on the structure tensor matrix to obtain the eigenvector space orientation, and generate the initial diffusion direction field and the initial advection direction field based on the eigenvector space orientation. S3. Input the preprocessed image into a dual-branch feature extraction network, and perform parallel feature extraction through the texture preservation branch and the ink structure branch to obtain paper texture feature map and ink structure feature map; wherein, the texture preservation branch is constructed based on discrete wavelet transform, and the ink structure branch is constructed based on enhanced text edge attention mechanism and Swin-Transformer; S4. Based on the channel attention-guided fusion mechanism, adaptive gated fusion of the paper texture feature map and the ink structure feature map is performed through global pooling and multilayer perceptron to output a multimodal fusion feature map; S5. Input the multimodal fusion feature map into the physical information neural network decoder, and use the initial diffusion direction field and the initial advection direction field as prior guidance to perform inverse physical evolution decoding based on the two-dimensional advection-diffusion equation, and output a high-definition restored image.

2. The method according to claim 1, characterized in that, The training and optimization steps of the dual-branch feature extraction network and the physical information neural network decoder include: S61. Obtain a real high-definition image, input the high-definition restored image and the real high-definition image into the global-local dual discriminator respectively, perform feature extraction and downsampling on the high-definition restored image and the real high-definition image respectively through a convolutional layer, and output a first discrimination score corresponding to the high-definition restored image and a second discrimination score corresponding to the real high-definition image; S62. Calculate the first hinge loss based on the first discrimination score and the preset false label, and calculate the second hinge loss based on the second discrimination score and the preset true label; take the sum of the first hinge loss and the second hinge loss as the hinge resistance loss; S63. Calculate the absolute difference pixel by pixel between the high-definition restored image and the real high-definition image to obtain the L1 pixel reconstruction loss; S64. Input the high-definition restored image and the real high-definition image into the pre-trained VGG network respectively, extract the high-level feature map corresponding to the high-definition restored image and the high-level feature map corresponding to the real high-definition image, calculate the mean square error between the high-level feature maps, and obtain the VGG perceptual loss. S65. Calculate the Gram matrix of the high-level feature map of the high-definition restored image and the high-level feature map of the real high-definition image respectively, and calculate the mean square error of the Gram matrix to obtain the Gram matrix texture loss. S66. Obtain the initial decoding feature map, local equivalent inverse diffusion tensor field, and reverse advection velocity field generated during the forward propagation of the physical information neural network decoder; based on the local equivalent inverse diffusion tensor field and the reverse advection velocity field, calculate the mean square error of the partial differential equation residual tensor of the initial decoding feature map on the time step of the two-dimensional advection-diffusion equation, and obtain the partial differential equation constraint loss. S67. The hinge adversarial loss, the L1 pixel reconstruction loss, the VGG perception loss, the Gram matrix texture loss, and the partial differential equation constraint loss are weighted and summed according to preset dynamic weight coefficients to obtain the multi-objective joint loss. S68. Based on the multi-objective joint loss function, perform multi-stage progressive gradient updates on the dual-branch feature extraction network, the physical information neural network decoder, and the global-local dual discriminator, and output updated network parameters. S69. Repeat the multi-stage progressive gradient update until the absolute value of the change in the multi-objective joint loss function within a consecutive preset number of rounds is less than a preset threshold; update the dual-branch feature extraction network and the physical information neural network decoder by updating the network parameters.

3. The method according to claim 1, characterized in that, S2 includes: S21. Perform eigenvalue decomposition on the structure tensor matrix to extract the maximum eigenvalue, the minimum eigenvalue, the first eigenvector corresponding to the maximum eigenvalue, and the second eigenvector corresponding to the minimum eigenvalue; wherein, the direction of the first eigenvector represents the image gradient direction, and the direction of the second eigenvector represents the local texture tangency. S22. Calculate the local edge anisotropy intensity based on the difference between the maximum eigenvalue and the minimum eigenvalue, and generate anisotropic gradient scalar field; S23. Map the anisotropic gradient scalar field along the direction of the second eigenvector to obtain the initial diffusion direction field; map the anisotropic gradient scalar field along the direction of the first eigenvector to obtain the initial advection direction field; wherein, the calculation formula for the initial diffusion direction field is: in, Representing spatial coordinates The initial diffusion direction field at the location; The largest eigenvalue, The minimum eigenvalue; This represents the diffusion attenuation adjustment coefficient; This represents the eigenvector corresponding to the minimum eigenvalue; The formula for calculating the initial advection direction field is: in, Representing spatial coordinates The initial advection direction field at the location; This represents the eigenvector corresponding to the largest eigenvalue; This represents the image gradient operator of the preprocessed image; This represents the scaling factor for advection velocity.

4. The method according to claim 1, characterized in that, The S4 includes: S41. The paper texture feature map and the ink structure feature map are spliced ​​together by channel dimension to obtain a spliced ​​feature map; S42. Perform global average pooling on the spliced ​​feature map to obtain a global channel compression vector; S43. Input the global channel compression vector into a multilayer perceptron for nonlinear mapping to generate a channel attention weight vector, and activate the channel attention weight vector through the Sigmoid function to generate an adaptive gating activation map. S44. Based on the adaptive gating activation map, perform channel weighted summation on the paper texture feature map and the ink structure feature map to obtain a multimodal fusion feature map.

5. The method according to claim 2, characterized in that, S68 includes: S681. In the first stage of progressive training, the global-local dual discriminator is frozen, and the gradient of the dual-branch feature extraction network and the physical information neural network decoder is updated through the L1 pixel reconstruction loss and the VGG perception loss to obtain the first intermediate network parameters. S682. In the second stage of progressive training, the global-local dual discriminator is unfrozen, the first intermediate network parameters are loaded, and the dual-branch feature extraction network, the physical information neural network decoder and the global-local dual discriminator are alternately trained through the L1 pixel reconstruction loss, the VGG perception loss, the hinge adversarial loss and the Gram matrix texture loss to obtain the second intermediate network parameters. S683. In the third stage of progressive training, the second intermediate network parameters are loaded, and the dual-branch feature extraction network, the physical information neural network decoder, and the global-local dual discriminator are jointly fine-tuned through the multi-objective joint loss, and the updated network parameters are output.

6. A system for restoring handwriting in degraded archival images, used to implement the method according to any one of claims 1 to 5, characterized in that, The system includes: The image preprocessing module is used to acquire degraded archive images, perform size and pixel scaling preprocessing on the degraded archive images to obtain preprocessed images, and perform structure tensor calculation on the preprocessed images to obtain structure tensor matrices. The physics field initialization module is used to perform eigenvalue decomposition on the structure tensor matrix to obtain the eigenvector space orientation, and generate the initial diffusion direction field and the initial advection direction field according to the eigenvector space orientation. The parallel feature extraction module is used to input the preprocessed image into the dual-branch feature extraction network, and perform parallel feature extraction through the texture preservation branch and the ink structure branch to obtain the paper texture feature map and the ink structure feature map. The gated feature fusion module is used to perform adaptive gated fusion of the paper texture feature map and the ink structure feature map through global pooling and multilayer perceptron based on the channel attention-guided fusion mechanism, and output a multimodal fusion feature map. The physical evolution decoding module is used to input the multimodal fusion feature map into the physical information neural network decoder, and use the initial diffusion direction field and the initial advection direction field as prior guidance to perform inverse physical evolution decoding based on the two-dimensional advection-diffusion equation, and output a high-definition restored image.

7. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the computer program, it implements the method of any one of claims 1 to 5.

8. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the method of any one of claims 1 to 5.

Images