Traditional picture repairing method fusing low-resolution prior and efficient visual selection

Abstract

The invention belongs to the technical field of digital restoration of computer vision and cultural heritage, and particularly relates to a traditional picture restoration method fusing low-resolution prior and efficient visual selection, which comprises the following steps: constructing a multi-source image data set, taking images in the multi-source image data set as high-resolution images, preprocessing the high-resolution images to obtain low-resolution images, and carrying out high-resolution priori and high-efficiency visual selection on the low-resolution images. The high-resolution image and the low-resolution image are respectively masked to generate simulated damage mask images, and the simulated damage mask images comprise a regular damage mask image and an irregular damage mask image; taking the multi-source image data set and the preprocessed multi-source image data set as training data, and training a multi-source image model; the dual-stage repair network comprises a coarse repair network and a fine repair network; according to the method, the problems of structural semantic loss, high priori information dependency and insufficient global and local coordination when an existing image restoration method is used for processing a complex scene and a large-range missing region are solved.

Description

Technical Field

[0001] This invention belongs to the field of computer vision and digital restoration technology of cultural heritage, specifically a traditional painting restoration method that integrates low-resolution priors and efficient visual selection. Background Technology

[0002] Ancient Chinese painting is a cultural treasure of the Chinese nation. As a concentrated embodiment of Chinese culture, aesthetics and philosophical concepts, it occupies an important position in China's intangible cultural heritage. Restoring ancient paintings is a key means of protecting and inheriting cultural relics.

[0003] However, due to their age, ancient paintings generally suffer from varying degrees of damage, such as damage to calligraphy and paintings, missing pieces, alteration of content, and damage to murals. A shortage of restoration talent makes manual restoration insufficient to meet the demand. While deep learning-based image restoration technology can effectively improve image quality and restore lost details when processing traditional paintings, many problems remain when applying these technologies to ancient Chinese paintings, which possess complex semantic information and unique artistic expressions.

[0004] First, ancient painting datasets are scarce and difficult to annotate, resulting in poor model adaptability.

[0005] Second, traditional techniques are unique, and conventional techniques cannot simulate fine textures and brushstrokes.

[0006] Third, the color matching and composition layout are highly abstract, which traditional algorithms cannot effectively express.

[0007] Fourth, the cultural connotations are profound, and existing technologies struggle to capture the deep semantic connections, easily losing the original artistic conception.

[0008] Therefore, it is necessary to design a traditional painting restoration method to improve the above-mentioned problems. Summary of the Invention

[0009] To address the problems of existing technologies, this invention provides a traditional image restoration method that integrates low-resolution priors and efficient visual selection, comprising the following steps:

[0010] Step 1: Construct a multi-source image dataset. The images in the multi-source image dataset are high-resolution images, including traditional painting image sets, historical artifact image sets, and natural landscape image sets. Preprocess the high-resolution images to obtain low-resolution images, and then perform masking on the low-resolution images to generate simulated damage mask images. The simulated damage mask images include regular damage mask images and irregular damage mask images.

[0011] Step 2: Use the multi-source image dataset and the preprocessed multi-source image dataset as training data to train the multi-source image restoration model;

[0012] The multi-source image restoration model includes a two-stage restoration network, which consists of a coarse restoration network and a fine restoration network. The coarse restoration network is based on an autoregressive Transformer network, which captures global context information through the autoregressive Transformer module, dynamically updates mask weights, and generates preliminary restoration results.

[0013] The fine restoration network includes a dynamic gated fusion module and a Mamba-driven efficient visual feature selection module. It achieves adaptive fusion of structural and texture features on the initial restoration results through a dual-branch structure and selective scanning mechanism, thereby improving detail and style consistency.

[0014] Step 3: During the training of the multi-source image restoration model, the parameters of the multi-source image restoration model are adjusted using a joint loss function until the model converges, resulting in a traditional image restoration model.

[0015] Furthermore, the high-resolution image is preprocessed to obtain a low-resolution image, and a mask is generated from the low-resolution and high-resolution images to produce a simulated damage mask image. Specifically, this includes:

[0016] The high-resolution image is preprocessed by Gaussian filtering for noise reduction and histogram equalization, and then uniformly adjusted to a resolution of 256×256. A dual-channel mask generation mechanism is used to generate regular damage mask images and irregular damage mask images, which are then saved in binary image format.

[0017] Furthermore, in step 2, the coarse repair network is based on an autoregressive Transformer network. It captures global context information through the autoregressive Transformer module, dynamically updates the mask weights, and generates preliminary repair results, specifically including:

[0018] S201: The coarse-grained repair network is based on an autoregressive Transformer network, and the input is a low-resolution damaged image. And the corresponding low-resolution damage mask image M. Low-resolution damage image From high-resolution image I H The low-resolution image I obtained after downsampling L By combining the low-resolution damage mask image M with the composite image, we obtain:

[0019] I L =↓(I H ),

[0020] Where C(·) represents fusion;

[0021] The coarse repair network F r Low-resolution damaged images in low-resolution space The model is constructed based on the corresponding low-resolution damage mask image M, and the preliminary repair result R is output. c The calculation formula is as follows:

[0022]

[0023] in, This represents the set of valid pixel coordinates, and δ(i,j) is an indicator function that indicates whether the pixels within the selected sliding window are valid.

[0024] The Transformer network computes the context representation through an attention mechanism, and the output representation for each position i is O. i Obtained by weighted aggregation of local attention values:

[0025]

[0026] in, These are query, key, and value vectors, respectively. Let M' represent the sliding window neighborhood centered at position i. ij This serves as an auxiliary mask matrix to limit the interference of invalid pixels on the attention score. This indicates the calculation of the attention score, α. ij This represents the attention weight.

[0027] S202: During the layer-by-layer forward propagation of the Transformer network, the mask information is dynamically updated based on the repair status; if the current sliding window... If at least one valid pixel exists, the mask corresponding to position i is updated to be valid; otherwise, it remains invalid. This mechanism allows valid context information to be passed layer by layer to the missing region, significantly improving the continuity and accuracy of structure restoration.

[0028] The mask update strategy is as follows: during the training phase, the auxiliary mask M′ is continuously updated; when any pixel in the window is a valid pixel, the remaining areas are considered to be able to participate in the attention mechanism calculation, thereby dynamically expanding the available context area and enhancing the model generation quality.

[0029] Furthermore, the coarse repair network also includes the following during training:

[0030] S203: Training objective function, used to enhance the structural rationality and semantic consistency of model generation, specifically including:

[0031] Autoregressive loss:

[0032]

[0033] in, For an autoregressive Transformer network, φ represents all the learnable parameters of the network, and Y... <i This refers to all known pixels or features preceding position i in the sequence, y i This is the pixel value at the current position i that the model needs to predict based on the information above.

[0034] Mask modeling loss:

[0035]

[0036] in, This represents the set of pixel locations in the input image that are randomly obscured. This refers to all visible pixels except for the obscured areas. The covered position S j The original true pixel values ​​on the surface.

[0037] Joint Autoregressive-Mask Training Objective:

[0038]

[0039] in, It represents the set of all known pixels or features that are not masked. This indicates that the object is within the covered area, but at the current location S. j Previously, the content had already been generated by the model in an autoregressive manner.

[0040] Intermediate output reconstruction loss is used to constrain the consistency between the low-resolution restored image and the high-resolution image:

[0041]

[0042] Among them, R c (·) indicates mapping intermediate features to F c A convolutional decoder that converts images to RGB.

[0043] S204: Parameter optimization strategy, employing the Adam optimization algorithm to backpropagate and iteratively update the parameters of the coarse-repair network, including the loss function gradient and joint objective. The optimized parameters of the model are calculated together to improve convergence efficiency and prevent overfitting.

[0044] Combined with learning rate scheduling, the initial learning rate is set to η = 1 × 10 -4 Whenever the performance on the validation set stops improving, the learning rate is halved; a gradient pruning mechanism is introduced to limit the maximum gradient norm and avoid model oscillation; Dropout and layer normalization are used to improve the network's generalization ability.

[0045] Furthermore, in step 2, the fine-tuning network includes a dynamically gated fusion module and a Mamba-driven efficient visual feature selection module. It adaptively fuses structural and texture features based on the initial repair results through a dual-branch structure and a selective scanning mechanism. Specifically, this includes:

[0046] S301: Feature encoding and multi-scale fusion, for the low-resolution restoration result R obtained in step 2. c The image is restored to a high-resolution scale using an upsampling module, maintaining consistency with the high-resolution damage mask image. The same spatial dimensions; then, the upsampled low-resolution repair result R c The input is fed into a structural feature encoder to extract its corresponding coarse structural feature map F. s Meanwhile, high-resolution damage mask images The texture feature map F is obtained after processing by another encoder. t Both encoders employ a combination of multi-layer convolution, normalization, and activation, exhibiting strong texture detail preservation capabilities; for coarse-structured feature maps F... s With texture feature map F t Residual connections and multi-scale convolution operations are introduced to integrate semantic information from different receptive fields, thereby enhancing its ability to express local and global features.

[0047] S302: Dynamic Gated Fusion Module, which generates channel attention gating weights through bidirectional perception gating paths, dynamically adjusting the weight ratio of structural and texture features; specifically including:

[0048] Let F be the coarse-structure feature. s ∈R C×H×W Texture feature F t ∈R C×H×W First, the two are concatenated along the channel dimension to form a fused feature tensor X:

[0049] X = [F] s ,F t ]∈R 2C×H×W ;

[0050] To achieve bidirectional dynamic modulation between structural and texture features, perceptual gating paths are constructed for each of the two branches. For any branch i∈{s,t}, and the remaining branches j, the perceptual process is as follows:

[0051]

[0052] Among them, W i ∈R C×2C×3×3 For convolution kernel, For the normalization function, φ i (·) is a non-linear activation function;

[0053] Then, channel attention gating weights G are generated using the Sigmoid function. i :

[0054] G i =σ(Z) i );

[0055] The features are modulated and fused based on this gating tensor:

[0056]

[0057] Where, α i β i These are trainable weight coefficients, and ⊙ represents element-wise multiplication;

[0058] Finally, the two updated features are concatenated to obtain the fused feature F. u :

[0059]

[0060] This module implements a dynamic modulation mechanism from structural completion to texture refinement, enabling the network to automatically adjust feature weights based on the local context of the image, thereby improving the coordination between structure and texture and the quality of restoration.

[0061] S303: A high-efficiency visual feature selection module driven by Mamba, used to enhance the modeling ability of fine-grained repair networks for spatial and temporal information, and to achieve selective feature enhancement;

[0062] Furthermore, in step S303, the MambaEV module is introduced into the Mamba-driven efficient visual feature selection module. This module is based on state-space modeling technology, which fuses features through branching in the temporal and spatial dimensions, and utilizes a learnable selective scanning mechanism and a nonlinear fusion strategy, specifically including:

[0063] Input fusion feature F u After entering the MambaEV module, two branches are loaded:

[0064] The first branch, time-dimensional modeling, first performs RMSnorm and linear transformation, then enters the SSD selective scan module and applies SiLU activation to obtain time features:

[0065]

[0066] Where x is the input feature, w is the learnable parameter of RMSNorm, ∈ is the stability factor, and SSD realizes linear state modeling of sequence information;

[0067] The second branch involves spatial information fusion. The features first pass through a linear layer, then through Sigmoid activation, to form auxiliary channel features.

[0068] F b =W2·Sigmoid(F in )+b2;

[0069] Then, the outputs of the two branches are merged element-wise and linearly mapped:

[0070]

[0071] in, ⊕ represents element-wise multiplication, ⊕ represents addition fusion, and L(·) is a linear mapping layer. This module uses a "state-space-sequence" modeling approach to automatically filter highly correlated local features while preserving global consistency, thereby improving the continuity and naturalness of the repair details.

[0072] Furthermore, in step 3, a joint loss function is used to adjust the parameters of the multi-source image model during training until the multi-source image model converges. The joint loss function specifically includes:

[0073] Pixel reconstruction loss

[0074]

[0075] Perceived loss

[0076]

[0077] Style loss

[0078]

[0079] in It is a Gram matrix.

[0080] Combating losses

[0081]

[0082] Where G is the generator network and D is the discriminator network.

[0083] intermediate feature supervision loss

[0084]

[0085] The joint loss function is in the form of a weighted linear combination:

[0086]

[0087] The weights for each loss are:

[0088] λ rec =20,λ perc =0.8,λ style =20, λ adv =0.5, λ m =1;

[0089] During training, the Adam optimizer is used, along with a learning rate scheduling and weight decay mechanism to iteratively update parameters until the multi-source image model converges. After the validation set stabilizes, the learning rate is adjusted to ensure that the model reaches the global optimum in terms of structure preservation and detail recovery.

[0090] The beneficial effects of this invention are:

[0091] This invention addresses the problems of structural semantic loss, strong dependence on prior information, and insufficient global-local coordination in existing image inpainting methods when dealing with complex scenes and large missing areas by using a two-stage inpainting network, including a coarse inpainting network and a fine inpainting network, combined with low-resolution images as global structural prior information. Specifically, the low-resolution prior eliminates the need for manual annotation, improving inpainting efficiency; dynamic masking simulates real damage; and bidirectional perceptual gating dynamically fuses structural and texture features to enhance coordination. The MambaEV module processes long sequence data through a state-space model, improving detail and style consistency. A joint loss function ensures a balance between pixel similarity, structure preservation, style consistency, and visual realism. Attached Figure Description

[0092] Figure 1 This is a schematic diagram of the traditional image restoration method of the present invention;

[0093] Figure 2 This is a general framework diagram of the repair model used in this invention;

[0094] Figure 3 This is a schematic diagram of the attention mechanism of the Transformer used in this invention;

[0095] Figure 4 This is a comparison chart of the restoration model used in this invention with other image restoration methods on the MaskCLP dataset;

[0096] Figure 5 This is a comparison chart of the restoration model used in this invention with other image restoration methods on multiple datasets;

[0097] Figure 6 This is a comparison chart of ablation experiment results for masks with different proportions according to the present invention. Detailed Implementation

[0098] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0099] Please see Figure 1-6 This invention provides a traditional image restoration method that integrates low-resolution priors and efficient visual selection, comprising the following steps:

[0100] Step 1: Construct a multi-source image dataset. The images in the multi-source image dataset are high-resolution images, including MaskCLP (8273 traditional landscape paintings), MuralVerse (6530 Dunhuang murals, etc.), and Places (8000 natural landscapes). Preprocess the high-resolution images to obtain low-resolution images, and then perform masking on the low-resolution images to generate simulated damage mask images. The simulated damage mask images include regular damage mask images and irregular damage mask images.

[0101] Step 2: Use the multi-source image dataset and the preprocessed multi-source image dataset as training data to train the image restoration model;

[0102] The image restoration model includes a two-stage restoration network, which consists of a coarse restoration network and a fine restoration network. The coarse restoration network is based on an Autoregressive Transformer network, which uses the overall structure of the low-resolution image to capture global contextual information, dynamically updates the mask weights, and generates preliminary restoration results.

[0103] The fine restoration network includes a dynamic gated fusion module and a state-space model (Mamba) driven efficient visual feature selection module. It achieves adaptive fusion of structural and texture features through a dual-branch structure and selective scanning mechanism on the preliminary restoration results, improving detail and style consistency. The low-resolution image and the corresponding mask low-resolution image are input into the coarse restoration network, which generates preliminary structural filling results. These results are then input into the fine restoration network along with the high-resolution image to output the final restored image, without the need for manual intervention.

[0104] Step 3: During the training of the multi-source image model, the parameters of the multi-source image model are adjusted using a joint loss function until the multi-source image model converges, resulting in a traditional image restoration model.

[0105] Furthermore, the high-resolution image is preprocessed to obtain a low-resolution image, and the low-resolution and high-resolution images are masked to generate a simulated damage mask image. Specifically, this includes:

[0106] The high-resolution image is preprocessed by Gaussian filtering for noise reduction and histogram equalization, and then uniformly adjusted to a resolution of 256×256. A dual-channel mask generation mechanism is used to generate regular damage mask images and irregular damage mask images, which are then saved in binary image format.

[0107] Specifically, to eliminate noise interference caused by differences between the image acquisition equipment and the original environment, all input data are first smoothed and denoised using a 3×3 core Gaussian filter, removing high-frequency noise while preserving brushstroke texture features. Subsequently, adaptive histogram equalization is used to normalize the image brightness distribution. By adjusting the dynamic range of image contrast, statistical consistency in brightness levels is maintained between images from different sources. This preprocessing process not only improves the overall stability of the dataset distribution but also enhances the learning efficiency of the restoration network for cross-domain features. MaskCLP contains 8,273 traditional painting images, covering landscapes, flowers and birds, figures, murals, and other types. MuralVerse is generated using data augmentation techniques and contains 6,530 training samples of Dunhuang murals and historical artifacts. 6,530 training samples are synthesized using data augmentation techniques such as color perturbation, style transfer, and random occlusion to expand data diversity. Places, as a general scene dataset, contains 8,000 natural landscape training samples for cross-domain generalization validation. All images were preprocessed and uniformly adjusted to a resolution of 256×256. Gaussian filtering and histogram equalization were used in the preprocessing stage to improve image quality consistency. Simultaneously, a corresponding mask image was generated for each image, simulating regular (rectangular, circular) and irregular (insect-eaten, mold) damage patterns. Missing regions were marked as 0, and valid regions as 1, forming a binary mask matrix M∈{0,1}H×W. The low-resolution images, high-resolution images, and their corresponding damage mask images were divided into a training set (7446 images), a validation set (827 images), and a test set (827 images) in an 8:1:1 ratio.

[0108] Furthermore, in step 2, such as Figure 2 As shown, the coarse repair network is based on an autoregressive Transformer network, which uses the overall structure of the low-resolution image to capture global contextual information, dynamically updates the mask weights, and generates preliminary repair results.

[0109] Specifically, it includes:

[0110] S201: The coarse-grained repair network is based on an autoregressive Transformer network, and the input is a low-resolution damaged image. And the corresponding low-resolution damage mask image M. Low-resolution damage image From high-resolution image I H The low-resolution image I obtained after downsampling L By combining the low-resolution damage mask image M with the composite image, we obtain:

[0111] I L =↓(I H ),

[0112] Where C(·) represents fusion;

[0113] The coarse repair network F r Low-resolution damaged images in low-resolution space The model is constructed based on the corresponding low-resolution damage mask image M, and the preliminary repair result R is output. c The calculation formula is as follows:

[0114]

[0115] in, This represents the set of valid pixel coordinates, and δ(i,j) is an indicator function that indicates whether the pixels within the selected sliding window are valid.

[0116] like Figure 3 As shown, the Transformer network computes the context representation through an attention mechanism, and the output representation for each position i is O. i Obtained by weighted aggregation of local attention values:

[0117]

[0118] in, These are query, key, and value vectors, respectively. Let M' represent the sliding window neighborhood centered at position i. ij This serves as an auxiliary mask matrix to limit the interference of invalid pixels on the attention score. This indicates the calculation of the attention score, α. ij This represents the attention weight.

[0119] S202: As Figure 2 As shown in (a), during the layer-by-layer forward propagation of the Transformer network, the mask information is dynamically updated according to the repair status; if the current sliding window... If at least one valid pixel exists, the mask corresponding to position i is updated to be valid; otherwise, it remains invalid. This mechanism allows valid context information to be passed layer by layer to the missing region, significantly improving the continuity and accuracy of structure restoration.

[0120] The mask update strategy is as follows: during the training phase, the auxiliary mask M′ is continuously updated; when any pixel in the window is a valid pixel, the remaining areas are considered to be able to participate in the attention mechanism calculation, thereby dynamically expanding the available context area and enhancing the model generation quality.

[0121] Furthermore, the coarse repair network also includes the following during training:

[0122] S203: Training objective function, used to enhance the structural rationality and semantic consistency of model generation, specifically including:

[0123] Autoregressive Loss:

[0124]

[0125] in, For an autoregressive Transformer network, φ represents all the learnable parameters of the network, and Y... <i This refers to all known pixels or features preceding position i in the sequence, y i This is the pixel value at the current position i that the model needs to predict based on the information above.

[0126] Masked Language Modeling Loss:

[0127]

[0128] in, This represents the set of pixel locations in the input image that are randomly obscured. This refers to all visible pixels except for the obscured areas. The covered position S j The original true pixel values ​​on the surface.

[0129] Joint Autoregressive-Mask Training Objective:

[0130]

[0131] in, It represents the set of all known pixels or features that are not masked. This indicates that the object is within the covered area, but at the current location S. j Previously, the content had already been generated by the model in an autoregressive manner.

[0132] Intermediate output reconstruction loss is used to constrain the consistency between the low-resolution restored image and the high-resolution image:

[0133]

[0134] Among them, R c (·) indicates mapping intermediate features to F c A convolutional decoder that converts images to RGB.

[0135] S204: Parameter optimization strategy. The Adaptive Moment Estimation (Adam) optimization algorithm is used to backpropagate and iteratively update the parameters of the coarse-repaired network, including the loss function gradient and joint objective. The optimized parameters of the model are calculated together to improve convergence efficiency and prevent overfitting.

[0136] Combined with learning rate scheduling, the initial learning rate is set to η = 1 × 10 -4 Whenever the performance on the validation set stops improving, the learning rate is halved; a gradient pruning mechanism is introduced to limit the maximum gradient norm and avoid model oscillations; random deactivation (Dropout) and layer normalization (LayerNorm) are used to improve the network's generalization ability.

[0137] Furthermore, in step 2, the fine-tuning network includes a dynamically gated fusion module and a Mamba-driven efficient visual feature selection module. It adaptively fuses structural and texture features based on the initial repair results through a dual-branch structure and a selective scanning mechanism. Specifically, this includes:

[0138] S301: Feature encoding and multi-scale fusion: After upsampling the coarse restoration result, structural and texture features are extracted, and residual connections and multi-scale convolution operations are introduced; the coarse restoration result and texture details in the high-resolution image are effectively encoded and fused to improve the precision and structural consistency of image restoration. For example... Figure 2 As shown in (b), the specific process is as follows: First, the low-resolution repair result R obtained in step 2 is processed... c The image is restored to a high-resolution scale using an upsampling module, maintaining consistency with the high-resolution damage mask image. Same spatial dimensions; then, the upsampled R c The input is fed into a structural feature encoder to extract its corresponding coarse structural feature map F. s Meanwhile, high-resolution damage mask images The texture feature map F is obtained after processing by another encoder. t Both encoders employ a combination of multi-layer convolution, normalization, and activation, exhibiting strong texture detail preservation capabilities; for coarse-structured feature maps F... s With texture feature map F t Residual connections and multi-scale convolution operations are introduced to integrate semantic information from different receptive fields, enhancing its ability to express local and global features. Through this stage of processing, the coarse-structured feature F... sWith texture feature F t They are extracted separately to provide basic feature representations for subsequent fusion and selection.

[0139] S302: Dynamic gating fusion module, such as Figure 2 As shown in (c), channel attention gating weights are generated through bidirectional perceptual gating paths to dynamically adjust the weight ratio of structural and texture features; specifically including:

[0140] Let F be the coarse-structure feature. s ∈R C×H×W Texture feature F t ∈R C×H×W First, the two are concatenated along the channel dimension to form a fused feature tensor X:

[0141] X = [F] s ,F t ]∈R 2C×H×W ;

[0142] To achieve bidirectional dynamic modulation between structural and texture features, perceptual gating paths are constructed for each of the two branches. For any branch i∈{s,t}, and the remaining branches j, the perceptual process is as follows:

[0143]

[0144] Among them, W i ∈R C×2C×3×3 For convolution kernel, For the normalization function, φ i (·) is a non-linear activation function;

[0145] Then, the channel attention gating weights G are generated using a sigmoid function. i :

[0146] G i =σ(Z) i );

[0147] The features are modulated and fused based on this gating tensor:

[0148]

[0149] Where, α i β i These are trainable weight coefficients, and ⊙ represents element-wise multiplication;

[0150] Finally, the two updated features are concatenated to obtain the fused feature F. u :

[0151]

[0152] This module implements a dynamic modulation mechanism from structural completion to texture refinement, enabling the network to automatically adjust feature weights based on the local context of the image, thereby improving the coordination between structure and texture and the quality of restoration.

[0153] S303: A high-efficiency visual feature selection module driven by Mamba, such as Figure 2 As shown in (d), to enhance the modeling ability of the fine-grained repair network for spatial and temporal information, the MambaEV module is introduced. This module is based on state-space modeling technology, which fuses features through branches in the temporal and spatial dimensions. It utilizes a learnable selective scanning mechanism and a nonlinear fusion strategy to achieve selective feature enhancement. The specific process is as follows:

[0154] Input fusion feature F u After entering the MambaEV module, two branches are loaded:

[0155] The first branch, time-dimensional modeling, first performs RMSnorm and linear transformation, then enters the Structured State Space Duality (SSD) module, and applies Sigmoid-weighted Linear Unit (SiLU) activation to obtain the time features:

[0156]

[0157] Where x is the input feature, w is the learnable parameter of Root Mean Square Layer Normalization (RMSNorm), ∈ is the stabilization factor, and SSD realizes linear state modeling of sequence information;

[0158] The second branch involves spatial information fusion. The features first pass through a linear layer, then through Sigmoid activation, to form auxiliary channel features.

[0159] F b =W2·Sigmoid(F in )+b2;

[0160] Then, the outputs of the two branches are merged element-wise and linearly mapped:

[0161]

[0162] in, This represents element-wise multiplication, ⊕ represents addition fusion, and L(·) is a linear mapping layer. This module uses...

[0163] The joint loss function is in the form of a weighted linear combination:

[0164]

[0165] The weights for each loss are:

[0166] λ rec =20,λ perc =0.8,λ style =20, λ adv =0.5, λ m =1;

[0167] During training, the Adam optimizer is used, along with a learning rate scheduling and weight decay mechanism to iteratively update parameters until the multi-source image model converges. After the validation set stabilizes, the learning rate is adjusted to ensure that the model reaches the global optimum in terms of structure preservation and detail recovery.

[0168] Step 4: Input the images from the test set into the trained multi-source image model and output the repaired image.

[0169] Specifically, after the model is trained, inputting a damaged image and its corresponding mask image allows the inference process to be executed. First, a coarse-grained repair network generates a structure-filled image, which is then compared with the original image. Figure 1 The input is processed by a fine-grained restoration network, which ultimately outputs visually realistic and stylistically consistent restored results. The entire inference process requires no manual intervention and has a high degree of automation and generalization capability.

[0170] Experimental verification and effect comparison:

[0171] To fully verify the effectiveness and advancement of the method presented in this invention, we conducted systematic experiments on multiple datasets and compared this method with current mainstream image inpainting models, including Transformer series, Generative Adversarial Networks (GAN) series, and diffusion-based methods. Evaluation metrics included Peak Signal-to-Noise Ratio (PSNR), Structural Similarity Index Measure (SSIM), and Learned Perceptual Image Patch Similarity (LPIPS).

[0172] The "state-space-sequence" modeling approach, while preserving global consistency, automatically filters highly relevant local features, thereby improving the continuity and naturalness of the repair details.

[0173] Furthermore, in step 3, a joint loss function is used to adjust the parameters of the multi-source image model during training until the multi-source image model converges. The joint loss function specifically includes:

[0174] Pixel reconstruction loss

[0175]

[0176] Perceived loss

[0177]

[0178] Style loss

[0179]

[0180] Combating losses

[0181]

[0182] Where G is the generator network and D is the discriminator network.

[0183] intermediate feature supervision loss

[0184]

[0185] Table 1. Evaluation of the repair effect of this invention compared with other existing repair models (↑ higher is better, ↓ lower is better).

[0186]

[0187] Experimental results show that under mild impairment (0–15%), this method can maintain good image detail and structure restoration capabilities, and can still stably output high-quality images under moderate occlusion (15–30%). Under high occlusion (30–45%), this method still outperforms all comparative methods in all three metrics. SSIM improves by 52.5% and reduces LPIPS by 60.9% compared to the Conditional Texture and Structure Dual Generation (CTSDG) model, verifying its strong robustness to large missing regions.

[0188] like Figures 4-5The results show that, on typical test images, Mamba-LPGNet can accurately restore the contours of mountains, brushstrokes of figures, and the natural transition of ink distribution. Compared with models such as Stochastic Differential Equation (SDE) and Prompting for All-in-One Blind Image Restoration (PromptIR), the images output by this method have more natural edges and more coherent textures, especially showing a significant advantage in details such as tree branches and water ripples. Subjective visual perception is more in line with the aesthetic characteristics of traditional paintings.

[0189] To further demonstrate the effectiveness of the method of this invention, multiple ablation experiments were designed and implemented to verify the specific contributions of each key module to the final repair performance and to clarify the role mechanism of each component in the overall architecture. All ablation experiments were conducted on a unified training configuration and dataset (MaskCLP). Specifically, the ablation experiments included the following variant settings: (1) removing the Mamba-driven visual feature selection module (denoted as w / oMambaEV) to verify the role of state space modeling in global style consistency and key region recognition; (2) removing the Dynamic Gated Fusion Module (DGF) (denoted as w / oDGF) to observe the changes in repair quality after the lack of adaptive fusion of structural and texture features; (3) removing the coarse repair network and retaining only the fine branch (denoted as w / oCoarse) to evaluate the effect of low-resolution prior guidance on global structural support in the repair task; (4) training using only L1 reconstruction loss (denoted as w / oJointLoss) to analyze the contribution of the joint loss function in ensuring semantic consistency and style coordination.

[0190] Table 2 shows the ablation experiment results on the landscape painting dataset (↑ higher is better, ↓ lower is better).

[0191]

[0192] Experimental results show that after removing the MambaEV module, the model is prone to texture drift and style abrupt changes in areas with large occlusions, resulting in a decrease in PSNR of approximately 1.47 dB and an increase in LPIPS of approximately 0.021. Figure 6 As shown, after removing DGF, the model's ability to control detailed textures is significantly reduced, especially in high-frequency regions where there are fusion breakage issues; without introducing a coarse repair network, the overall contour structure reconstruction ability is significantly reduced, and in subjective evaluation, it is considered to have "obvious repair traces"; the model trained using L1 loss alone exhibits problems such as blurry images and inconsistent styles, and the overall indicators all decline.

[0193] The ablation experiments described above fully demonstrate the rationality and necessity of the design of each core module in the method of this invention, and their synergistic effect is of key value in achieving high-quality image restoration results. In particular, the linkage mechanism between the MambaEV and DGF modules not only improves the model's ability to perceive image semantics, but also enhances the consistency of detail and structure fusion.

[0194] Based on the above experimental verification and comparison with mainstream methods on multiple datasets, the results show that this method significantly outperforms existing technologies in PSNR, SSIM, and LPIPS metrics, especially demonstrating excellent performance under high occlusion ratios. Ablation experiments validate the necessity of each module (such as MambaEV and DGF). Applications and advantages: This method eliminates the need for manual edge map annotation, has a high degree of automation, is suitable for traditional painting restoration tasks, effectively restores details and overall artistic style, and possesses high cultural preservation value.

[0195] It is worth noting that by using low-resolution images as structural priors, the overall layout of traditional paintings can be preserved without the need for manual annotation of edge maps, which solves the bottleneck of traditional methods that rely on complex annotation data and significantly improves the restoration efficiency. During the restoration process, the effect is further improved by continuously and dynamically updating the mask to simulate different damage conditions.

[0196] A bidirectional perceptual gating network is proposed to dynamically allocate the weight ratio of coarse structural features and fine-grained texture information, thereby achieving a synergistic enhancement of structural consistency and detail richness.

[0197] The efficient feature selection module driven by Mamba enables effective processing of long sequence data by introducing a state-space model, further improving the detail and style consistency of the restored images.

[0198] The joint reconstruction loss, perceptual loss, style loss and adversarial loss are optimized end-to-end to ensure that the restoration results achieve a balance in four aspects: pixel-level similarity, global structure preservation, style consistency and visual realism.

[0199] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A traditional image restoration method that integrates low-resolution priors and efficient visual selection, characterized in that, Includes the following steps: Step 1: Construct a multi-source image dataset, including traditional painting image set, historical artifact image set, and natural landscape image set; the images in the multi-source image dataset are high-resolution images. The high-resolution images are preprocessed to obtain low-resolution images, and the low-resolution images are masked to generate simulated damage mask images. The simulated damage mask images include regular damage mask images and irregular damage mask images. Step 2: Use the multi-source image dataset and the preprocessed multi-source image dataset as training data to train the multi-source image model; The multi-source image model includes a two-stage inpainting network, which includes a coarse inpainting network and a fine inpainting network. The coarse inpainting network is based on an autoregressive Transformer network. It captures global context information through the autoregressive Transformer module, dynamically updates mask weights, and generates preliminary inpainting results. The fine restoration network includes a dynamic gated fusion module and a Mamba-driven efficient visual feature selection module. It achieves adaptive fusion of structural and texture features on the initial restoration results through a dual-branch structure and selective scanning mechanism, thereby improving detail and style consistency. Step 3: During the training of the multi-source image restoration model, the parameters of the multi-source image restoration model are adjusted using a joint loss function until the multi-source image restoration model converges, resulting in a traditional image restoration model.

2. The traditional image restoration method according to claim 1, which integrates low-resolution prior and efficient visual selection, is characterized in that... The high-resolution image is preprocessed to obtain a low-resolution image. Then, the low-resolution and high-resolution images are masked to generate a simulated damage mask image. Specifically, this includes: The high-resolution image is preprocessed by Gaussian filtering for noise reduction and histogram equalization, and then uniformly adjusted to a resolution of 256×256. A dual-channel mask generation mechanism is used to generate regular damage mask images and irregular damage mask images, which are then saved in binary image format.

3. The traditional image restoration method according to claim 1, which integrates low-resolution prior and efficient visual selection, is characterized in that... In step 2, the coarse repair network is based on an autoregressive Transformer network. It captures global context information through the autoregressive Transformer module, dynamically updates the mask weights, and generates preliminary repair results, specifically including: S201: The coarse-grained repair network is based on an autoregressive Transformer network, and the input is a low-resolution damaged image. and the corresponding low-resolution damage mask image M; low-resolution damage image From high-resolution image I H The low-resolution image I obtained after downsampling L By combining the low-resolution damage mask image M with the composite image, we obtain: I L =↓(I H ), Where C(·) represents fusion; The coarse repair network F r Low-resolution damaged images in low-resolution space The model is constructed based on the corresponding low-resolution damage mask image M, and the preliminary repair result R is output. c The calculation formula is as follows: in, This represents the set of valid pixel coordinates, and δ(i,j) is an indicator function that indicates whether the pixels within the selected sliding window are valid. The Transformer network computes the context representation through an attention mechanism, and the output representation for each position i is O. i Obtained by weighted aggregation of local attention values: in, These are query, key, and value vectors, respectively. M represents the neighborhood of the sliding window centered at position i. ′ ij This serves as an auxiliary mask matrix to limit the interference of invalid pixels on the attention score. This indicates the calculation of the attention score, α. ij Indicates attention weight; S202: During the layer-by-layer forward propagation of the Transformer network, the mask information is dynamically updated based on the repair status; if the current sliding window... If at least one valid pixel exists, the mask corresponding to position i is updated to be valid; otherwise, it remains invalid. This mechanism allows valid context information to be passed layer by layer to the missing region, significantly improving the continuity and accuracy of structure restoration. The mask update strategy is as follows: during the training phase, the auxiliary mask M is updated. ′ Continuously updated; when any pixel within the window is a valid pixel, the remaining areas are considered to be eligible for attention mechanism calculation, thereby dynamically expanding the available context area and enhancing the quality of model generation.

4. The traditional image restoration method according to claim 3, which integrates low-resolution prior and efficient visual selection, is characterized in that... The coarse-grained repair network also includes the following during training: S203: Training objective function, used to enhance the structural rationality and semantic consistency of model generation, specifically including: Autoregressive loss: in, For an autoregressive Transformer network, φ represents all the learnable parameters of the network, and Y... <i This refers to all known pixels or features preceding position i in the sequence, y i This is the pixel value at the current position i that the model needs to predict based on the information above; Mask modeling loss: in, This represents the set of pixel locations in the input image that are randomly obscured. This refers to all visible pixels except for the obscured area; The covered position S j The original true pixel values ​​on; Joint Autoregressive-Mask Training Objective: in, Represents the set of all known pixels or features that are not occluded; This indicates that the object is within the covered area, but at the current location S. j Previously, the content had already been generated by the model using an autoregressive approach; Intermediate output reconstruction loss is used to constrain the consistency between the low-resolution restored image and the high-resolution image: Among them, R c (·) indicates mapping intermediate features to F c A convolutional decoder that converts images to RGB. S204: Parameter optimization strategy, employing the Adam optimization algorithm to backpropagate and iteratively update the parameters of the coarse-repair network, including the loss function gradient and joint objective. The optimized parameters of the model are calculated together to improve convergence efficiency and prevent overfitting. Combined with learning rate scheduling, the initial learning rate is set to η = 1 × 10 -4 Whenever the performance on the validation set stops improving, the learning rate is halved; a gradient pruning mechanism is introduced to limit the maximum gradient norm and avoid model oscillations; Dropout and layer normalization are used to improve the network's generalization ability.

5. The traditional image restoration method according to claim 1, which integrates low-resolution prior and efficient visual selection, is characterized in that... In step 2, the fine-grained restoration network includes a dynamically gated fusion module and a Mamba-driven efficient visual feature selection module. It adaptively fuses structural and texture features based on the initial restoration results through a dual-branch structure and a selective scanning mechanism. Specifically, it includes: S301: Feature encoding and multi-scale fusion, for the low-resolution restoration result R obtained in step 2. c The image is restored to a high-resolution scale using an upsampling module, maintaining consistency with the high-resolution damage mask image. The same spatial dimensions; then, the upsampled low-resolution repair result R c The input is fed into a structural feature encoder to extract its corresponding coarse structural feature map F. s Meanwhile, high-resolution damage mask images The texture feature map F is obtained after processing by another encoder. t Both encoders employ a combination of multi-layer convolution, normalization, and activation, exhibiting strong texture detail preservation capabilities; for coarse-structured feature maps F... s With texture feature map F t Residual connections and multi-scale convolution operations are introduced to integrate semantic information from different receptive fields, thereby enhancing its ability to express local and global features. S302: Dynamic Gated Fusion Module, which generates channel attention gating weights through bidirectional perception gating paths, dynamically adjusting the weight ratio of structural and texture features; specifically including: Let F be the coarse-structure feature. s ∈R C×H×W Texture feature F t ∈R C×H×W First, the two are concatenated along the channel dimension to form a fused feature tensor X: X=[F s ,F t ]∈R 2C×H×W ; To achieve bidirectional dynamic modulation between structural and texture features, perceptual gating paths are constructed for each of the two branches. For any branch i∈{s,t}, and the remaining branches j, the perceptual process is as follows: Among them, W i ∈R C×2C×3×3 For convolution kernel, For the normalization function, φ i (·) is a non-linear activation function; Then, channel attention gating weights G are generated using the Sigmoid function. i : G i =σ(Z i ); The features are modulated and fused based on this gating tensor: Where, α i β i These are trainable weight coefficients, and ⊙ represents element-wise multiplication; Finally, the two updated features are concatenated to obtain the fused feature F. u : This module implements a dynamic modulation mechanism from structural completion to texture refinement, enabling the network to automatically adjust feature weights according to the local context of the image, thereby improving the coordination between structure and texture and the quality of restoration. S303: A high-efficiency visual feature selection module driven by Mamba, used to enhance the ability of fine-grained repair networks to model spatial and temporal information, and to achieve selective feature enhancement.

6. The traditional image restoration method according to claim 5, which integrates low-resolution prior and efficient visual selection, is characterized in that... In step S303, the MambaEV module is introduced into the Mamba-driven efficient visual feature selection module. This module is based on state-space modeling technology, which fuses features through branching in the temporal and spatial dimensions. It utilizes a learnable selective scanning mechanism and a nonlinear fusion strategy, specifically including: Input fusion feature F u After entering the MambaEV module, two branches are loaded: The first branch, time-dimensional modeling, first performs RMSnorm and linear transformation, then enters the SSD selective scan module and applies SiLU activation to obtain time features: Where x is the input feature, w is the learnable parameter of RMSNorm, ∈ is the stability factor, and SSD realizes linear state modeling of sequence information; The second branch involves spatial information fusion. The features first pass through a linear layer, then through Sigmoid activation, to form auxiliary channel features F. b : F b =W2·Sigmoid(F in )+b2; Then, the outputs of the two branches are merged element-wise and linearly mapped: in, This represents element-wise multiplication. L represents additive fusion, and L(·) is a linear mapping layer.

7. The traditional image restoration method according to claim 1, which integrates low-resolution prior and efficient visual selection, is characterized in that... In step 3, during the training of the multi-source image model, a joint loss function is used to adjust the parameters of the multi-source image model until the multi-source image model converges. The joint loss function specifically includes: Pixel reconstruction loss Perceived loss Style loss Combat losses Where G is the generator network and D is the discriminator network; intermediate feature supervision loss Among them, F c R represents the intermediate feature map generated by the coarse-fix network. c For projection functions, it maps the feature map back to the RGB image space; The joint loss function is in the form of a weighted linear combination: The weights for each loss are: l rec =20,l perc =0.8,λ style =20,λ adv =0.5,λ m =1; During training, the Adam optimizer is used, along with a learning rate scheduling and weight decay mechanism to iteratively update parameters until the multi-source image model converges. After the validation set stabilizes, the learning rate is adjusted to ensure that the model reaches the global optimum in terms of structure preservation and detail recovery.

Images