Wall painting digital restoration method based on layered line draft constraint
Through layered line drawing constraints and deep learning technology, the problems of insufficient accuracy, low efficiency and limited applicability in traditional mural restoration methods were solved, and high-precision and high-efficiency digital restoration of murals was achieved.
Patent Information
- Application Number
- CN202510432925.1
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-04-08
- Publication Date
- 2025-09-12
AI Technical Summary
Traditional digital restoration methods for murals have problems such as insufficient restoration accuracy, low efficiency and limited applicability. It is especially difficult to achieve high-precision and high-efficiency restoration when dealing with complex and damaged murals.
Using the layered line drawing constraint method, the mural is decomposed into contour layer, detail layer and texture layer. A restoration algorithm is designed for each layer, and the restoration effect is improved through iterative optimization and deep learning technology.
It achieves high-precision and high-efficiency digital restoration of murals, is suitable for complex and damaged scenes, and improves the naturalness and intelligence of the restoration results.
Smart Images
Figure CN120634906A_ABST
Abstract
Description
Technical Field
[0001] This invention belongs to the field of image processing and digital restoration technology, and specifically relates to a method and system for digital restoration of murals based on layered line drawing constraints. Combining multi-scale image analysis and layered constraint optimization techniques, this invention decomposes the mural line drawing into contour, detail, and texture layers, and designs a restoration algorithm tailored to the characteristics of each layer. Deep learning technology is used to optimize the overall restoration effect, achieving high-precision and high-efficiency digital restoration of murals. This technology can be widely applied in fields such as cultural relic protection, cultural heritage digitization, and historical mural restoration. It is particularly suitable for repairing complex murals damaged by natural erosion, human vandalism, or age. Background Art
[0002] As an important cultural heritage, the digital restoration of murals is one of the key technologies in the field of cultural relic protection. Traditional mural digital restoration methods mainly rely on one-time line drawing and global restoration strategies, which have the following significant drawbacks:
[0003] 1. Insufficient repair accuracy
[0004] Traditional methods typically use a single line drawing to outline a mural, making it difficult to accurately express the multi-layered structure of a complex mural (such as outlines, details, and textures). For example, when repairing a mural containing fine patterns or complex geometric structures, a single line drawing cannot capture visual information at different scales, resulting in deviations between the repaired outline and the original. Furthermore, global repair algorithms may ignore layering constraints when dealing with localized damage, resulting in loss of detail or discontinuous textures.
[0005] 2. Inefficient repair
[0006] Traditional global restoration strategies (such as algorithms based on full-image diffusion) require processing high-dimensional data from the entire image, resulting in computational complexity that increases exponentially with image size and the area of damage. For murals with large areas of damage or complex structures, traditional methods consume significant computational resources, significantly reducing restoration efficiency. For example, gradient-domain restoration methods based on the Poisson equation face a bottleneck when solving linear equations for large murals.
[0007] 3. Limited applicability:
[0008] Existing methods are poorly adaptable to complex damage scenarios. For example, when a mural simultaneously exhibits structural fractures, localized peeling, and color fading, traditional algorithms struggle to reconcile different restoration objectives (such as structural integrity and texture consistency), resulting in stiff or unnatural restoration results. Furthermore, most methods rely on manual intervention to adjust parameters and lack intelligent restoration capabilities.
[0009] Analysis of existing technologies
[0010] Inpainting methods based on structure propagation: This type of method guides the inpainting by extracting image edge information, but does not introduce a hierarchical constraint mechanism. It is difficult to distinguish the inpainting priorities of the contour layer and the detail layer, resulting in conflicts between structure inpainting and detail filling.
[0011] Restoration method based on texture synthesis: Texture synthesis technology based on block matching or Markov random field (MRF) can generate locally consistent textures, but lacks global structural constraints and is prone to breakage or blurring at the edges of contours.
[0012] Deep learning restoration methods: Models such as generative adversarial networks (GANs) can generate realistic restoration results by learning from large amounts of data. However, their "black box" nature makes the restoration process uncontrollable, difficult to combine with expert knowledge, and has poor compatibility with line drawing constraints.
[0013] Technical pain points and improvement directions
[0014] The core problem of existing methods is that they fail to deeply integrate the multi-layered characteristics of murals with the restoration process. Specifically,
[0015] The structure, details, and texture information of the murals are not decoupled in layers, making it difficult to optimize the restoration algorithm in a targeted manner.
[0016] The lack of a phased constraint mechanism makes it impossible to gradually approach high-precision restoration results through iterative optimization;
[0017] There is insufficient integration of intelligent repair and expert experience, making it difficult to meet the diverse needs of complex damage scenarios. Summary of the Invention
[0018] This paper provides a method and system for digital mural restoration based on layered line drawing constraints. This approach aims to address the shortcomings of traditional mural restoration methods in terms of accuracy, efficiency, and applicability through layered line drawing, phased constraint restoration, and deep learning-assisted optimization. The core of this method lies in decomposing the mural line drawing into contour, detail, and texture layers, designing restoration algorithms tailored to the visual characteristics of each layer, and gradually refining the restoration results through iterative optimization, ultimately achieving high-fidelity and high-efficiency digital restoration.
[0019] The digital restoration method of murals based on layered line drawing constraints includes the following steps:
[0020] Step 1: Layered line drawing:
[0021] Perform hierarchical decomposition on the mural to be restored, separating the visual information of the restored mural into contour layer, detail layer and texture layer;
[0022] Step 2: Layered repair based on line drawing constraints:
[0023] Contour layer restoration: Using a structured propagation algorithm, the geometric contour information extracted from the line drawing is combined with the structural features of the surrounding area of the mural to guide pixel filling in the damaged area, ensuring that the geometric structure of the repaired area is consistent with the line drawing.
[0024] Detail layer restoration: Using texture synthesis algorithms, texture blocks matching the line drawing details are extracted from known areas to generate locally consistent patterns and lines, ensuring natural detail information in the restored area.
[0025] Texture layer restoration: A color migration algorithm based on the Lab color space is used to accurately match the original color tone of the faded areas of the mural. A Poisson equation is constructed to ensure the consistency of the texture gradient of the restored image. Texture layer restoration is achieved by minimizing the energy function.
[0026] Step 3: Iteratively optimize parameters:
[0027] The restoration results of the contour layer, detail layer, and texture layer are regarded as interdependent variables. A joint optimization objective function is constructed to dynamically adjust the restoration parameters of each layer according to the current restoration quality. The restoration of each layer in step 2 is performed until the energy function of each layer decreases by less than the preset threshold or the maximum number of iterations is reached.
[0028] Step 4: Secondary repair based on neural network:
[0029] The restored contour layer, detail layer and texture layer are fused to obtain the restored image, and the trained neural network is used to perform secondary restoration on the restored image to generate a high-quality image consistent with the style of the original mural.
[0030] First, the original mural image is decomposed hierarchically using multi-scale analysis techniques, separating its visual information into contour, detail, and texture layers. The contour layer focuses on the mural's primary structure and geometry, the detail layer captures local features (such as ornamentation and line direction), and the texture layer extracts color distribution and microtexture patterns. Layered line drawings are automatically generated using an algorithm. This layering mechanism not only reduces the complexity of drawing a single line drawing but also provides clear constraints for subsequent restoration, ensuring a progressive, layer-by-layer restoration process that accurately matches the multi-layered features of the original mural.
[0031] For each layer of line drawing, the present invention designs a targeted repair algorithm:
[0032] The contour layer restoration uses a structured propagation algorithm, which extracts geometric contour information from the line drawing and combines it with the structural features of the surrounding area to guide pixel filling in the damaged area. This algorithm uses an energy minimization model to ensure that the restored contours are strictly aligned with the line drawing while maintaining geometric continuity with adjacent areas.
[0033] Detail-level restoration is based on texture synthesis technology. Using Markov Random Fields (MRFs), we extract texture patches from known areas that match the line drawing details, generating locally consistent patterns and lines. This stage prioritizes the local naturalness of the restoration results, avoiding the blurring or fragmentation of details that can occur with global restoration methods, as is the case with traditional methods.
[0034] Texture layer restoration combines color migration with gradient domain processing techniques, adjusting the color distribution and gradient information of the restored area to achieve a smooth transition with the surrounding area. A color migration algorithm based on the Lab color space is used to accurately match the original hues of faded areas of the mural, and a Poisson equation is constructed to optimize color consistency in the gradient domain.
[0035] After completing the layered restoration, the present invention performs multiple rounds of adjustments to the restoration results through an iterative optimization strategy. Each round of optimization takes the current restoration result as input, combines the line draft constraints with the surrounding area features, and gradually refines the matching accuracy of the contour, details, and texture. In addition, the present invention introduces a deep learning model (generative adversarial network GAN), uses real mural images to simulate damaged images, and after layered restoration (contour layer + detail layer + texture layer restoration results), it is paired with the corresponding real complete mural image to form a "restored image-real image" pair for supervised training. This model can globally optimize the restoration results and significantly improve the restoration quality of complex damaged areas.
[0036] The core innovation of this invention lies in the deep integration of the multi-layered characteristics of murals with the restoration process. Through the layered line draft constraint mechanism, the problems of insufficient precision and low efficiency caused by global restoration in traditional methods are solved; combined with structural propagation, texture synthesis and color migration algorithms, accurate restoration from macro contours to micro textures is achieved; at the same time, the introduction of iterative optimization and deep learning technology further enhances the naturalness and intelligence of the restoration results. Compared with existing technologies, this invention shows significant advantages in the restoration of complex and damaged murals, and is particularly suitable for the urgent need for high-precision and high-efficiency restoration in the field of cultural relics protection. BRIEF DESCRIPTION OF THE DRAWINGS Figure 1 This is a flow chart of the mural digital restoration method based on layered line drawing constraints of the present invention. DETAILED DESCRIPTION
[0037] The following will provide a clear and complete description of the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. All other embodiments obtained by ordinary technicians in this field based on the embodiments of the present invention without making any creative efforts shall fall within the scope of protection of the present invention.
[0038] S10. Layered line draft drawing, hierarchical decomposition of the mural to be restored, separating the visual information of the restored mural into contour layer, detail layer and texture layer. The contour layer focuses on the main structure and geometric form of the mural, corresponding to the large-scale geometric structure of the mural, and is low-frequency information. The detail layer captures local features (such as patterns, line direction), corresponding to local features, and is medium- and high-frequency information. The texture layer extracts color distribution and micro-texture patterns, corresponding to color distribution, tiny particles or traces of weathering, and is high-frequency information. The layered line draft is automatically generated by an algorithm. This layering mechanism not only reduces the complexity of drawing a single line draft, but also provides clear constraints for subsequent restoration, ensuring that the restoration process can be progressive layer by layer and accurately match the multi-level features of the original mural.
[0039] The specific steps include:
[0040] S101, multi-scale preprocessing and feature extraction.
[0041] Input preprocessing: denoise the original mural, enhance the contrast, and reduce interference.
[0042] Use multi-scale decomposition tools to perform multi-layer slicing of images: use Gaussian pyramids to generate multi-scale image sequences through multiple downsampling and Gaussian filtering to extract different frequency components; use wavelet transform (such as Daubechies wavelet) to decompose the image into low-frequency (LL), horizontal high-frequency (LH), vertical high-frequency (HL) and diagonal high-frequency (HH) sub-bands to separate contours and details; use edge-preserving filters (such as bilateral filtering and guided filtering) to smooth textures while preserving edges to assist in contour extraction.
[0043] S102: Contour layer extraction (extracting the continuous closed contour of the main structure of the mural).
[0044] Technical Implementation: The Canny algorithm is used to obtain initial edges at the lowest-resolution pyramid level. The low-resolution contours are then mapped to the original scale to serve as initialization for the Snake algorithm. Local breaks are corrected in the mid-resolution level using morphological operations (morphological closing: five dilations and erosions using a 3×3 elliptical kernel to fill gaps ≤ 5 pixels) to generate continuous contours.
[0045] S103, detail layer extraction: separation of local features, such as patterns, brush strokes, and peeling marks.
[0046] Technical Implementation: First, high-frequency information is extracted, and details are extracted from the high-frequency subbands (LH / HL / HH) of the wavelet transform or the Gaussian pyramid residual map. Texture saliency analysis is performed using the Local Binary Pattern (LBP) (to quantify local texture patterns and distinguish between detail and smooth areas) and the Gray Level Co-occurrence Matrix (GLCM) (to calculate indicators such as texture contrast and correlation and screen for detail features). Detailed line drawings are generated through threshold segmentation.
[0047] S104: Texture layer extraction to separate color distribution, grain texture, and weathering traces.
[0048] Technical Implementation: First, color space decomposition is performed, converting the image to Lab color space and extracting the luminance (L) and chrominance (a / b) channels. The luminance channel is used for texture analysis, and the chrominance channel is used for color transfer. Texture modeling is then performed using a Gabor filter bank and multi-directional, multi-scale filtering to capture periodic texture patterns. Local phase quantization (LPQ) is performed to analyze the frequency domain characteristics of the texture and distinguish between uniform and complex textured areas.
[0049] S20 performs layered repair based on line drawing constraints, including repairing the outline layer, detail layer, and texture layer. The specific steps are as follows:
[0050] S201: Contour layer restoration. This step repairs the main geometric structure of the mural based on the outline sketch, ensuring that the outline of the repaired area is consistent with the outline. Contour layer restoration is implemented using the structure propagation algorithm, using the outline sketch as a hard constraint to define the geometric boundaries of the repaired area. An energy function is constructed to minimize the geometric deviation of the repaired area from the outline sketch while maintaining continuity with the surrounding structure. The energy function is as follows:
[0051]
[0052] Where Ω is the area to be repaired (damaged area) in the image. The damaged area needs to be accurately marked manually or automatically by an algorithm. u is the image function after repair, which is the target variable of energy optimization. By adjusting u, the contour energy function E contour Minimum, u0 is the image function of the complete area around the damaged area (correct pixel value is known). is the gradient operator, which calculates the rate of change of the image in the horizontal and vertical directions. λ1 is the weight coefficient, which is used to control the strength of the line draft constraint. The boundary of the area to be repaired (the boundary between the damaged area and the intact area) is used to limit the calculation range of the line draft constraint item and force the line draft to match only at the boundary; u 线稿 The ideal contour value defined for the contour layer line draft is the contour layer line draft generated by step S10.
[0053] The physical meaning of the energy function is as follows:
[0054] Item 1 Gradient difference item: ensures that the internal structure of the repaired area (such as edge direction and geometry) is consistent with the surrounding area to avoid abrupt or broken contours after repair.
[0055] Item 2 Line draft constraint: The boundary outline after repair is strictly aligned with the line draft to ensure the accuracy of large-scale structures.
[0056] Solution: The conjugate gradient method is used to iteratively solve the partial differential equation (PDE) to obtain the repaired contour layer u.
[0057] S202, detail layer restoration: This step generates local features based on the detailed line drawing to ensure that details in the repaired area (such as patterns and cracks) are natural and realistic. This stage focuses on the local naturalness of the restoration results, avoiding the blurring or fragmentation of details caused by global restoration in traditional methods.
[0058] This embodiment adopts a block-based texture synthesis algorithm, and uses an energy function to constrain the matching of texture blocks and the consistency of adjacent blocks. The energy function formula is as follows:
[0059]
[0060] D(p)=||Line(Patch candidate )-Line(p)|| 2
[0061] V(p,q)=||Boundary(p)-Boundary(q)|| 2
[0062] Among them, D(p) is the line drawing matching item. Its core task is to measure the matching degree between the candidate texture block and the line drawing features at the corresponding position of the damaged area. p is the coordinate of a pixel point or sub-block (Patch) in the damaged area. candidate is a candidate texture block, which is selected from the intact area around p and is used to fill the damaged area at position p. Line(·) is the line drawing corresponding to the texture layer generated by step S10 (including the damaged and intact areas). V(p,q) is a neighborhood smoothing term, whose core task is to ensure a natural texture transition between block p and domain block q. Boundary(·) is a function used to measure the consistency of texture block boundaries, and is used to calculate the gradient or color difference of pixels at the boundary of adjacent blocks.
[0063] Boundary(·) is calculated based on the following two methods:
[0064] Method 1: Calculate pixel difference to control color consistency:
[0065]
[0066] Method 2: Gradient difference to control structural consistency:
[0067]
[0068] Where R represents the block and λ2 is the weight coefficient, which balances the importance of block matching and consistency between adjacent blocks.
[0069] In this implementation, we define a domain system, constrain the texture consistency of adjacent blocks, and optimize the block matching process through belief propagation to ensure that details align with the line art. We also use bilateral filtering to eliminate seams between blocks and enhance the natural transition of details.
[0070] S203, texture layer restoration. This step combines color migration and gradient domain processing techniques to adjust the color distribution and gradient information of the restored area to achieve a smooth transition with the surrounding area. A color migration algorithm based on the Lab color space is used to accurately match the original color tone of the faded area of the mural. A Poisson equation is constructed to ensure the consistency of the texture gradient of the restored image. Texture layer restoration is achieved by minimizing the energy function. The energy function is as follows:
[0071]
[0072] in, Used to force the color (Lab space) of the repaired area to match the surrounding area to achieve color alignment. It is used to ensure that the texture gradient of the repaired area transitions naturally with the surrounding area. λ3 is the weight coefficient used to balance the importance of color and gradient constraints.
[0073] In this embodiment, the specific process of texture layer repair is as follows:
[0074] S2031. First, use color scaling technology to provide a preliminary basis for color alignment:
[0075] By adjusting the mean and variance, a linear transformation is used to map the damaged area color to the target color distribution:
[0076] I 修复 =αI 源 +β
[0077] I 修复 is the color value of the damaged area after repair, I 源 is the original color value of the damaged area; α is the color scaling factor, which is used to adjust the contrast of the repaired area color; β is the color shift factor, which is used to adjust the brightness of the repaired area color;
[0078] α and β are determined by fitting the mean and variance of the surrounding area using the least squares method, as follows:
[0079] β=μ 周围 -αμ 源 ,
[0080] Among them, μ 周围 and σ 周围are the mean and standard deviation of the color values around the damaged area, μ 源 and σ 源 The mean and standard deviation of the color values of the damaged areas respectively;
[0081] S2032. Then, based on the color alignment, the gradient term is further optimized to ensure a natural texture transition. Specifically, the Poisson equation is solved. Make the gradient field of the repair area consistent with the surrounding area; where ΔI 修复 is the Laplace operator (second-order derivative) of the repair area, The gradient field of the damaged area is calculated using a differential operator (such as Sobel and Prewitt). Div() is a divergence operation, and finally a repaired image that satisfies the gradient constraint is obtained.
[0082] S2033, finally by minimizing the energy function E texture Perform joint optimization while ensuring the consistency of color and structure to complete the texture layer repair.
[0083] S30, iterative optimization of the number of layers: the restoration results of the contour layer, detail layer, and texture layer are regarded as interdependent variables, a joint optimization objective function is constructed, the restoration parameters of each layer are dynamically adjusted according to the current restoration quality, and the restoration of each layer in step 2 is performed until the energy function of each layer decreases by less than the preset threshold or the maximum number of iterations is reached. The weight of each layer is dynamically adjusted according to the current restoration quality (for example, the weight of the contour layer is initially higher and gradually decreases in the later stage).
[0084] The formula of the joint optimization objective function is as follows:
[0085] E total =w1E contour +w2E detail +w3E texture
[0086] Where w1 represents the weight coefficient of the energy function of the contour layer restoration, w2 is the weight coefficient of the energy function of the detail layer restoration, and w3 is the weight coefficient of the energy function of the texture layer restoration. total , the initial weights w1>w2>w3 (prioritize the correctness of the contour), and gradually adjust the weights to balance the details and textures.
[0087] The optimization algorithm used is the alternating direction method of multipliers (ADMM), which decomposes the optimization problem into subproblems and solves them iteratively:
[0088] Contour layer update: fix details and textures, optimize contour energy function B contour .
[0089] Detail layer update: fixed outline and texture, optimized detail energy function Edet ail .
[0090] Texture layer update: fixed outline and details, optimized texture energy function E texture .
[0091] Termination condition: The optimization is terminated when the energy function decreases less than the preset threshold (ΔE<0.1%) or the maximum number of iterations (100 times) is reached.
[0092] Data transfer and loop: After each iteration, the optimization results are fed back to step S20 for repair of each layer, and the repair parameters (such as block size and color migration coefficient) are dynamically adjusted. If inconsistencies between layers are detected (such as misalignment of outlines and details), local re-repair is triggered.
[0093] S40. Secondary restoration based on neural network: The restored contour layer, detail layer, and texture layer are fused to obtain the restored image. The restored image is then restored using the trained neural network to generate a high-quality image consistent with the style of the original mural.
[0094] The contour layer, detail layer, and texture layer correspond to visual information of different frequencies (low-frequency structure, mid- and high-frequency details, and high-frequency texture). Each layer may produce independent gradient fields after restoration, and direct superposition can lead to conflicts. In this embodiment, the gradient fields of each layer are unified and fused using the Poisson equation, coordinating the restoration results of different scales and avoiding misalignment or contradictions between layers (such as mismatching of contours and textures).
[0095] The Poisson equation is as follows:
[0096]
[0097] Among them, I is the restored image function, w1, w2, w3 and the three weight coefficients in the joint optimization objective function, G contour Represents the gradient of the contour layer after repair, G det ail Represents the gradient of the detail layer after restoration, G texture Represents the gradient of the restored texture layer. By discretizing the Poisson equation, the fused gradient field is converted into a system of linear equations and solved. Specifically, the conjugate gradient method (for images of normal resolution) or the multigrid method (for images above 4k) is used to reconstruct the image from the second-order gradient to obtain the restored image.
[0098] The trained neural network is then used to perform a secondary restoration on the restored image, generating a high-quality image consistent with the style of the original mural. In this embodiment, a real mural image is used to simulate a damaged image (artificial damage). After layering and restoration (contour layer + detail layer + texture layer restoration results), the restored image is obtained and fused. This restored image is then paired with the corresponding real, intact mural image to form a "restored image-real image" pair. This "restored image-real image" pair is used as sample data to train the neural network.
[0099] The neural network of this embodiment uses the GAN model (proposed by Ian Goodfellow et al. in their 2014 paper "Generative Adversarial Nets" (arXiv:1406.2661)). The GAN model structure is as follows:
[0100] Generator Architecture: Utilizes a U-Net structure, consisting of an encoder (downsampling) and a decoder (upsampling), with skip connections between them to preserve multi-scale features. Core Module: Residual Block: Enhances the network's ability to capture detail. Attention Gate: Focuses on damaged areas, improving the targeted nature of the repair. Input: Repaired image (RGB channels). Output: Optimized image (same size as the input).
[0101] Discriminator Architecture: Using the PatchGAN architecture, the discriminator distinguishes authenticity from counterfeit images within local regions (e.g., 70×70 pixel patches), enhancing sensitivity to detail. Core Module: Convolutional layer stacking: Gradually extracts high-level features. Spectral Normalization: Stabilizes the training process and prevents mode collapse. Input: Generator output image or real mural image. Output: Discrimination probability (real / fake) for each image patch.
[0102] Training Strategy: A two-stage training approach is employed. First, the generator is pre-trained, using only content and style losses to quickly converge to reasonable restoration results. The discriminator is then added, and the generator and discriminator are alternately optimized to gradually improve the fidelity of the generated images.
[0103] Evaluation and Verification:
[0104] Quantitative indicators:
[0105] PSNR (Peak Signal-to-Noise Ratio): Measures the pixel-level difference between the generated image and the real image. SSIM (Structural Similarity): Assess the consistency of the image in terms of structure, brightness, and contrast. FID (Fréchet Inception Distance): Calculates the distance between the generated image and the real image distribution by extracting features using a pre-trained Inception network.
[0106] Qualitative assessment:
[0107] Invite cultural relics experts to conduct a visual inspection of the restoration results, focusing on stylistic consistency, detail fidelity, and overall coordination. Compare local areas (such as damaged edges and texture transitions) before and after restoration to verify the restoration results.
[0108] While embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that various changes, modifications, substitutions, and variations may be made to these embodiments without departing from the principles and spirit of the invention, and that the scope of the invention is defined by the appended claims and their equivalents.
Claims
1. A mural digital restoration method based on layered line drawing constraints, comprising the following steps: Step 1: Layered line drawing: Perform hierarchical decomposition on the mural to be restored, separating the visual information of the restored mural into contour layer, detail layer and texture layer; Step 2: Layered repair based on line drawing constraints: Contour layer restoration: Using a structured propagation algorithm, the geometric contour information extracted from the line drawing is combined with the structural features of the surrounding area of the mural to guide pixel filling in the damaged area, ensuring that the geometric structure of the repaired area is consistent with the line drawing. Detail layer restoration: Using texture synthesis algorithms, texture blocks matching the line drawing details are extracted from known areas to generate locally consistent patterns and lines, ensuring natural detail information in the restored area. Texture layer restoration: A color migration algorithm based on the Lab color space is used to accurately match the original color tone of the faded areas of the mural. A Poisson equation is constructed to ensure the consistency of the texture gradient of the restored image. Texture layer restoration is achieved by minimizing the energy function. Step 3: Iteratively optimize parameters: The restoration results of the contour layer, detail layer, and texture layer are regarded as interdependent variables. A joint optimization objective function is constructed to dynamically adjust the restoration parameters of each layer according to the current restoration quality. The restoration of each layer in step 2 is performed until the energy function of each layer decreases by less than the preset threshold or the maximum number of iterations is reached. Step 4: Secondary repair based on neural network: The restored contour layer, detail layer and texture layer are fused to obtain the restored image, and the trained neural network is used to perform secondary restoration on the restored image to generate a high-quality image consistent with the style of the original mural.
2. The method for digital restoration of murals based on layered line drawing constraints according to claim 1, characterized in that: In step 1, the contour layer corresponds to the large-scale geometric structure of the mural and is low-frequency information; the detail layer corresponds to the local features and is medium- and high-frequency information; and the texture layer corresponds to the color distribution, tiny particles or weathering traces and is high-frequency information.
3. The method for digital restoration of murals based on layered line drawing constraints according to claim 1, characterized in that: The energy function of the contour layer repair in step 2 is as follows: Among them, Ω is the area to be repaired in the image, u is the image function after repair, which is the target variable of energy optimization. By adjusting u, the contour energy function E contour minimum, u0 is the image function of the intact area around the damaged area, is the gradient operator, which calculates the rate of change of the image in the horizontal and vertical directions. λ1 is the weight coefficient, which is used to control the strength of the line draft constraint. The boundary of the area to be repaired is used to limit the calculation range of the line draft constraint item, and the line draft is forced to match only at the boundary; u 线稿 Ideal contour values defined for the contour layer line art generated in step 1.
4. The method for digital restoration of murals based on layered line drawing constraints according to claim 1, characterized in that: The specific process of detail layer restoration in step 2 is as follows: a block-based texture synthesis algorithm is used to constrain the matching of texture blocks and the consistency of adjacent blocks through an energy function. The energy function formula is as follows: D(p)=||Line(Patch candidate )-Line(p)|| 2 V(p,q)=||Boundary(p)-Boundary(q)|| 2 Among them, D(p) is the line drawing matching item. Its core task is to measure the matching degree between the candidate texture block and the line drawing features at the corresponding position of the damaged area. P is the coordinate of a pixel point or sub-block in the damaged area. candidate is a candidate texture block selected from the intact area around p and used to fill the damaged area position p. Line(·) is the line drawing corresponding to the texture layer generated by step 1. V(p,q) is a neighborhood smoothing term, whose core task is to ensure a natural texture transition between block p and domain block q. Boundary(·) is a function used to measure the consistency of texture block boundaries, which is used to calculate the gradient or color difference of pixels at the boundary of adjacent blocks.
5. The method for digital restoration of murals based on layered line drawing constraints according to claim 6, characterized in that: Boundary(·) is calculated in one of two ways: Method 1: Calculate pixel difference to control color consistency: Method 2: Gradient difference to control structural consistency: Where R represents the block and λ2 is the weight coefficient, which balances the importance of block matching and consistency between adjacent blocks.
6. The method for digital restoration of murals based on layered line drawing constraints according to claim 1, characterized in that: The energy function of texture layer repair in step 2 is as follows: in, Used to force the color (Lab space) of the repaired area to match the surrounding area to achieve color alignment. It is used to ensure that the texture gradient of the repaired area transitions naturally with the surrounding area. λ3 is the weight coefficient used to balance the importance of color and gradient constraints.
7. The method for digital restoration of murals based on layered line drawing constraints according to claim 6, characterized in that: The specific process of texture layer repair in step 2 is as follows: S2031. First, use color scaling technology to provide a preliminary basis for color alignment: By adjusting the mean and variance, a linear transformation is used to map the damaged area color to the target color distribution: I 修复 =αI 源 +b I 修复 is the color value of the damaged area after repair, I 源 is the original color value of the damaged area; α is the color scaling factor, which is used to adjust the contrast of the repaired area color; β is the color shift coefficient, which is used to adjust the brightness of the repaired area color; α and β are determined by fitting the mean and variance of the surrounding area using the least squares method, as follows: β=μ 周围 -am 源 , Among them, μ 周围 and σ 周围 are the mean and standard deviation of the color values around the damaged area, μ 源 and σ 源 The mean and standard deviation of the color values of the damaged areas respectively; S2032. Then, based on the color alignment, the gradient term is further optimized to ensure a natural texture transition. Specifically, the Poisson equation is solved. Make the gradient field of the repair area consistent with the surrounding area; where ΔI 修复 is the Laplacian operator of the repair area, The gradient field of the damaged area is calculated using the difference operator, div() is a divergence operation, and finally the repaired image that satisfies the gradient constraint is obtained; S2033, finally by minimizing the energy function E texture Perform joint optimization while ensuring the consistency of color and structure to complete the texture layer repair.
8. The method for digital restoration of murals based on layered line drawing constraints according to claim 1, characterized in that: In step 3, the formula of the joint optimization objective function is as follows: E total =w1E contour +w2E det ail +w3E texture Where w1 represents the weight coefficient of the energy function of the contour layer repair, w2 is the weight coefficient of the energy function of the detail layer repair, and w3 is the weight coefficient of the energy function of the texture layer repair. Joint Optimization E total , decompose the optimization problem into sub-problems and solve them iteratively: Contour layer update: fix details and textures, optimize contour energy function E contour , Detail layer update: fixed outline and texture, optimized detail energy function E det ail , Texture layer update: fixed outline and details, optimized texture energy function E texture .
9. The method for digital restoration of murals based on layered line drawing constraints according to claim 8, characterized in that: In step 4, a Poisson equation is constructed, and the gradient fields of the contour layer, detail layer, and texture layer are unified and fused through the Poisson equation. By discretizing the Poisson equation, the fused gradient field is converted into a linear equation system and solved to obtain the repaired image. The formula of the Poisson equation is as follows: Among them, I is the restored image function, w1, w2, w3 are consistent with the three weight coefficients in the joint optimization objective function, G contour Represents the gradient of the contour layer after repair, G det ail Represents the gradient of the detail layer after restoration, G texture Represents the gradient of the texture layer after repair.
10. The mural digital restoration method based on layered line drawing constraints according to claim 1 is characterized by: In step 4, a real mural image is used to simulate a damaged image. After step 1 to step 3, the layered repair and fusion are performed to obtain a repaired image. The repaired image is paired with the corresponding real complete mural image to form a "repaired image-real image" pair. This "repaired image-real image" is used as sample data to train the neural network.