Historical cultural relic image restoration method and system based on deep learning

Abstract

The invention relates to a historical cultural relic image restoration method and system based on deep learning, in particular to the field of historical cultural relic image restoration, and aims to realize accurate separation and extraction of geometric, material and pattern features of cultural relics through a physical attribute decoupling encoder and lay an accurate parameterization foundation for subsequent restoration; a structure reasoning and complementing module is utilized, and a geometric prior rule base and symmetry and curvature continuity double loss constraints are combined, so that the complemented structure is ensured not only to be semantic coherent, but also to conform to the objective physical form of the cultural relics; on the basis, a condition content generation module generates materials and patterns which are consistent with ontology vision and unified in historical style through a cross-modal attention and multi-scale discrimination mechanism by taking a completion structure as a condition; and finally, multi-source features are fused through an image synthesis decoder, and a repair result with high fidelity in structure, material and vision is output, so that a reliable visual simulation basis is provided for entity repair, and the repair risk is remarkably reduced.

Description

Technical Field

[0001] This invention relates to the field of historical artifact image restoration, and more specifically, to a method and system for historical artifact image restoration based on deep learning. Background Technology

[0002] Historical artifacts suffer damage, contamination, and fading over time due to physical, chemical, or biological processes. High-quality digital image restoration is crucial for the digital preservation, research, and display of these artifacts. With the development of deep learning technology, especially breakthroughs in generative models such as generative adversarial networks and diffusion models, image restoration has made significant progress. Numerous deep learning-based image restoration methods have emerged in current technologies.

[0003] Currently, deep learning-based image inpainting technology has made significant progress, with advanced generative models such as generative adversarial networks and diffusion models showing great potential in image completion tasks.

[0004] However, directly applying such general model frameworks to the restoration of historical artifact images still faces core challenges: existing methods usually focus on pixel-level content filling and high-level semantic coherence, while lacking a mechanism for explicit modeling and decoupling representation of the artifact's multi-dimensional physical properties (such as geometric structure, surface material, and visual patterns). This neglect of the underlying physical mechanism makes it difficult for the model to learn the essential laws of artifact composition, resulting in inherent deviations between the restoration results and the original artifact in terms of material properties and structural logic. Although the generated content is visually reasonable, it lacks accurate physical basis and cannot meet the high requirements of objectivity and verifiability in artifact restoration. Summary of the Invention

[0005] This invention addresses the technical problems existing in the prior art by providing a method and system for restoring historical artifact images based on deep learning, thereby resolving the issues raised in the background section.

[0006] The technical solution of the present invention to solve the above-mentioned technical problems is as follows: a historical artifact image restoration system based on deep learning, specifically including: a physical attribute decoupling encoder module, a structural reasoning and completion module, a conditional content generation module, and an image synthesis decoder module;

[0007] Physical property decoupling encoder module: used to extract input maps using a multi-branch deep convolutional neural network.

[0008] The geometric structure, surface material, and visual pattern features of the image are used to achieve a decoupled representation of physical properties in the latent space by minimizing the loss function based on correlation.

[0009] Structural Reasoning and Completion Module: This module receives the decoupled geometric structural features, infers the structure of the missing regions based on the geometric prior rule base, and optimizes it through symmetry constraint loss and curvature continuity loss to output a complete geometric structural representation.

[0010] Conditional content generation module: Based on the completed geometric structure, it generates material textures and decorative patterns, and ensures that the generated content is visually consistent with the artifact itself.

[0011] Image synthesis decoder module: used to fuse and complete the structure, generate material and pattern features, and output the repair result through feature reconstruction.

[0012] In a preferred embodiment, the physical property decoupling encoder module is a three-branch parallel deep convolutional neural network, and the functions and structures of each branch are as follows:

[0013] Geometric structure feature extraction branch: Multi-layer convolutional network is constructed, specific padding maintains consistent resolution, ReLU activation captures geometric contours and key points, outputs geometric structure features, and encoder parameters are independently optimized;

[0014] Surface material feature extraction branch: Multi-layer convolutional network is constructed, and batch normalization is added after each convolution. Activation functions that alleviate neuron death are used to capture material-related features and output surface material features. Encoder parameters are optimized independently.

[0015] Visual pattern feature extraction branch: a multi-layer convolutional network with dilated convolutions to expand the receptive field, using an adaptive activation function to capture pattern-related features and output visual pattern features. The encoder parameters are independent of the first two branches.

[0016] In a preferred embodiment, the physical attribute decoupling encoder module introduces a correlation minimization loss function constraint network, as follows:

[0017] Based on the three types of features extracted by the three branches, the Frobenius norm of the covariance matrix of the two types of features is used to measure their linear correlation. The smaller the Frobenius norm value, the lower the linear correlation and the better the decoupling effect. Weight coefficients are set for the correlation of the three sets of feature combinations to ensure that the importance is balanced in the loss calculation. The constraint objective is to keep the loss value at a low level. If the loss does not meet the target during training, the parameters of the branch encoder are adjusted through backpropagation and iterated until the loss meets the requirements, so as to achieve effective decoupling of features.

[0018] In a preferred embodiment, the structural reasoning and completion module takes the decoupled geometric structural features as the processing object and employs a graph attention network, as follows:

[0019] Input layer: Selects key structural features from geometric features, transforms them into graph node features, and forms graph structure data;

[0020] Attention layer: The multi-head attention mechanism is constructed by first calculating the original attention scores between nodes, and then normalizing the scores to obtain the attention weights of neighboring nodes to the central node.

[0021] Output layer: Linearly fused and weighted node features, outputting a geometric structure representation containing complete topological relationships;

[0022] The module has a built-in geometric prior rule base. When reasoning about missing regions, it compares the preliminary prediction results with the rule base and selects reasonable results.

[0023] In a preferred embodiment, the structural reasoning and completion module introduces symmetry constraint loss and curvature continuity loss to optimize completion, as follows:

[0024] Symmetry constraint loss: Identify symmetrical node pairs in the geometric structure, calculate the feature differences between node pairs and sum them up and average them. The lower the loss value, the better the symmetry, ensuring that the structure conforms to the inherent symmetry properties.

[0025] Curvature continuity loss: Select curvature sampling points on the outline of the cultural relic, calculate the absolute value of the curvature difference between adjacent sampling points and sum them up and average them. The lower the loss value, the better the continuity, ensuring a smooth and natural outline.

[0026] The two losses are weighted to construct the total loss. The graph neural network parameters are adjusted through backpropagation and iterated until the total loss reaches the target, outputting a high-quality geometric structure representation.

[0027] In a preferred embodiment, the conditional content generation module adopts a conditional diffusion model, injects attention modules with geometric structures as conditions, and aligns material and pattern features through cross-modal attention to ensure that the generated content conforms to the geometric shape and historical style.

[0028] In a preferred embodiment, the conditional content generation module includes a multi-scale discriminator for evaluating the visual consistency between the generated content and the artifact itself.

[0029] In a preferred embodiment, the image synthesis decoder module fuses features and reconstructs the image using a U-Net architecture with skip connections, and jointly optimizes the output using perceptual loss and adversarial loss.

[0030] This application also provides a processing method for a deep learning-based image restoration system for historical artifacts, specifically including the following steps:

[0031] Step S1, Decoupling and Extraction of Physical Attribute Features: The physical attribute decoupling encoder module is called to extract the geometric structure, surface material and visual pattern features of the input image through a three-branch parallel deep convolutional neural network; the correlation minimization loss function is introduced to constrain the network, and the branch encoder parameters are adjusted through backpropagation to achieve the decoupling representation of physical attributes in the latent space;

[0032] Step S2, Geometric Structure Reasoning and Completion: Call the structure reasoning and completion module, take the decoupled geometric structure features as input, and use graph attention network to learn the topological relationship of the cultural relic image; reason about the structure of the missing region based on the built-in geometric prior rule library, optimize the completion result through symmetry constraint loss and curvature continuity loss, and output the complete geometric structure representation;

[0033] Step S3, Conditional Material and Pattern Generation: Call the conditional content generation module, use the completed geometric structure as a condition, and use the conditional diffusion model to generate material textures and decorative patterns; align the material, pattern and geometric features through a cross-modal attention mechanism, and use a multi-scale discriminator to evaluate the visual consistency between the generated content and the artifact itself.

[0034] Step S4, Image Synthesis and Restoration Output: Call the image synthesis decoder module to stitch and fuse the completed geometric structure, generated material texture and visual pattern features; realize feature-to-image reconstruction through the U-Net architecture with skip connections, and use perceptual loss and adversarial loss for joint optimization to output the final cultural relic image restoration result.

[0035] The beneficial effects of this invention are as follows: A physical attribute decoupling encoder enables precise separation and extraction of the geometric, material, and pattern features of cultural relics, laying an accurate parametric foundation for subsequent restoration. Furthermore, a structural reasoning and completion module, combined with a geometric prior rule base and dual loss constraints of symmetry and curvature continuity, ensures that the completed structure is not only semantically coherent but also conforms to the objective physical form of the cultural relic. Based on this, a conditional content generation module, using the completed structure as a condition, generates materials and patterns consistent with the ontological vision and historical style through cross-modal attention and multi-scale discrimination mechanisms. Finally, an image synthesis decoder fuses multi-source features to output a restoration result with high fidelity in structure, material, and vision, providing a reliable visual simulation basis for physical restoration and significantly reducing restoration risks. Attached Figure Description

[0036] Figure 1 This is a flowchart of the method of the present invention;

[0037] Figure 2 This is a block diagram of the system structure of the present invention. Detailed Implementation

[0038] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0039] In the description of this application, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of indicated technical features. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more of the stated features. In the description of this application, "multiple" means two or more, unless otherwise explicitly specified.

[0040] In the description of this application, the term "for example" is used to mean "used as an example, illustration, or description." Any embodiment described as "for example" in this application is not necessarily to be construed as being more preferred or advantageous than other embodiments. The following description is provided to enable any person skilled in the art to make and use the invention. Details are set forth in the following description for purposes of explanation. It should be understood that those skilled in the art will recognize that the invention can be made without using these specific details. In other instances, well-known structures and processes will not be described in detail to avoid obscuring the description of the invention with unnecessary detail. Therefore, the invention is not intended to be limited to the embodiments shown, but is consistent with the broadest scope of the principles and features disclosed in this application.

[0041] This embodiment provides, for example Figure 1 The image restoration system for historical artifacts based on deep learning, as shown, includes the following steps:

[0042] Step S1, Decoupling and Extraction of Physical Attribute Features: The physical attribute decoupling encoder module is called to extract the geometric structure, surface material and visual pattern features of the input image through a three-branch parallel deep convolutional neural network; the correlation minimization loss function is introduced to constrain the network, and the branch encoder parameters are adjusted through backpropagation to achieve the decoupling representation of physical attributes in the latent space;

[0043] Step S2, Geometric Structure Reasoning and Completion: Call the structure reasoning and completion module, take the decoupled geometric structure features as input, and use graph attention network to learn the topological relationship of the cultural relic image; reason about the structure of the missing region based on the built-in geometric prior rule library, optimize the completion result through symmetry constraint loss and curvature continuity loss, and output the complete geometric structure representation;

[0044] Step S3, Conditional Material and Pattern Generation: Call the conditional content generation module, use the completed geometric structure as a condition, and use the conditional diffusion model to generate material textures and decorative patterns; align the material, pattern and geometric features through a cross-modal attention mechanism, and use a multi-scale discriminator to evaluate the visual consistency between the generated content and the artifact itself.

[0045] Step S4, Image Synthesis and Restoration Output: Call the image synthesis decoder module to stitch and fuse the completed geometric structure, generated material texture and visual pattern features; realize feature-to-image reconstruction through the U-Net architecture with skip connections, and use perceptual loss and adversarial loss for joint optimization to output the final cultural relic image restoration result.

[0046] Example 2

[0047] This embodiment provides, for example Figure 2 The image restoration system for historical artifacts based on deep learning is shown, specifically including: a physical attribute decoupling encoder module, a structural reasoning and completion module, a conditional content generation module, and an image synthesis decoder module;

[0048] Physical Attribute Decoupling Encoder Module: Based on a multi-material cultural relic image dataset, this module uses a multi-branch deep convolutional neural network to extract the geometric structure, surface material, and visual pattern features of the input image. By constraining the branch network through a correlation minimization loss function, it achieves decoupled representation of different physical attributes in the latent space. This module is designed to accurately separate the three core physical attribute features of geometric, material, and pattern in cultural relic images, avoiding mutual interference between different attribute information. It provides independent and pure feature inputs for subsequent modules such as structure completion and content generation, ensuring the targetedness and accuracy of each stage of processing from the source.

[0049] The structural reasoning and completion module receives decoupled geometric structural features and learns the topological relationships of the artifact image using a graph neural network. Based on a geometric prior rule base, it uses an attention mechanism to reason about the structure of missing regions. Through optimization using symmetry constraint loss and curvature continuity loss, it outputs a complete geometric structural representation. This module is designed to accurately restore the missing geometric structure of the artifact. Relying on the topological learning ability of the graph neural network and geometric prior rules, it ensures that the completed structure conforms to the inherent physical properties of the artifact (such as symmetry and curvature smoothness), avoiding unreasonable structures that violate the objective characteristics of the artifact, and providing an accurate structural framework for subsequent material and pattern generation.

[0050] Conditional Content Generation Module: Based on the completed geometric structure, a parallel conditional diffusion model is used. Based on the historical distribution of material and pattern features, cross-modal attention is used to generate material textures and decorative patterns. A multi-scale discriminator is used to ensure that the generated content is visually consistent with the artifact itself. This module is designed to ensure that the generated material textures and decorative patterns closely fit the completed geometric structure, while conforming to the historical style and visual characteristics of the artifact. The multi-scale discriminator ensures the visual coherence between the generated content and the artifact itself, avoiding the problems of content and structure being disconnected or style being misaligned, thereby improving the authenticity and naturalness of the restored content.

[0051] Image synthesis decoder module: stitches and fuses to complete the structure, generated material and pattern features; realizes feature-to-image reconstruction through U-Net architecture with skip connections; adopts joint optimization of perceptual loss and adversarial loss to output restoration results that meet the requirements of cultural relic protection. This module is designed to efficiently integrate the structural, material and pattern features processed in the early stage, reduce the loss of details in the reconstruction process by using U-Net architecture with skip connections, and ensure that the restored image is close to the real cultural relic in terms of semantics and visual effect through joint loss function optimization. Finally, it outputs a high-quality restored image with complete details, consistent style and in line with the standards of cultural relic protection.

[0052] In this embodiment, it should be specifically noted that the physical attribute decoupling encoder module is a three-branch parallel deep convolutional neural network, and the functions and structures of each branch are as follows:

[0053] Geometric structure feature extraction branch: Multi-layer convolutional network is constructed, specific padding maintains consistent resolution, ReLU activation captures geometric contours and key points, outputs geometric structure features, and encoder parameters are independently optimized;

[0054] The specific padding is that each convolution operation uses zero padding of size k divided by 2 and then rounded down, while the stride is set to 1 to ensure that the input feature map and the output feature map have the same resolution.

[0055] Considering resolution-preserving convolution operations with specific padding and ReLU activation, the geometric feature extraction process can be represented as:

[0056]

[0057] in, Indicates geometric structural features, Represents the linear rectified activation function. This indicates a convolution operation with specific padding. Indicates the first One input image of a cultural relic sample. This represents the independently learnable parameters of the geometric branch. This formula represents the kernel size and is used to extract geometric features from the input artifact image. It maintains resolution through convolution operations with specific padding, and combines ReLU activation to enhance nonlinear feature extraction capabilities, ultimately outputting clean geometric attribute features to provide a foundation for subsequent structural completion.

[0058] Surface material feature extraction branch: A multi-layer convolutional network is constructed, with batch normalization applied after each convolutional layer. An activation function that mitigates neuron death is used to capture material-related features, outputting surface material features. Encoder parameters are independently optimized. The formula for the surface material feature extraction branch can be expressed as:

[0059]

[0060] in, Indicates surface material characteristics, This represents a leaky ReLU activation function. This indicates a batch normalization operation. Indicates the basic convolution operation. Indicates the first Image sample of a cultural relic to be restored. The formula represents the independently learnable parameters of the material branch. It is used to extract surface material features from the input cultural relic image. Through batch normalization for stable training, combined with LeakyReLU activation to preserve subtle texture information, the final output is a pure material attribute feature, which provides a foundation for subsequent material texture generation.

[0061] Visual pattern feature extraction branch: A multi-layer convolutional network with dilated convolutions expands the receptive field. An adaptive activation function is used to capture pattern-related features and output visual pattern features. The encoder parameters are independent of the first two branches. The formula for the visual pattern feature extraction branch can be expressed as:

[0062]

[0063] in, Indicates visual pattern characteristics. This represents the adaptive activation function. Indicates the expansion rate Hollow convolution, Indicates the first Image sample of a cultural relic to be restored. The formula represents the independent learnable parameters of the pattern branch. It is used to extract visual pattern features from the input cultural relic image, expand the receptive field through dilated convolution to capture global pattern patterns, and combine Swish adaptive activation to enhance the expressive ability of complex patterns. Finally, it outputs pure pattern attribute features, providing a foundation for subsequent decorative pattern generation.

[0064] In the physical attribute decoupling encoder module, a correlation minimization loss function is introduced to constrain the multi-branch deep convolutional neural network, specifically as follows: the geometric structure features extracted by the three branches of the physical attribute decoupling encoder module are used respectively. Surface material characteristics Visual pattern features Based on this, the linear correlation between any two sets of features is measured by calculating the covariance matrix. The smaller the Frobenius norm of the covariance matrix, the better the feature decoupling effect. Its expression is:

[0065]

[0066] in, , For any two types of features, For the sample size, , They are respectively , The mean vector;

[0067] Weighting coefficients are assigned to the correlations between the three sets of feature combinations: geometric structure features and surface material features, geometric structure features and visual pattern features, and surface material features and visual pattern features. Through formula Ensure that the importance of the correlation of the three sets of features is balanced in the loss calculation;

[0068] The constraint objective of the correlation minimization loss function is to make the loss value... If the loss value does not meet the preset requirements during model training, the learnable parameters of each branch encoder of the three-branch deep convolutional neural network are adjusted through backpropagation algorithm, and iterative training is performed until the loss value meets the preset requirements, thereby achieving effective decoupling of the geometric structure features, surface material features, and visual pattern features in the latent space.

[0069] The structural reasoning and completion module takes the decoupled geometric structural features as its processing object and uses a graph attention network, as detailed below:

[0070] Input layer: This layer filters key structural features from geometric features. Key features include, but are not limited to, significant geometric features such as inflection points of the artifact's outline, centers of symmetry, and intersections of patterns. The output geometric structural features, containing the coordinates and feature vectors of the key points, are transformed into graph node features, forming graph structure data. The formula for constructing graph node features can be expressed as:

[0071]

[0072] in, Represents the set of graph nodes. This indicates the operation of filtering key structural points. This represents the geometric structural features output by the physical property decoupling encoder. The formula represents the constructed graph structure. It filters key points in the geometric structural features, transforms them into graph nodes, and constructs the graph structure. This provides a suitable data format for graph neural networks to learn the topological relationships of cultural relics and is a preliminary data processing step for structural reasoning and completion.

[0073] Attention Layer: A multi-head attention mechanism is constructed. First, the initial attention scores between nodes are calculated. Then, the scores are normalized to obtain the attention weights of neighboring nodes towards the center node. The formula for calculating the initial attention scores between nodes can be expressed as:

[0074]

[0075] in, Indicates the first Nodes in the head For nodes The original attention score, This indicates an activation function with leakage. This represents the attention vector of the h-th head. This represents the linear transformation matrix of the h-th head. Represents a node and nodes eigenvectors, Indicates feature concatenation operation;

[0076] Normalizing the raw scores, the formula for calculating the attention weights can be expressed as:

[0077]

[0078] in, Represented as the first Nodes in the head For nodes Normalized attention weights Represents a node The set of neighboring nodes, The formula for calculating attention weights, which represents an exponential function, can transform the original score into a probability distribution in the interval [0,1], clearly define the contribution ratio of each neighboring node to the central node, and ensure that the weights are interpretable and sum to 1.

[0079] The calculation formula for updating node features based on attention weights (multi-head integration) is as follows:

[0080]

[0081] Will The output features of each attention head are concatenated and then subjected to a linear transformation to unify the dimensions, resulting in the final updated node features:

[0082]

[0083] The attention layer formula uses a multi-head attention mechanism to calculate the association weights between nodes from multiple dimensions, and weightedly fuses neighborhood features to update node information, thereby achieving accurate learning of the topological relationships of the geometric structure of cultural relics and providing a basis for structural reasoning of missing areas.

[0084] Output layer: Linearly fused and weighted node features, outputting a geometric structure representation containing complete topological relationships;

[0085] The module has a built-in geometric prior rule base, which includes axisymmetric and centrally symmetric types, as well as curvature ranges and angle intervals based on historical artifact statistics. When reasoning about missing regions, it calculates the structural completion loss using the symmetry constraint loss formula and the curvature continuity loss formula, compares the preliminary prediction results with the geometric prior rule base, and selects reasonable results. The symmetry constraint loss formula is as follows:

[0086]

[0087] in, For a set of symmetric node pairs, The number of symmetric node pairs;

[0088] The formula for curvature continuity loss is:

[0089]

[0090] in, For curvature sampling points, This represents the number of curvature sampling points.

[0091] The structural reasoning and completion module introduces symmetry constraint loss and curvature continuity loss to optimize completion, specifically as follows: Symmetry constraint loss: Determine the set S of symmetric node pairs in the geometric structure representation, and use the formula... Calculate the loss value;

[0092] in, For symmetrical node pairs, , For the updated feature vector of the symmetric node, The number of symmetric node pairs. The value is the square of the Euclidean distance. The lower the loss value, the better the structural symmetry, which is used to ensure that the completed geometric structure conforms to the inherent symmetry properties of the cultural relic.

[0093] Curvature continuity loss: Select M curvature sampling points on the geometric contour of the artifact. Through formula Calculate the loss value;

[0094] in, Let M be the curvature value of the Mth sampling point. The value represents the curvature continuity loss. The lower the value, the smoother the curvature change of the artifact's outline and the better the continuity. M is the number of curvature sampling points, which needs to be set according to the complexity of the artifact's outline.

[0095] The lower the loss value, the better the continuity of the outline, which is used to ensure that the outline of the artifact is smooth and natural after completion.

[0096] The conditional content generation module outputs a complete geometric structure representation from the structure reasoning and completion module. To constrain this, a conditional diffusion model is used to generate material textures and visual patterns that fit the geometric shape and conform to historical style. Its core logic is "progressive denoising + geometric constraints + style alignment," specifically implemented as follows:

[0097] The conditional diffusion model generates content through an iterative process of "forward noise addition and reverse noise reduction". In the reverse noise reduction stage, geometric constraints are injected to ensure that the generated content conforms to the geometric shape of the cultural relics.

[0098] The forward process starts with real material / pattern features and gradually injects Gaussian noise to construct a degradation sequence of "clear → blurry", providing a learning target for inverse denoising.

[0099] Input: Real material features from a multi-material cultural relic dataset Such as the verdigris texture of bronzes, the glaze texture of ceramics, or the characteristics of real patterns. (such as the scrolling floral pattern on blue and white porcelain, and the auspicious animal pattern on bronze mirrors), the dimension is , Where C is the feature resolution and C is the number of channels.

[0100] Noise addition operation: for time steps ( (For 1000 steps), generate noisy features according to the following rules:

[0101]

[0102] in, , , It is standard Gaussian noise.

[0103] Output: The Noise features of the step Approximate pure noise is used as the initial input for inverse denoising.

[0104] The reverse process starts from pure noise and gradually recovers clear features through a denoising network, while simultaneously restoring the complete geometric structure. Inject attention modules at each time step to constrain the consistency between the generated content and the geometric shape.

[0105] Denoising network structure: The U-Net architecture is adopted, which contains 6 downsampling blocks (the number of channels increases from 64 to 1024) and 6 upsampling blocks (the number of channels decreases from 1024 to 64). Each upsampling block and the downsampling block at the same level fuse detailed features through skip connections.

[0106] Geometric constraint injection, in the attention module at each time step t, will As "conditional features" and noise-adding features The splicing process is as follows:

[0107] Will Mapped to the same value via a 1×1 convolution. Same number of channels;

[0108] In the self-attention module, geometric features are used as "positional codes" to participate in attention calculation, enabling the model to learn "what kind of material / pattern should be generated at the geometric contour" (e.g., the material wear is more severe at the corners, and the pattern at the contour line needs to be continuous).

[0109] Denoising Iteration: From Initially, at each step, a denoising network is used to predict noise and update features:

[0110]

[0111]

[0112] Output: Denoising features from step 0 The generated material / pattern features.

[0113] To ensure that the generated material textures adapt to the geometric shape and the decorative patterns are distributed along the geometric contour, the module introduces a cross-modal attention mechanism to quantify the correlation strength between geometric features and content features and dynamically adjust the generated results.

[0114] For example, the "raised areas" (high geometric feature values) of cultural relics are guided by attention weight to generate smoother material textures (such as the raised patterns on bronze artifacts having a brighter surface); "recessed areas" generate rougher textures (such as the fact that rust easily accumulates in recessed areas).

[0115] Pattern distribution along the contour: For example, the "bottle mouth contour line" in geometric features guides the pattern (such as the meander pattern) to be continuously distributed along the contour line through high attention weight, avoiding cross-contour breaks.

[0116] Based on the statistical analysis of the historical distribution of material / pattern features in multi-material cultural relics datasets, the distribution of generated features is constrained to be consistent with the historical distribution to avoid content that does not conform to the style of the era (such as generating Song Dynasty white glaze style from celadon glaze of Tang Dynasty ceramics).

[0117] The conditional content generation module ensures visual consistency of the generated content through its built-in multi-scale discriminator. The multi-scale discriminator employs a parallel, multi-branch architecture, evaluating the generated content at high, medium, and low resolutions. Each branch focuses on visual features at different scales, ensuring consistent evaluation covering local details, regional style, and overall artistic style.

[0118] High-resolution branch: The input is a high-resolution cropped block (such as a local texture area) of the generated content and an image of the real cultural relic. A deep convolutional network is used to capture the realism of local material details, such as the graininess of bronze rust and the gloss of ceramic glaze. This branch judges whether the generated content is consistent with the ontology at a microscale by perceiving differences in detail (such as texture clarity and color gradient).

[0119] Medium-resolution branch: The input is a downsampled medium-resolution image, focusing on the harmony of regional styles, such as the continuity of pattern layout and the naturalness of color transitions. This branch evaluates whether the generated content matches the regional features of the artifact itself (such as the symmetry of the patterns on the belly of the artifact).

[0120] Low-resolution branch: The input is a further downsampled low-resolution image, which controls the matching degree of the overall artistic style, such as the overall shape style of the cultural relic and the unique aesthetic characteristics of the historical period (such as the plump shape of Tang Dynasty ceramics and the elegant style of Song Dynasty porcelain). This branch ensures that the generated content does not deviate from the macro style tone of the cultural relic.

[0121] The architecture of the multi-scale discriminator is based on the concept of "scale-based fine-grained evaluation" to avoid the loss of details or style deviation that may be caused by single-resolution evaluation, thereby ensuring the visual consistency of the generated content.

[0122] The image synthesis decoder module employs a U-Net architecture with skip connections for image reconstruction, optimized through a joint loss function. The U-Net encoder-decoder structure fuses global features such as the completed geometry, generated materials, and patterns, while skip connections directly pass shallow detail information from the encoder to the decoder, effectively preserving key textures. The joint loss function combines perceptual loss, which focuses on high-level semantic structure matching, and adversarial loss, which aims to enhance the realism of the photograph, guiding the model to output a final restored image that is both structurally accurate and visually realistic.

[0123] It should be noted that the descriptions of each embodiment in the above embodiments have different focuses. For parts that are not described in detail in a certain embodiment, please refer to the relevant descriptions in other embodiments.

[0124] Those skilled in the art will understand that embodiments of the present invention can be provided as methods, systems, or computer program products. Therefore, the present invention can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention can take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0125] This invention is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded computer, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart illustrations. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0126] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0127] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0128] Although preferred embodiments of the invention have been described, those skilled in the art, upon learning the basic inventive concept, can make other changes and modifications to these embodiments. Therefore, the appended claims are intended to be interpreted as including both the preferred embodiments and all changes and modifications falling within the scope of the invention.

[0129] Obviously, those skilled in the art can make various modifications and variations to this invention without departing from its spirit and scope. Therefore, if these modifications and variations fall within the scope of the claims of this invention and their equivalents, this invention also intends to include these modifications and variations.

Claims

1. A deep learning-based system for the restoration of historical artifact images, characterized in that, Specifically, it includes: The module includes a physical property decoupling encoder, a structural reasoning and completion module, a conditional content generation module, and an image synthesis decoder module. Physical attribute decoupling encoder module: used to extract the geometric structure, surface material and visual pattern features of the input image through a multi-branch deep convolutional neural network, and realize the decoupling representation of physical attributes in the latent space by constraining the correlation minimization loss function; Structural Reasoning and Completion Module: This module receives the decoupled geometric structural features, infers the structure of the missing regions based on the geometric prior rule base, and optimizes it through symmetry constraint loss and curvature continuity loss to output a complete geometric structural representation. Conditional content generation module: Based on the completed geometric structure, it generates material textures and decorative patterns, and ensures that the generated content is visually consistent with the artifact itself. Image synthesis decoder module: used to fuse and complete the structure, generate material and pattern features, and output the repair result through feature reconstruction.

2. The system according to claim 1, characterized in that: The physical attribute decoupling encoder module is a three-branch parallel deep convolutional neural network, and the functions and structures of each branch are as follows: Geometric structure feature extraction branch: Multi-layer convolutional network is constructed, specific padding maintains consistent resolution, ReLU activation captures geometric contours and key points, outputs geometric structure features, and encoder parameters are independently optimized; Surface material feature extraction branch: Multi-layer convolutional network is constructed, and batch normalization is added after each convolution. Activation functions that alleviate neuron death are used to capture material-related features and output surface material features. Encoder parameters are optimized independently. Visual pattern feature extraction branch: a multi-layer convolutional network with dilated convolutions to expand the receptive field, using an adaptive activation function to capture pattern-related features and output visual pattern features. The encoder parameters are independent of the first two branches.

3. The system according to claim 2, characterized in that: In the physical attribute decoupling encoder module, a correlation minimization loss function is introduced to constrain the network, as detailed below: Based on the three types of features extracted by the three branches, the covariance matrix of the two types of features is calculated. The Frobenius norm measures the degree of linear correlation. The smaller the Frobenius norm value, the lower the linear correlation and the better the decoupling effect. Weight coefficients are set for the correlation of the three sets of features to ensure that the importance is balanced in the loss calculation. The constraint objective is to keep the loss value at a low level. If the loss does not meet the target during training, the branch encoder parameters are adjusted through backpropagation and iterated until the loss meets the requirements, thus achieving effective decoupling of features.

4. The system according to claim 3, characterized in that: The structural reasoning and completion module takes the decoupled geometric structural features as its processing object and uses a graph attention network, as detailed below: Input layer: Selects key structural features from geometric features, transforms them into graph node features, and forms graph structure data; Attention layer: The multi-head attention mechanism is constructed by first calculating the original attention scores between nodes, and then normalizing the scores to obtain the attention weights of neighboring nodes to the central node. Output layer: Linearly fused and weighted node features, outputting a geometric structure representation containing complete topological relationships; The module has a built-in geometric prior rule base. When reasoning about missing regions, it compares the preliminary prediction results with the rule base and selects reasonable results.

5. The system according to claim 4, characterized in that: The structural reasoning and completion module introduces symmetry constraint loss and curvature continuity loss to optimize completion, as detailed below: Symmetry constraint loss: Identify symmetrical node pairs in the geometric structure, calculate the feature differences between node pairs and sum them up and average them. The lower the loss value, the better the symmetry, ensuring that the structure conforms to the inherent symmetry properties. Curvature continuity loss: Select curvature sampling points on the outline of the cultural relic, calculate the absolute value of the curvature difference between adjacent sampling points and sum them up and average them. The lower the loss value, the better the continuity, ensuring a smooth and natural outline. The two losses are weighted to construct the total loss. The graph neural network parameters are adjusted through backpropagation and iterated until the total loss reaches the target, outputting a high-quality geometric structure representation.

6. The system according to claim 5, characterized in that: The conditional content generation module adopts a conditional diffusion model, injects attention modules with geometric structures as conditions, and aligns material and pattern features through cross-modal attention to ensure that the generated content conforms to the geometric shape and historical style.

7. The system according to claim 6, characterized in that: The conditional content generation module includes A multi-scale discriminator is used to assess the visual consistency between the generated content and the artifact itself.

8. The system according to claim 7, characterized in that: The image synthesis decoder module fuses features and reconstructs images using a U-Net architecture with skip connections, and jointly optimizes the output using perceptual loss and adversarial loss.

9. A processing method based on the system according to any one of claims 1-8, characterized in that, Includes the following steps: Step S1, Decoupling and Extraction of Physical Attribute Features: The physical attribute decoupling encoder module is called to extract the geometric structure, surface material and visual pattern features of the input image through a three-branch parallel deep convolutional neural network; the correlation minimization loss function is introduced to constrain the network, and the branch encoder parameters are adjusted through backpropagation to achieve the decoupling representation of physical attributes in the latent space; Step S2, Geometric Structure Reasoning and Completion: Call the structure reasoning and completion module, take the decoupled geometric structure features as input, and use a graph attention network to learn the topological relationships of the cultural relic image; Based on the built-in geometric prior rule base, the missing region structure is inferred, and the completion result is optimized by symmetry constraint loss and curvature continuity loss, outputting a complete geometric structure representation. Step S3, Conditional Material and Pattern Generation: Call the conditional content generation module, use the completed geometric structure as a condition, and use the conditional diffusion model to generate material textures and decorative patterns; align the material, pattern and geometric features through a cross-modal attention mechanism, and use a multi-scale discriminator to evaluate the visual consistency between the generated content and the artifact itself. Step S4, Image Synthesis and Restoration Output: Call the image synthesis decoder module to stitch and fuse the completed geometric structure, generated material texture and visual pattern features; realize feature-to-image reconstruction through the U-Net architecture with skip connections, and use perceptual loss and adversarial loss for joint optimization to output the final cultural relic image restoration result.

Images