A method and system for digitally repairing a damaged area of a calligraphy or painting work

Abstract

This invention discloses a digital restoration method and system for damaged areas of calligraphy and painting works, relating to the field of image processing technology. The method includes: acquiring visible light images and multispectral auxiliary spectral band images of the calligraphy and painting, performing geometric correction, color consistency correction, and separation and alignment of the ink layer and the paper / silk substrate texture layer to generate a reference image package containing layer alignment relationships; performing damaged area segmentation and damaged type labeling on the reference image package, extracting the brushstroke direction field, and generating a damaged semantic map; using the damaged semantic map to conduct brushstroke structure-guided generative adversarial restoration and implementing style consistency preservation processing, while simultaneously performing multispectral consistency verification, and completing substrate texture resynthesis according to layer alignment relationships to generate a candidate restoration image set. This invention reduces the risk of visually reasonable but materially inconsistent results caused by relying solely on visible light by recording and constraining the spectral response consistency of the restoration area through multispectral consistency verification and spectral band verification.

Description

Technical Field

[0001] This invention relates to the field of image processing technology, and in particular to a method and system for digitally restoring damaged areas of calligraphy and painting works. Background Technology

[0002] With the development of digital cultural relic preservation and computer vision technology, the digital restoration of calligraphy and painting works has gradually expanded from single visible light scanning to joint acquisition of visible light and multispectral imaging, and is combined with lens distortion correction, perspective registration, scale matching and white balance consistency to obtain high-quality image data in a unified coordinate and color space. On this basis, commonly used methods for missing area segmentation, type labeling, texture reconstruction and image inpainting have become increasingly mature, and algorithms such as generative restoration and style transfer have also been used to improve the visual continuity and artistic consistency of the restored area.

[0003] However, existing methods are still prone to the accumulation of cross-spectral alignment errors when dealing with the characteristics of real calligraphy and painting materials. Coupling between the ink layer and the texture layer of the paper and silk substrate leads to texture backfilling distortion. Differences in the type of defects are difficult to form controllable constraints during the reconstruction process. Furthermore, the restoration results may appear "reasonable" in the visible light but are inconsistent with the multispectral response. At the same time, the differences between multiple versions of restoration results lack quantitative comparison and traceable encapsulation of defects and texture details, making it difficult to objectively recommend, reproduce, and review restoration solutions. Summary of the Invention

[0004] In view of the aforementioned existing problems, the present invention is proposed.

[0005] Therefore, this invention provides a digital restoration method for damaged areas of calligraphy and painting works, which solves the problems of difficulty in achieving unified implementation of cross-spectral alignment, consistent layering of ink and base material, and optimal traceability schemes in the restoration of damaged calligraphy and painting works.

[0006] To solve the above-mentioned technical problems, the present invention provides the following technical solution: In a first aspect, the present invention provides a method for digitally restoring damaged areas of calligraphy and painting works, comprising, The system acquires visible light images and multispectral auxiliary spectral band images of calligraphy and paintings, performs geometric correction, color consistency correction, and separation and alignment of the ink layer and the paper / silk substrate texture layer, and generates a reference image package containing layer alignment relationships. The missing region segmentation and missing type labeling are performed on the reference image package, and the pen stroke direction field is extracted to generate a missing semantic map; We utilize the missing semantic map to carry out generative adversarial repair guided by brush stroke structure and implement style consistency maintenance processing. At the same time, we perform multispectral consistency verification and complete the re-synthesis of substrate texture according to the layer alignment relationship to generate a candidate repair image set. Perform multi-version alignment comparison on the candidate image set for restoration, associate consistency evaluation information, generate the optimal restoration scheme recommendation, and package it into a digital restoration result package.

[0007] As a preferred embodiment of the digital restoration method for damaged areas of calligraphy and painting works according to the present invention, the specific steps for completing geometric correction and color consistency correction are as follows. The system reads the shooting parameters and imaging calibration information of the visible light image and the multispectral auxiliary spectral band image, performs lens distortion correction and viewing angle difference correction on the visible light image and the multispectral auxiliary spectral band image, and generates a unified viewing angle image sequence. In a unified viewpoint image sequence, the boundaries of calligraphy and painting are located and reference marks are established, and a coordinate mapping relationship is generated to generate a unified coordinate reference. By using a unified coordinate reference, translation, rotation registration, and scale matching are performed on a unified viewpoint image sequence to generate a geometrically aligned image sequence. Extract the background reference region of the geometrically aligned image sequence and construct color reference description information to generate a color reference model; Based on the color reference model, white balance and tone mapping are performed on the geometrically aligned image sequence to generate a color-consistent image sequence, which is then encapsulated with a unified coordinate reference to form a corrected image set.

[0008] As a preferred embodiment of the digital restoration method for damaged areas of calligraphy and painting works according to the present invention, the separation and alignment of the ink layer and the paper / silk substrate texture layer are carried out through the following specific steps. Based on the corrected image set, the brush and ink response features and substrate texture response features of visible light images are extracted, and brush and ink layer estimation is performed on the calligraphy and painting images to generate brush and ink layer images. At the same time, substrate texture modeling is performed on the calligraphy and painting images to generate paper and silk substrate texture layer images. Pixel-level registration is performed on the ink layer image and the paper / silk substrate texture layer image, and the corresponding pixel mapping relationship is recorded to form a layer alignment relationship; The ink layer image, the paper / silk substrate texture layer image, and the layer alignment relationship are bound and registered to generate a reference image package.

[0009] As a preferred embodiment of the digital restoration method for damaged areas of calligraphy and painting works according to the present invention, the specific steps for generating the semantic map of the damage are as follows: Retrieve the ink layer image and layer alignment relationship from the reference image package, perform defect region segmentation, obtain the defect mask, and locate the defect boundary; Perform defect type discrimination on the area covered by the defect mask to obtain a defect type label map, and extract the stroke edges and stroke directions for the area not covered by the defect mask to obtain a stroke skeleton set; Directional statistics are performed on the stroke skeleton set to form a stroke direction field, which is then combined with the defect mask and defect type label map to generate a defect semantic map.

[0010] As a preferred embodiment of the digital restoration method for damaged areas of calligraphy and painting works according to the present invention, the specific steps of using the semantic map of the damage to perform brushstroke structure-guided generative adversarial restoration and implement style consistency maintenance processing are as follows. Extract continuous segments of the stroke direction field at the defect boundary in the defect semantic map to form a set of boundary stroke guidance segments, and jointly encode them with the defect type label map to form a generative constraint description; Based on the generative constraint description, the configuration structure guide channel is configured and generative adversarial repair is performed to obtain the ink reconstruction result that extends continuously along the stroke direction field; The ink style feature description is extracted from the ink reconstruction result, and the original style feature description is extracted from the area not covered by the missing mask to generate a style alignment description. Based on the style alignment description, the ink tones and brushstroke textures of the restored area are kept consistent with the original area to generate a style-preserving restoration result. The style-preserving repair results, style alignment descriptions, and generation constraint descriptions are merged to obtain the generation adversarial repair record.

[0011] As a preferred embodiment of the digital restoration method for damaged areas of calligraphy and painting works according to the present invention, the specific steps for performing multispectral consistency verification are as follows: Extract the location index of the repaired region from the style-preserving repair result to form a repaired region index table, and locate the corresponding spectral region in the multispectral auxiliary spectral region image of the reference image package to form a spectral alignment region group; Extract spectral response descriptions of faded and missing regions from the spectral alignment region group to generate a spectral consistency reference description; Extract texture and brightness descriptions related to spectral response from the style-preserving restoration results to generate visible light consistency descriptions; The spectral band consistency reference description is compared with the visible light consistency description to obtain the spectral band consistency verification conclusion, and combined with the repair area index table to generate spectral band verification records. Based on the spectral consistency verification results, the style preservation repair results are corrected, and a verification-passed repair result is generated.

[0012] As a preferred embodiment of the digital restoration method for damaged areas of calligraphy and painting works according to the present invention, the specific steps for re-synthesizing the base texture according to the layer alignment relationship are as follows: The verified repair results are combined with the paper and silk substrate texture layer image in the baseline image package to obtain the recombined material group; Retrieve the layer alignment relationship and establish a mapping table between the pixels of the verified repair result and the pixels of the paper and silk substrate texture layer, and generate a recombination mapping table; Based on the resynthesis mapping table, the texture details of the paper and silk substrate texture layer image are backfilled to the corresponding positions of the verified repair results. Texture transition fusion processing is performed at the defect boundary while maintaining the continuity of substrate particles to generate a consistent substrate repair result. The substrate-consistent repair results are encapsulated with a resynthesis mapping table to obtain a candidate repair image set.

[0013] As a preferred embodiment of the digital restoration method for damaged areas of calligraphy and painting works according to the present invention, the specific steps of performing multi-version alignment comparison on the candidate restoration image set and associating consistency evaluation information are as follows: Extract a unified coordinate reference for each candidate version in the candidate image set for restoration, and perform image registration on each candidate version to generate an aligned candidate version group; Perform difference localization on the aligned candidate version group, generate a version difference localization map, and associate and merge it with the consistency assessment information to generate version comparison results; The generated adversarial repair records, spectral verification records, resynthesis mapping tables and version comparison results are summarized and packaged to generate consistency assessment information. Based on the version comparison results and consistency assessment information, the optimal remediation solution is recommended and a recommended version identifier is generated; The candidate versions corresponding to the recommended version identifier, the defect semantic map, the hierarchical alignment relationship, the version comparison results and the consistency assessment information are archived to generate a digital repair result package.

[0014] As a preferred embodiment of the digital restoration method for damaged areas of calligraphy and painting works described in this invention, the difference localization refers to calculating pixel differences and extracting edge gradient differences and texture-related differences for the aligned candidate version group according to the damaged boundary area marked by the damaged mask.

[0015] Secondly, the present invention provides a digital restoration system for damaged areas of calligraphy and painting works, including an acquisition and correction module, which acquires visible light images and multispectral auxiliary spectrum images of calligraphy and painting works and completes geometric correction, color consistency correction and separation and alignment of ink layer and paper / silk substrate texture layer, and generates a reference image package containing layer alignment relationship. The defect semantic module performs defect region segmentation and defect type labeling on the baseline image package, and extracts the pen stroke direction field to generate a defect semantic map; The generation and repair module utilizes the defect semantic map to carry out brushstroke structure-guided generative adversarial repair and implements style consistency maintenance processing. At the same time, it performs multispectral consistency verification and completes the re-synthesis of the substrate texture according to the layer alignment relationship, generating a candidate repair image set and consistency evaluation information. The version encapsulation module performs multi-version alignment comparison on the candidate image set for restoration, associates consistency evaluation information, generates the optimal restoration scheme recommendation, and encapsulates it into a digital restoration result package.

[0016] The beneficial effects of this invention are as follows: By using geometric correction, color consistency correction, and the establishment of a unified coordinate benchmark through visible light images and multispectral auxiliary spectral band images, subsequent defect detection, spectral band verification, and version alignment have a consistent spatial reference, reducing the propagation of errors across spectral bands and versions; by estimating the ink layer and modeling the paper / silk substrate texture layer and forming a layered alignment relationship, the repair process can maintain the continuity of brushstroke shape, ink color level, and substrate particles, avoiding repair artifacts caused by mistaking substrate texture for brushstroke structure; by combining defect masks, defect type label maps, and brushstroke direction fields to form a defect semantic map, and using generative constraint descriptions to guide the generation of adversarial repair, the reconstruction of damage forms such as missing parts, cracks, fading, and stains is differentiated and focused while maintaining the continuity of brushstroke direction at the defect boundary; by recording the consistency of spectral response in the constrained repair area through multispectral consistency verification and spectral band verification, the risk of visual reasonableness but inconsistent material response caused by relying solely on visible light is reduced. Attached Figure Description

[0017] To more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings used in the following description of the embodiments will be briefly introduced. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0018] Figure 1 A flowchart illustrating the digital restoration method for damaged areas in calligraphy and painting works.

[0019] Figure 2 This is a flowchart for geometric correction and color consistency correction.

[0020] Figure 3 This is a flowchart of the training process for the defect detection and classification model.

[0021] Figure 4 This is a flowchart for multispectral consistency verification. Detailed Implementation

[0022] To make the above-mentioned objects, features and advantages of the present invention more apparent and understandable, the specific embodiments of the present invention will be described in detail below with reference to the accompanying drawings.

[0023] Many specific details are set forth in the following description in order to provide a full understanding of the invention. However, the invention may also be practiced in other ways different from those described herein, and those skilled in the art can make similar extensions without departing from the spirit of the invention. Therefore, the invention is not limited to the specific embodiments disclosed below.

[0024] Secondly, the term "one embodiment" or "embodiment" as used herein refers to a specific feature, structure, or characteristic that may be included in at least one implementation of the present invention. The phrase "in one embodiment" appearing in different places in this specification does not necessarily refer to the same embodiment, nor is it a single or selective embodiment that is mutually exclusive with other embodiments.

[0025] Reference Figures 1-4 As an embodiment of the present invention, this embodiment provides a method for digitally restoring damaged areas of calligraphy and painting works, including the following steps: S1. Acquire visible light images and multispectral auxiliary spectrum images of calligraphy and paintings, and complete geometric correction, color consistency correction, and separation and alignment of the ink layer and the paper / silk substrate texture layer to generate a reference image package containing layer alignment relationships.

[0026] S1.1. Perform high-resolution scanning or multispectral imaging to acquire the original image sequence of the visible light image and the multispectral auxiliary spectrum image, and simultaneously record the shooting parameters and imaging calibration information such as lens focal length, principal point position, distortion parameters, imaging distance, imaging angle, resolution, exposure and light source color temperature. The shooting parameters and imaging calibration information are then output as a set of shooting parameters and imaging calibration information after establishing a one-to-one correspondence between the shooting parameters and imaging calibration information and the original image sequence.

[0027] In this embodiment, the visible light image is acquired using a high-resolution area array color industrial camera, and the multispectral auxiliary spectral band image is acquired using an 8-channel multispectral imaging device. The multispectral auxiliary spectral band image includes eight auxiliary spectral bands from B1 to B8.

[0028] Based on the shooting parameters and imaging calibration information set, lens distortion correction is performed on the visible light image and the multispectral auxiliary spectral band image to eliminate radial and tangential distortion, respectively. Then, perspective difference correction is performed to eliminate perspective differences caused by different imaging angles, resulting in visible light images and multispectral auxiliary spectral band images with consistent perspectives. These images are then organized into a unified perspective image sequence according to time and spectral band order.

[0029] S1.2. Locate the boundaries of calligraphy and painting in the unified view image sequence and extract the boundary contour point set. At the same time, locate the reference marker in the unified view image sequence and extract the reference marker feature point set. Establish a coordinate mapping relationship between the boundary contour point set and the reference marker feature point set, and output a unified coordinate reference. The unified coordinate reference is represented by a planar coordinate system, so that the position of each pixel in the unified view image sequence is mapped to the coordinate position in the unified coordinate reference.

[0030] Based on a unified coordinate reference, a translation and rotation registration is performed on a unified viewpoint image sequence. In the unified viewpoint image sequence, corresponding feature point pairs are selected from the reference marker feature point set and a matching point pair set is established. Then, the translation amount and rotation angle of mapping the unified viewpoint image sequence to the unified coordinate reference are calculated based on the matching point pair set. Subsequently, translation and rotation transformations are applied to the unified viewpoint image sequence frame by frame and bilinear interpolation is used for resampling, so that the visible light image and the multispectral auxiliary band image are aligned at the pixel level under the unified coordinate reference.

[0031] When performing scale matching after translation and rotation registration, the spatial scaling ratio is first calculated based on the actual spacing of the reference marker feature point set under the unified coordinate reference and the pixel spacing in the unified view image sequence. Then, the unified view image sequence that has completed translation and rotation registration is scaled and resampled proportionally according to the spatial scaling ratio. The scaled image is then cropped or filled to the image size corresponding to the unified coordinate reference, so that the unified view image sequence achieves a consistent spatial resolution and image size under the unified coordinate reference, and a geometrically aligned image sequence is output.

[0032] In the geometrically aligned image sequence, the area outside the boundaries of calligraphy and painting or the blank area of ​​paper and silk is selected as the background reference area. The pixels of the background reference area are statistically analyzed according to color channels to obtain the channel mean and channel variance. The channel mean and channel variance are then correlated with the spectral band labels of the visible light image and the multispectral auxiliary spectral band image to form a color reference model for characterizing the color benchmark of the background reference area. The color reference model includes the channel mean set, the channel variance set and the mapping relationship of the spectral band label.

[0033] S1.3. Based on the color reference model, perform white balance unification on the geometrically aligned image sequence to eliminate the color temperature difference of the light source, and then perform tone mapping unification to unify the brightness and darkness levels and color response curves, and output a color-consistent image sequence; encapsulate the color-consistent image sequence and the unified coordinate reference together to form a calibration image set.

[0034] Based on the corrected image set, the ink response features and substrate texture response features of visible light images are extracted. The ink response features include gray-scale change features and edge gradient features related to the edge of the brushstroke and the level of ink density. The substrate texture response features include texture statistics features and frequency domain texture features related to the direction of paper and silk fibers and grain texture. The ink response features and substrate texture response features are aligned by pixel position and summarized to output a separate feature set.

[0035] Furthermore, based on the multispectral auxiliary spectral band image, the spectral intensity sequence of each pixel under the B1 to B8 auxiliary spectral bands is extracted, and the intensity difference between adjacent spectral bands, the spectral mean, and the normalized spectral band ratio are calculated to obtain the spectral response features. The brush and ink response features, the substrate texture response features, and the spectral response features are jointly analyzed to divide the brush and ink dominant region, the substrate dominant region, and the coupling transition region in the calligraphy and painting image. Among them, the coupling transition region refers to the region where the brush and ink response and the substrate texture response are both strong and the spectral response fluctuation exceeds the preset fluctuation threshold. The preset fluctuation threshold is preferably 0.08, which is set according to the 95th percentile value of the spectral fluctuation of the non-defective substrate region in the training sample. This value can distinguish the natural fluctuation of the substrate and the coupling fluctuation of the brush and ink-substrate.

[0036] S1.4. Based on the separation feature set, perform brush and ink layer estimation on the calligraphy and painting image to obtain a brush and ink layer image that retains only the brushstroke shape and ink color layer information; based on the separation feature set, perform substrate texture modeling on the calligraphy and painting image to obtain a paper and silk substrate texture layer image that retains only the fiber texture and particle texture information.

[0037] It should be noted that the ink layer estimation is performed as follows: edge gradient features and grayscale change features corresponding to the ink response features are selected from the separation feature set to form an ink response map. Morphological connected component filtering is performed on the ink response map to retain continuous brushstroke areas. Then, edge preservation filtering is performed on the retained continuous brushstroke areas to suppress substrate texture interference, resulting in an ink layer image with preserved brushstroke shape. At the same time, the pixel value range of the ink layer image is kept consistent with the calibration image set to retain the ink density level information.

[0038] For the coupling transition region, local weight decomposition is performed by combining the gray-scale change characteristics, edge gradient characteristics, and spectral response characteristics of similar strokes in adjacent non-defective regions. When the residual coefficient of the partial solution exceeds the preset residual threshold, the corresponding region is registered as a high-coupling retention region, and only constraint correction is implemented in the subsequent repair and resynthesis stages without forced complete separation. The preset residual threshold is preferably 0.10, which is set according to the balance point between the high-coupling region identification accuracy and the false retention rate on the validation set. This value can take into account both the retention of real high-coupling regions and the separation of normal regions.

[0039] The execution method of substrate texture modeling is as follows: Select the texture statistical features and frequency domain texture features corresponding to the substrate texture response features in the separation feature set to form a substrate texture response map, and perform bandpass frequency domain filtering on the substrate texture response map to highlight fiber texture and particle texture. Then, perform smoothing and noise reduction on the filtering result to suppress the residual response of brush stroke edges, and obtain a paper and silk substrate texture layer image with continuous texture. At the same time, keep the paper and silk substrate texture layer image pixel-level aligned with a unified coordinate reference.

[0040] Perform pixel-level registration on the ink layer image and the paper / silk substrate texture layer image to ensure that the pixel positions of the ink layer image and the pixel positions of the paper / silk substrate texture layer image correspond point by point under a unified coordinate reference, record the corresponding pixel mapping relationship, and output the layer alignment relationship.

[0041] The ink layer image, paper / silk substrate texture layer image, and layer alignment relationship are bound and registered. The spectral segment identifiers of the calibration image set and the multispectral auxiliary spectral segment image are registered together under the same unified coordinate reference and encapsulated to generate a reference image package containing the layer alignment relationship.

[0042] S2. Perform defect region segmentation and defect type labeling on the reference image package, and extract the pen stroke direction field to generate a defect semantic map.

[0043] S2.1. Read the ink layer image, paper / silk substrate texture layer image, multispectral auxiliary spectral band image, unified coordinate reference, and layer alignment relationship from the reference image package. Organize the ink layer image and multispectral auxiliary spectral band image in situ according to the unified coordinate reference to form a defect analysis image group. The defect analysis image group is formed by sequentially stitching together 1 ink layer image and 8 multispectral auxiliary spectral band images.

[0044] The defect analysis image set is fed into the defect detection and classification model to obtain pixel-level defect region segmentation results; the pixel-level defect region segmentation results are subjected to connected component normalization to eliminate isolated fragments and maintain the continuity of the defect region, and a defect mask is output; the boundary pixel set is extracted along the region contour of the defect mask and the defect boundary is output.

[0045] It should be noted that the construction steps of the defect detection and classification model are as follows: The defect analysis image group is organized into a multi-channel input tensor. The channels of the input tensor are formed by sequentially stitching together the spectral band images of the ink layer image and the multispectral auxiliary spectral band image. An encoder network structure is set up, which is formed by cascading multiple levels of convolutional layers and downsampling layers to extract multi-scale texture and structural features. A decoder network structure is set up, which is formed by cascading multiple levels of upsampling layers and convolutional layers, and is connected to the feature maps of the corresponding encoder layers through skip connections to restore pixel-level spatial resolution. At the end of the decoder, a defect segmentation output head and a defect type output head are set up. The defect segmentation output head outputs a pixel-level defect probability map, and the defect type output head outputs a pixel-level damage morphology category response map. The pixel-level damage morphology category response map covers the category labels of stains, fading, missing parts, and cracks.

[0046] The training steps for the defect detection and classification model are as follows: First, collect and organize the calligraphy and painting image data for training. This data is processed according to S1 geometric correction, color consistency correction, and unified coordinate reference to form a training correction image set. Then, according to S1 ink layer estimation and substrate texture modeling, training ink layer images and training multispectral auxiliary spectral band images are obtained. Defect mask annotation and defect type annotation are performed on the training correction image set to form training defect masks and training defect type label maps. The training ink layer images and training multispectral auxiliary spectral band images are organized co-located according to a unified coordinate reference to form a training defect analysis image group. A pixel-level correspondence is established between the training defect mask and training defect type label map and the training defect analysis image group. The training defect analysis image group is input using a mini-batch iterative method. Pixel-level defect segmentation loss and pixel-level defect type classification loss are calculated respectively, and backpropagation is performed to update the network parameters. During training, pixel-level defect region segmentation results and defect type label maps are output round by round on the validation set, and validation segmentation and validation classification metrics are calculated. The validation segmentation metric uses the intersection-union ratio (the ratio of the intersection area of ​​the defect mask prediction region and the union area of ​​the training defect mask, used to measure the consistency between the pixel-level defect region segmentation results and the training defect mask). The validation classification metric uses the category accuracy (the ratio of the number of pixels correctly assigned to the damage morphology category in the defect type label map to the number of pixels covered by the defect mask, used to measure the accuracy of the defect type labeling). When the change in the validation segmentation and validation classification metrics is less than the preset change threshold and the validation segmentation loss and validation classification loss no longer decrease within several consecutive rounds (e.g., 10 rounds), the defect detection and classification model is determined to have reached the training completion state. The network parameters corresponding to the training completion state are used as the inference parameters of the defect detection and classification model.

[0047] In this embodiment, a total of 5,600 sets of calligraphy and painting image data are used for training, including 4,200 sets of training data, 700 sets of validation data, and 700 sets of test data.

[0048] The inference parameters of the defect detection and classification model are used to perform forward inference on the defect analysis image group to obtain a pixel-level defect probability map and a pixel-level damage morphology category response map. The pixel-level defect probability map is selected according to the pixel position to form a pixel-level defect region segmentation result. The pixel-level defect region segmentation result is then subjected to connected component normalization to eliminate isolated fragments and maintain the continuity of the defect region, and a defect mask is output. The boundary pixel set is extracted along the region contour of the defect mask and the defect boundary is output.

[0049] To further explain, the preset change threshold can be set as the range in which the changes in the validation segmentation index and the validation classification index in the most recent 10 rounds do not exceed a stable range of historical fluctuations (e.g., not exceeding the median level of the changes in the most recent 50 rounds), specifically based on the natural fluctuation range of the validation set index.

[0050] S2.2. Within the coverage area of ​​the defect mask, each defect pixel is assigned a damage morphology category based on the defect type output of the defect detection and classification model. The defect type label map covers the category labels of dirt, fading, missing and crack and is output in alignment with a unified coordinate reference. The boundary consistency of the defect type label map is preserved in the neighborhood of the defect boundary to ensure the continuity of the labels on both sides of the defect boundary.

[0051] Within the area not covered by the defective mask, the brushstroke edges are extracted based on the ink layer image, and the brushstroke direction is tracked to form a set of brushstroke direction line segments. The set of brushstroke direction line segments is then refined. During the refinement process, the set of brushstroke direction line segments is first rasterized into a brushstroke direction binary map. Then, under the eight-neighborhood connectivity constraint, the boundary pixels of the brushstroke direction binary map are iteratively deleted. The boundary pixel deletion condition simultaneously satisfies connectivity preservation, endpoint preservation, and no change in line segment topology. The iteration terminates when there are no boundary pixels that satisfy the deletion condition. After the iteration terminates, a brushstroke skeleton set with a single pixel width is obtained.

[0052] For local skeleton regions containing bifurcation points, intersection points, or overlapping strokes, pixels with a skeleton degree greater than 2 are detected as complex nodes, and the stroke skeleton set is divided into multiple skeleton sub-segments with complex nodes as the boundary. Among them, the continuous length of the skeleton sub-segment participating in the main direction determination must not be less than a preset length threshold, which is 12 pixels, based on the minimum stable length of the real continuous stroke segment under common scanning resolution.

[0053] It should be noted that brush stroke edge extraction can be expressed using gradient magnitude, specifically as follows: ; in, The gradient magnitude represents the pixel location and is used to highlight the stroke edge response and support the tracking of the stroke's line segment set. This represents the gradient component of the ink layer image in the horizontal direction. This represents the gradient component of the ink layer image in the vertical direction.

[0054] S2.3. Construct local direction vectors for adjacent skeleton points in the stroke skeleton set and perform direction statistics to obtain a stroke direction field that is in the same position as the unified coordinate reference. The stroke direction field describes the stroke direction with the direction angle of each skeleton point.

[0055] For multiple skeleton segments obtained from complex node decomposition, priority is given to selecting skeleton segments that are directly adjacent to the defect boundary and have a continuous length greater than a preset length threshold for participation in the main direction statistics. For multiple candidate directions within the same local neighborhood, a continuity score is calculated based on grayscale continuity, length continuity, and smoothness of direction change. Specifically, the following steps are taken: first, compare whether the light and dark transitions in the corresponding ink areas on both sides of the defect boundary are natural and whether the texture connections are coherent to form a grayscale continuity evaluation; then, compare the effective extension length of the skeleton segments corresponding to each candidate direction and the length connection of the retained brushstrokes within the boundary's adjacent range. The system first evaluates the continuity of length by comparing the smoothness of the transitions of each candidate direction in the local neighborhood and whether they are consistent with the existing stroke directions near the boundary. This evaluates the smoothness of the direction change. Finally, the three evaluation results are unified and normalized to form a continuity score, which is used to determine the main direction. When the continuity score is lower than a preset continuity threshold, the corresponding area is marked as a low-confidence direction area. The preset continuity threshold is 0.65, which is set based on the inflection point with the highest accuracy of main direction determination on the validation set. This value can balance the stability of cross-stroke discrimination and the sufficiency of direction preservation.

[0056] The direction angle can be expressed using the arctangent, specifically: ; in, It represents the orientation angle of the skeleton point, used to characterize the direction of the brushstroke and support the spatial distribution of the brushstroke orientation field. This represents the lateral displacement of adjacent skeleton points under a unified coordinate datum. It represents the longitudinal displacement of adjacent skeleton points under a unified coordinate datum.

[0057] The defect mask and defect type label map are combined and encoded to form a conditional repair description. The conditional repair description carries both defect presence and defect category information at the pixel location and is aligned with a unified coordinate reference. The defect mask, defect type label map, stroke direction field and conditional repair description are combined and encapsulated to output a defect semantic map containing the defect mask, defect type label map, stroke direction field and conditional repair description. The defect semantic map maintains a unified coordinate reference and layer alignment with the reference image package.

[0058] Conditional inpainting descriptions can employ a joint description of pixel locations, specifically: ; in, Indicates the pixel position under a unified coordinate system. Conditional repair descriptions representing pixel locations are used to provide distinguishable conditional information for different defect types in subsequent processing stages. Indicates the location of the defective mask at the pixel position. The value of is commonly 0 or 1, where 0 represents no defect and 1 represents a defect. The defect type label map is located at the pixel position. Category labeling, Indicates the pixel position Defect type labeling The vector obtained by one-hot encoding has only one position with a value of 1, and the rest with a value of 0; the position with a value of 1 corresponds to... The indicated defect type category is used to convert the category information of the defect type label map into concatenable and comparable vector components.

[0059] S3. Utilize the missing semantic map to carry out adversarial restoration guided by brushstroke structure and implement style consistency maintenance processing. At the same time, perform multispectral consistency verification and complete the re-synthesis of substrate texture according to the layer alignment relationship to generate a candidate restoration image set.

[0060] S3.1. Read the defect mask, defect type label map, brush stroke direction field, defect boundary and conditional repair description from the defect semantic map, and read the ink layer image, multispectral auxiliary spectral band image, paper and silk substrate texture layer image, unified coordinate reference and layer alignment relationship from the reference image package. Complete the spatial co-position check between the ink layer image and the defect mask, and the spectral co-position check between the multispectral auxiliary spectral band image and the defect mask. Output the repair preparation data set consistent with the unified coordinate reference.

[0061] In the repair preparation data set, continuous directional segments of the brushstroke direction field are searched along the defect boundary, and brushstroke context segments on both sides of the defect boundary are extracted to form a set of boundary brushstroke guidance segments. The set of boundary brushstroke guidance segments and the defect type label map are jointly encoded pixel by pixel and aligned and merged with the conditional repair description to obtain the generated constraint description. The generated constraint description provides brushstroke extension direction constraints and damage morphology category constraints for the defect mask coverage area under a unified coordinate reference.

[0062] S3.2. Based on the generative constraint description, a structure guidance channel is established in generative adversarial repair. The structure guidance channel is formed by splicing the brush stroke direction field and the generative constraint description in the channel order. The structure guidance channel is used to constrain the generation path of the area covered by the defect mask and maintain the continuity of the brush stroke direction at the defect boundary. The output is the ink reconstruction result that extends continuously along the brush stroke direction field. The ink reconstruction result and the defect boundary neighborhood of the ink layer image are spliced ​​to check the consistency and the splicing consistency check result is recorded to form the ink reconstruction record.

[0063] When the splicing consistency score drops beyond the preset continuity drop threshold, the process reverts to the previous generation result and reduces the generation step size for that region. The preset continuity drop threshold is set to 0.10, based on the score drop when visible breaks begin to appear at the defect boundary in the validation set. This value is chosen to trigger the revert before obvious breaks occur.

[0064] The ink color level features and brushstroke texture features of the repaired area are extracted from the ink reconstruction results to form an ink style feature description. The corresponding ink color level features and brushstroke texture features are extracted from the areas not covered by the defect mask to form an original style feature description. The ink style feature description is matched with the original style feature description to generate a style alignment description. Based on the style alignment description, the ink color level adjustment and brushstroke texture consistency processing of the repaired area are performed on the ink reconstruction results to output the style-preserving repair result. The style alignment description and the ink reconstruction record are merged to obtain the generated adversarial repair record.

[0065] S3.3. In the style-preserving restoration result, locate the defect mask coverage area to form a restoration area index table, and in the multispectral auxiliary spectral image, locate the corresponding spectral region area according to the restoration area index table to form a spectral alignment region group; extract the spectral response description of the faded and missing areas from the spectral alignment region group to generate a spectral consistency reference description, extract the texture and brightness description corresponding to the spectral response from the style-preserving restoration result to generate a visible light consistency description, calculate the difference between the spectral consistency reference description and the visible light consistency description to obtain the spectral consistency verification conclusion, and combine the spectral consistency verification conclusion with the restoration area index table to generate a spectral consistency verification record; perform local brightness mapping correction consistent with the spectral response on the style-preserving restoration result according to the spectral consistency verification conclusion, and output the verification passed restoration result.

[0066] The spectral response description refers to the set of spectral features statistically obtained within the spectral alignment region group according to a unified coordinate reference. It includes at least the regional pixel mean vector, regional pixel standard deviation vector, intensity difference vector between adjacent spectral segments, and normalized spectral segment ratio vector of auxiliary spectral segments B1 to B8. During extraction, the regional mean and regional standard deviation of the non-defective ring outside the defective region are calculated separately under each auxiliary spectral segment. Then, they are spliced ​​together in spectral segment order to form a spectral consistency reference description. The spectral consistency reference description and the visible light consistency description are weighted and the difference is calculated to obtain the spectral consistency deviation SCD. When the SCD is higher than the preset spectral consistency deviation threshold, local brightness and darkness mapping correction is performed. The preset spectral consistency deviation threshold is 0.11. Based on the SCD boundary interval setting of the sample of the present invention and the sample of the conventional method in Example 2, this value can retain the samples that have passed and remove samples with inconsistent material responses.

[0067] The verified repair results are combined with the paper / silk substrate texture layer image to form a recombined material group. The layer alignment relationship is retrieved and a mapping table between the pixels of the verified repair results and the pixels of the paper / silk substrate texture layer is established, generating a recombined mapping table. Based on the recombined mapping table, the texture details of the paper / silk substrate texture layer image are backfilled to the corresponding positions of the verified repair results. At the defect boundary, a boundary transition zone is formed by expanding outwards from the defect boundary pixel set as the center, and the boundary transition zone pixel set is output. In the boundary transition zone pixel set, the local grayscale mean of the verified repair results and the local grayscale mean of the paper / silk substrate texture layer image are calculated respectively, and the mean difference map is output.

[0068] Based on the mean difference map, brightness offset compensation is performed on the pixel set of the boundary transition zone to eliminate the brightness jump between the backfill texture and the ink background, and a brightness-compensated texture layer is output. Gradient blending is performed on the pixel set of the boundary transition zone on the brightness-compensated texture layer, gradually reducing the proportion of the paper / silk substrate texture layer image of the pixels near the outer edge of the defect and gradually increasing the proportion of the verified repair result, so that the texture amplitude within the boundary transition zone is continuous and the substrate particle orientation is consistent, resulting in a texture transition fusion result. The texture transition fusion result is written back to the position of the pixel set of the boundary transition zone of the verified repair result while maintaining a unified coordinate reference, and a substrate consistency repair result is output. The substrate consistency repair result is encapsulated with the resynthesis mapping table and associated with the generated adversarial repair record and spectral verification record to generate a candidate repair image set.

[0069] S4. Perform multi-version alignment comparison on the candidate image set for restoration, associate consistency evaluation information, generate the optimal restoration scheme recommendation, and package it into a digital restoration result package.

[0070] S4.1. Read the substrate consistency repair results and resynthesis mapping table of each candidate version from the candidate repair image set, and read the defect mask and defect boundary from the defect semantic map. Read the generation constraint description and style alignment description from the generative adversarial repair record, and read the spectral consistency verification conclusion from the spectral verification record. Perform co-position check on the substrate consistency repair results according to the unified coordinate benchmark and establish a one-to-one correspondence between the candidate version identifier and the substrate consistency repair results. Output the version alignment parameter set and the candidate version group to be aligned.

[0071] Using the substrate consistency repair result corresponding to the reference candidate version identifier in the candidate version group to be aligned as the reference image, corner features are extracted in the area not covered by the defect mask, and a reference feature point set and a reference feature descriptor set are established; corner features are extracted in the area not covered by the defect mask of the other candidate versions, and a candidate feature point set and a candidate feature descriptor set are established; for each reference feature descriptor in the reference feature descriptor set, the descriptor distance between the reference feature descriptor and each candidate feature descriptor in the candidate feature descriptor set is calculated, and the candidate feature descriptor with the smallest distance is selected as the nearest neighbor matching result to form an initial set of matching point pairs. The descriptor distance adopts Euclidean distance or Hamming distance. Euclidean distance is used for floating-point descriptors, and Hamming distance is used for binary descriptors.

[0072] For the initial set of matching point pairs, a nearest neighbor consistency screening is performed. Matching point pairs that retain only the nearest neighbors of the reference feature descriptors as candidate feature descriptors and whose nearest neighbors are the reference feature descriptors are output. When performing outlier screening and solving for 2D rigid transformation parameters based on the matching point pair set using random sampling consistency, the smallest set of point pairs that meets the solution conditions is randomly selected and candidate 2D rigid transformation parameters are calculated. The candidate 2D rigid transformation parameters are applied to the coordinates of the reference feature points in the set of matching point pairs, and the reprojection error between the transformed coordinates and the corresponding candidate feature point coordinates is calculated. Matching point pairs with a reprojection error less than the preset pixel tolerance are determined as interior points, and the number of interior points is counted. The candidate 2D rigid transformation parameters corresponding to the largest number of interior points are repeatedly randomly selected and updated as the optimal 2D rigid transformation parameters.

[0073] It should be noted that the preset pixel tolerance is determined based on the natural fluctuation of the registration residual in the area not covered by the defective mask. In the set of matching point pairs after the nearest neighbor consistency screening, the reprojection error set of each matching point pair is first calculated using the initial two-dimensional rigid transformation parameters. The median of the reprojection error set is taken as the error scale, and the preset pixel tolerance is set to a value that is not more than about twice the error scale and not less than 1 pixel (for example, 2 to 5 pixels under common scanning resolutions). This ensures that most normal matching point pairs are retained while significantly deviating outliers are removed.

[0074] After determining the optimal two-dimensional rigid transformation parameters, the two-dimensional rigid transformation parameters are re-estimated using only the set of interior points to obtain the final two-dimensional rigid transformation parameters, which include translation and rotation angles. The final two-dimensional rigid transformation parameters are applied to the consistent repair results of the substrate for each candidate version, and bilinear interpolation is used for resampling to align each candidate version under a unified coordinate reference, and the aligned candidate version group is output.

[0075] S4.2. In the aligned candidate version group, the aligned candidate version corresponding to the reference candidate version identifier is used as the reference aligned image. Pixel differences are calculated pixel by pixel in the defect boundary area marked by the defect mask, and edge gradient differences and texture-related differences are extracted. Version difference localization map is output. The version difference localization map is organized according to the candidate version identifier and a corresponding relationship is established with the defect boundary. Difference localization information is output.

[0076] It should be noted that pixel differences can be expressed as absolute differences, specifically: ; in, Indicates the pixel position under a unified coordinate system. Indicates the first candidate version in the aligned candidate version group The aligned candidate versions at pixel position pixel values, Indicates reference aligned image At pixel position pixel values, Indicates the first The aligned candidate versions are at pixel positions relative to the reference aligned image. The pixel differences are used to generate the pixel difference components of the version difference location map.

[0077] Edge gradient differences can be represented by gradient magnitude differences, specifically: ; in, Indicates the first The aligned candidate versions at pixel position gradient magnitude, Indicates reference aligned image At pixel position gradient magnitude, Indicates the first The edge gradient difference of the aligned candidate versions within the defect boundary region is used to characterize whether the boundary transition produces discontinuous edges.

[0078] Texture-related differences can be represented by local variance differences, specifically: ; in, Represented by pixel position The set of pixels in the neighborhood of the center. Represents the set of neighboring pixels A certain pixel position within, Indicates the first The aligned candidate versions at pixel position Pixel value at that location, This represents the number of pixels in the neighboring pixel set. Indicates the first Each aligned candidate version is in the set of neighboring pixels. The average pixel value within the range, Indicates the first The aligned candidate versions at pixel position The intensity of local texture undulations at that location. Indicates reference aligned image At pixel position The intensity of local texture undulations at that location. Indicates the first Texture-related differences in aligned candidate versions are used to characterize the particle continuity differences after backfilling the paper / silk substrate texture.

[0079] S4.3. Associate and merge the difference location information with the generated adversarial repair record, spectral band verification record, and resynthesis mapping table to form a version comparison result; the version comparison result records the pixel difference statistics, edge gradient difference statistics, and texture-related difference statistics according to the candidate version identifier, and records the correspondence between the spectral band consistency verification conclusion and the resynthesis mapping table, and outputs the version comparison result for scheme selection.

[0080] The generated adversarial repair records, spectral verification records, resynthesis mapping tables, and version comparison results are summarized and encapsulated to generate consistency assessment information. The consistency assessment information is indexed by candidate version identifiers and kept consistent with the unified coordinate benchmark.

[0081] Based on version comparison results and consistency assessment information, the optimal repair scheme recommendation is determined and a recommended version identifier is generated. The process of determining the optimal repair scheme recommendation adopts a sequential screening method: first, candidate versions with a failed spectral consistency verification result are screened out based on the spectral segment verification record; then, among the candidate versions with a passed spectral consistency verification result, the median of pixel differences within the defect boundary region is compared and the candidate version identifier with the smallest median is selected; if the median of pixel differences is the same, the median of edge gradient differences is compared and the candidate version identifier with the smallest median is selected; if the median of edge gradient differences is the same, the median of texture-related differences is compared and the candidate version identifier with the smallest median is selected. The optimal repair scheme recommendation and the recommended version identifier are then output.

[0082] Archive the candidate versions corresponding to the recommended version identifier, the defect semantic map, the hierarchical alignment relationship, the version comparison results and the consistency assessment information. The archived content is indexed according to the candidate version identifier, the unified coordinate benchmark and the hierarchical alignment relationship and is kept consistent and traceable. Output the digital repair result package.

[0083] This embodiment also provides a digital restoration system for damaged areas of calligraphy and painting works, including: The acquisition and correction module acquires visible light images and multispectral auxiliary spectrum images of calligraphy and paintings, and completes geometric correction, color consistency correction, and separation and alignment of the ink layer and the paper / silk substrate texture layer, generating a reference image package containing layer alignment relationships. The defect semantic module performs defect region segmentation and defect type labeling on the baseline image package, and extracts the pen stroke direction field to generate a defect semantic map; The generation and repair module utilizes the defect semantic map to carry out brushstroke structure-guided generative adversarial repair and implements style consistency maintenance processing. At the same time, it performs multispectral consistency verification and completes the re-synthesis of the substrate texture according to the layer alignment relationship, generating a candidate repair image set and consistency evaluation information. The version encapsulation module performs multi-version alignment comparison on the candidate image set for restoration, associates consistency evaluation information, generates the optimal restoration scheme recommendation, and encapsulates it into a digital restoration result package.

[0084] This embodiment also provides a computer device applicable to the digital restoration method for damaged areas of calligraphy and painting works, including: a memory and a processor; the memory is used to store computer-executable instructions, and the processor is used to execute the computer-executable instructions to realize the digital restoration method for damaged areas of calligraphy and painting works as proposed in the above embodiment.

[0085] The computer device can be a terminal, comprising a processor, memory, communication interface, display screen, and input devices connected via a system bus. The processor provides computing and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system and computer programs. The internal memory provides an environment for the operation of the operating system and computer programs stored in the non-volatile storage media. The communication interface is used for wired or wireless communication with external terminals; wireless communication can be achieved through Wi-Fi, carrier networks, NFC (Near Field Communication), or other technologies. The display screen can be an LCD screen or an e-ink screen. The input devices can be a touch layer covering the display screen, buttons, a trackball, or a touchpad on the computer device's casing, or an external keyboard, touchpad, or mouse.

[0086] This embodiment also provides a storage medium storing a computer program. When executed by a processor, the program implements the digital restoration method for damaged areas of calligraphy and painting works as proposed in the above embodiments. The storage medium can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as Static Random Access Memory (SRAM), Electrically Erasable Programmable Read-Only Memory (EEPROM), Erasable Programmable Read Only Memory (EPROM), Programmable Red-Only Memory (PROM), Read-Only Memory (ROM), magnetic storage, flash memory, magnetic disk, or optical disk.

[0087] In summary, this invention achieves consistent spatial reference for subsequent defect detection, spectral verification, and version alignment by using geometric correction, color consistency correction, and unified coordinate benchmarks for visible light images and multispectral auxiliary spectral band images, thus reducing cross-spectral and cross-version error propagation. By estimating the ink layer and modeling the paper / silk substrate texture layer to establish a layered alignment relationship, the restoration process can maintain the continuity of brushstroke shape, ink color levels, and substrate particles, avoiding restoration artifacts caused by mistaking substrate texture for brushstroke structure. A defect semantic map is formed by combining defect masks, defect type label maps, and brushstroke direction fields, and generative constraint descriptions guide adversarial restoration, enabling differentiated reconstruction of damage forms such as missing parts, cracks, fading, and stains while maintaining the continuity of brushstroke direction at defect boundaries. Multispectral consistency verification and spectral band verification record the consistency of spectral response in the constrained restoration area, reducing the risk of visually reasonable but inconsistent material responses caused by relying solely on visible light.

[0088] Example 2, referring to Table 1, is the second embodiment of the present invention. To further verify the technical solution of the present invention, experimental simulation data of the digital restoration method for damaged areas of calligraphy and painting works are provided. In this embodiment, six representative calligraphy and painting samples to be restored are selected as performance verification samples. These six samples are only used for method comparison and verification and are not used as a limitation on the amount of training data for the defect detection and classification model.

[0089] Six representative paintings and calligraphy samples to be restored (silk / paper, ink / color, including cases of slight fading, cracks, stains, complex missing edges, fading + missing parts, and low contrast in multiple spectral bands) were selected. Visible light images and multispectral auxiliary spectral band images were acquired under the same imaging conditions, and the shooting parameters and imaging calibration information were recorded simultaneously. Based on the shooting parameters and imaging calibration information, lens distortion correction and viewing angle difference correction were performed on the visible light images and multispectral auxiliary spectral band images respectively to form a unified viewing angle image sequence. The boundaries and reference marks of the paintings and calligraphy were located in the unified viewing angle image sequence, and a unified coordinate reference was generated. The visible light images were acquired by the aforementioned high-resolution area array color industrial camera, and the multispectral auxiliary spectral band images were acquired by the aforementioned 8-channel multispectral imaging device.

[0090] Subsequently, under a unified coordinate reference, translation, rotation registration, and scale matching are performed to obtain a geometrically aligned image sequence. Then, a color reference model is constructed using the background reference region, and white balance and tone mapping are performed to form a calibration image set. Based on the calibration image set, ink response features and substrate texture response features are extracted to form a separation feature set. Based on the separation feature set, ink layer estimation is performed to obtain the ink layer image, and substrate texture modeling is performed to obtain the paper / silk substrate texture layer image. At the same time, pixel-level registration is recorded to obtain the layer alignment relationship and encapsulated to generate a reference image package.

[0091] The ink layer image and multispectral auxiliary spectrum image are read from the reference image package and organized in situ according to a unified coordinate reference to form a defect analysis image group. The defect analysis image group is fed into the defect detection and classification model to output defect mask, defect boundary and defect type label map. In the area not covered by the defect mask, the brush stroke skeleton set is extracted and the direction is statistically analyzed to obtain the brush stroke direction field. The combined and encapsulated images form a defect semantic map. The boundary brush stroke guiding fragment set is extracted from the defect semantic map and jointly encoded with the defect type label map to obtain the generative constraint description. Generative adversarial repair is performed according to the generative constraint description to obtain the ink reconstruction result. The ink style feature description is extracted and the original style feature description is used to generate the style alignment description. The style preservation repair result is obtained according to the style alignment description and the generative adversarial repair record is formed.

[0092] In the style-preserving restoration results, a restoration region index table is generated, and spectral alignment region groups are located in the multispectral auxiliary spectral band image. The spectral consistency reference description is extracted and compared with the visible light consistency description to obtain the spectral consistency verification conclusion and generate a spectral consistency verification record. Based on the spectral consistency verification conclusion, the verification passed restoration result is output. The verification passed restoration result and the paper / silk substrate texture layer image are aligned according to the layer alignment relationship to generate a re-synthesis mapping table and backfill the texture details. After texture transition fusion processing at the defect boundary, the substrate consistency restoration result is output and packaged to obtain a candidate restoration image set. Multi-version alignment comparison and difference localization are performed on the candidate restoration image set and consistency evaluation information is associated to output the optimal restoration scheme recommendation and digital restoration result package. The control group "conventional method" uses a single visible light image for manual / heuristic registration, defect segmentation and type discrimination based on threshold or traditional texture operators, and texture re-synthesis without multispectral consistency verification and layer alignment relationship constraints, thus forming a comparable control result.

[0093] Specifically, the control group used a single visible light image for manual point selection and affine transformation registration, grayscale thresholding combined with morphological opening and closing operations to extract the defective area, traditional texture operators to identify the defect type, and texture filling based on sample block replication to complete the repair. This control group did not perform multispectral consistency verification, nor did it establish a layer alignment relationship between the ink layer and the substrate texture layer.

[0094] The details are shown in Table 1 below: Table 1 Comparison of Digital Repair Data for Defective Areas The paired results of the conventional method and the method of this invention in the table show that, under the closed-loop constraints of "unified coordinate reference + hierarchical alignment relationship + multispectral consistency verification", this invention significantly reduces error propagation and improves semantic consistency of defects: the average geometric alignment residual RMSE decreased from 0.923 pixels to 0.527 pixels (a reduction of approximately 43.0%), and the color consistency ΔE00 decreased from 3.81 to 2.35 (a reduction of approximately 38.3%), indicating that the normalization of geometry and color effectively suppressed the systematic bias caused by cross-spectral and cross-viewpoints; the defect segmentation intersection-union ratio (IoU) increased from 80.47% to 89.40% (an improvement of approximately 11.1%), and the defect type category accuracy increased from 80.37% to 88.13% (an improvement of approximately 9.7%), indicating that... The isotopic organization of "ink layer image + multispectral auxiliary band image" improved the separability and type identification of defect boundaries; the band consistency deviation (SCD) decreased by about 40.9%, the brightness jump of the boundary transition band (LJ) decreased by about 47.1%, and the texture correlation difference (ΔV) decreased by about 49.0%, reflecting that the texture backfilling and texture transition fusion processing supported by the layer alignment relationship can maintain the continuity of paper and silk substrate particles and reduce boundary artifacts; the version recommendation consistency score (CS) increased by about 15.6%, and the manual review time decreased from 52.17 minutes to 33.5 minutes (a reduction of about 35.8%), indicating that the association and merging of multi-version alignment comparison and consistency assessment information improved the interpretability and traceability of scheme selection, forming a stable output for digital preservation archiving and restoration process reproduction.

[0095] It should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit it. Although the present invention has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can be made to the technical solutions of the present invention without departing from the spirit and scope of the technical solutions of the present invention, and all such modifications or substitutions should be covered within the scope of the claims of the present invention.

Claims

1. A method for digitally restoring damaged areas of a painting or calligraphy work, characterized in that: include, The system acquires visible light images and multispectral auxiliary spectral band images of calligraphy and paintings, performs geometric correction, color consistency correction, and separation and alignment of the ink layer and the paper / silk substrate texture layer, and generates a reference image package containing layer alignment relationships. The missing region segmentation and missing type labeling are performed on the reference image package, and the pen stroke direction field is extracted to generate a missing semantic map; We utilize the missing semantic map to carry out generative adversarial repair guided by brush stroke structure and implement style consistency maintenance processing. At the same time, we perform multispectral consistency verification and complete the re-synthesis of substrate texture according to the layer alignment relationship to generate a candidate repair image set. Perform multi-version alignment comparison on the candidate image set for restoration, associate consistency evaluation information, generate the optimal restoration scheme recommendation, and package it into a digital restoration result package.

2. The method for digitally restoring damaged areas of calligraphy and painting works as described in claim 1, characterized in that: The specific steps for completing geometric correction and color consistency correction are as follows. The system reads the shooting parameters and imaging calibration information of the visible light image and the multispectral auxiliary spectral band image, performs lens distortion correction and viewing angle difference correction on the visible light image and the multispectral auxiliary spectral band image, and generates a unified viewing angle image sequence. In a unified viewpoint image sequence, the boundaries of calligraphy and painting are located and reference marks are established, and a coordinate mapping relationship is generated to generate a unified coordinate reference. By using a unified coordinate reference, translation, rotation registration, and scale matching are performed on a unified viewpoint image sequence to generate a geometrically aligned image sequence. Extract the background reference region of the geometrically aligned image sequence and construct color reference description information to generate a color reference model; Based on the color reference model, white balance and tone mapping are performed on the geometrically aligned image sequence to generate a color-consistent image sequence, which is then encapsulated with a unified coordinate reference to form a corrected image set.

3. The method for digitally restoring damaged areas of calligraphy and painting works as described in claim 1, characterized in that: The specific steps for separating and aligning the ink layer with the paper / silk substrate texture layer are as follows. Based on the corrected image set, the brush and ink response features and substrate texture response features of visible light images are extracted, and brush and ink layer estimation is performed on the calligraphy and painting images to generate brush and ink layer images. At the same time, substrate texture modeling is performed on the calligraphy and painting images to generate paper and silk substrate texture layer images. Pixel-level registration is performed on the ink layer image and the paper / silk substrate texture layer image, and the corresponding pixel mapping relationship is recorded to form a layer alignment relationship; The ink layer image, the paper / silk substrate texture layer image, and the layer alignment relationship are bound and registered to generate a reference image package.

4. The method for digitally restoring damaged areas of calligraphy and painting works as described in claim 1, characterized in that: The specific steps for generating the defective semantic map are as follows: Retrieve the ink layer image and layer alignment relationship from the reference image package, perform defect region segmentation, obtain the defect mask, and locate the defect boundary; Perform defect type discrimination on the area covered by the defect mask to obtain a defect type label map, and extract the stroke edges and stroke directions for the area not covered by the defect mask to obtain a stroke skeleton set; Directional statistics are performed on the stroke skeleton set to form a stroke direction field, which is then combined with the defect mask and defect type label map to generate a defect semantic map.

5. The method for digitally restoring damaged areas of calligraphy and painting works as described in claim 1, characterized in that: The specific steps for using the missing semantic map to perform stroke structure-guided generative adversarial repair and implement style consistency preservation processing are as follows: Extract continuous segments of the stroke direction field at the defect boundary in the defect semantic map to form a set of boundary stroke guidance segments, and jointly encode them with the defect type label map to form a generative constraint description; Based on the generative constraint description, the configuration structure guide channel is configured and generative adversarial repair is performed to obtain the ink reconstruction result that extends continuously along the stroke direction field; The ink style feature description is extracted from the ink reconstruction result, and the original style feature description is extracted from the area not covered by the missing mask to generate a style alignment description. Based on the style alignment description, the ink tones and brushstroke textures of the restored area are kept consistent with the original area to generate a style-preserving restoration result. The style-preserving repair results, style alignment descriptions, and generation constraint descriptions are merged to obtain the generation adversarial repair record.

6. The method for digitally restoring damaged areas of calligraphy and painting works as described in claim 1, characterized in that: The specific steps for performing multispectral consistency verification are as follows. Extract the location index of the repaired region from the style-preserving repair result to form a repaired region index table, and locate the corresponding spectral region in the multispectral auxiliary spectral region image of the reference image package to form a spectral alignment region group; Extract spectral response descriptions of faded and missing regions from the spectral alignment region group to generate a spectral consistency reference description; Extract texture and brightness descriptions related to spectral response from the style-preserving restoration results to generate visible light consistency descriptions; The spectral band consistency reference description is compared with the visible light consistency description to obtain the spectral band consistency verification conclusion, and combined with the repair area index table to generate spectral band verification records. Based on the spectral consistency verification results, the style preservation repair results are corrected, and a verification-passed repair result is generated.

7. The method for digitally restoring damaged areas of calligraphy and painting works as described in claim 1, characterized in that: The specific steps for re-compositing the substrate texture according to the layer alignment relationship are as follows. The verified repair results are combined with the paper and silk substrate texture layer image in the baseline image package to obtain the recombined material group; Retrieve the layer alignment relationship and establish a mapping table between the pixels of the verified repair result and the pixels of the paper and silk substrate texture layer, and generate a recombination mapping table; Based on the resynthesis mapping table, the texture details of the paper and silk substrate texture layer image are backfilled to the corresponding positions of the verified repair results. Texture transition fusion processing is performed at the defect boundary while maintaining the continuity of substrate particles to generate a consistent substrate repair result. The substrate-consistent repair results are encapsulated with a resynthesis mapping table to obtain a candidate repair image set.

8. The method for digitally restoring damaged areas of calligraphy and painting works as described in claim 1, characterized in that: The specific steps for performing multi-version alignment comparison on the candidate image set and associating it with consistency evaluation information are as follows: Extract a unified coordinate reference for each candidate version in the candidate image set for restoration, and perform image registration on each candidate version to generate an aligned candidate version group; Perform difference localization on the aligned candidate version group, generate a version difference localization map, and associate and merge it with the consistency assessment information to generate version comparison results; The generated adversarial repair records, spectral verification records, resynthesis mapping tables and version comparison results are summarized and packaged to generate consistency assessment information. Based on the version comparison results and consistency assessment information, the optimal remediation solution is recommended and a recommended version identifier is generated; The candidate versions corresponding to the recommended version identifier, the defect semantic map, the hierarchical alignment relationship, the version comparison results and the consistency assessment information are archived to generate a digital repair result package.

9. The method for digitally restoring damaged areas of calligraphy and painting works as described in claim 8, characterized in that: The difference localization refers to calculating pixel differences and extracting edge gradient differences and texture-related differences for the aligned candidate version group according to the defect boundary region marked by the defect mask.

10. A digital restoration system for damaged areas of calligraphy and painting works, based on the digital restoration method for damaged areas of calligraphy and painting works according to any one of claims 1 to 9, characterized in that: include, The acquisition and correction module acquires visible light images and multispectral auxiliary spectrum images of calligraphy and paintings, and completes geometric correction, color consistency correction, and separation and alignment of the ink layer and the paper / silk substrate texture layer, generating a reference image package containing layer alignment relationships. The defect semantic module performs defect region segmentation and defect type labeling on the baseline image package, and extracts the pen stroke direction field to generate a defect semantic map; The generation and repair module utilizes the defect semantic map to carry out brushstroke structure-guided generative adversarial repair and implements style consistency maintenance processing. At the same time, it performs multispectral consistency verification and completes the re-synthesis of the substrate texture according to the layer alignment relationship, generating a candidate repair image set and consistency evaluation information. The version encapsulation module performs multi-version alignment comparison on the candidate image set for restoration, associates consistency evaluation information, generates the optimal restoration scheme recommendation, and encapsulates it into a digital restoration result package.

Images