Heterogenous cultural relic fragment restoration method and system, terminal and storage medium

Abstract

The invention discloses a heterogeneous cultural relic fragment restoration method and system, a terminal and a storage medium, and the method comprises the steps: obtaining a plurality of to-be-restored cultural relic fragment images of a to-be-restored cultural relic, and carrying out the fragment source division processing of the plurality of to-be-restored cultural relic fragment images, and obtaining a plurality of source division compression packages; performing damage repair processing on the plurality of sub-source compressed packages to obtain a plurality of initial damage repair cultural relic images; and performing fragment splicing processing on the plurality of initial damaged and repaired cultural relic images to obtain a target damaged and repaired cultural relic image. According to the method, the to-be-repaired cultural relic fragment image is subjected to fragment source division processing, so that the cultural relic repairing accuracy and repairing efficiency can be effectively improved, further, the cultural relic after fragment source division processing is subjected to damage repairing processing and fragment splicing processing, repairing and reconstruction of the cultural relic fragments can be quickly realized, and the repairing efficiency of the cultural relic is improved. And the repairing effect of the cultural relic fragments is effectively improved.

Description

Technical Field

[0001] The present invention relates to the technical field of cultural relic image processing, and in particular to a method, system, terminal and computer-readable storage medium for repairing heterogeneous cultural relic fragments. Background Art

[0002] Cultural relics are important witnesses of the development of human civilization, and their protection and restoration work is of great significance for inheriting historical culture. In the field of fragment splicing and restoration of unearthed cultural relics, there are two completely different technical paths: traditional manual restoration (physically splicing and restoring cultural relic fragments directly) and digital-assisted restoration (guiding physical restoration through 3D modeling and virtual splicing).

[0003] In recent years, with the in-depth development of archaeological excavation work, the number of unearthed cultural relics has increased sharply. The traditional manual restoration method has been difficult to meet the current needs of cultural relic restoration work, and digital-assisted restoration technology is gradually becoming the key path to break through the bottleneck of traditional restoration. The virtual splicing prediction system realized by 3D scanning and AI algorithms effectively avoids the high risk of direct physical operation.

[0004] However, digital-assisted restoration technology mainly focuses on edge splicing or edge color matching. When the number of cultural relic fragments is too large or the edge similarity of cultural relic fragments is high, it is impossible to ensure effective restoration quality and effect, resulting in the inability to meet the cultural relic restoration needs of users.

[0005] Therefore, the existing technology still needs to be improved and developed. Summary of the Invention

[0006] The main purpose of the present invention is to provide a method, system, terminal and computer-readable storage medium for repairing heterogeneous cultural relic fragments, aiming to solve the problem that digital-assisted restoration technology in the existing technology mainly focuses on edge splicing or edge color matching, and when the number of cultural relic fragments is too large or the edge similarity of cultural relic fragments is high, it is impossible to ensure effective restoration quality and effect.

[0007] To achieve the above object, the present invention provides a method for repairing heterogeneous cultural relic fragments, and the method for repairing heterogeneous cultural relic fragments includes the following steps:

[0008] Obtain a plurality of images of cultural relic fragments to be repaired of the cultural relic to be repaired, and perform fragment source separation processing on the plurality of images of cultural relic fragments to be repaired to obtain a plurality of source-separated compressed packages;

[0009] Perform damage repair processing on the plurality of source-separated compressed packages to obtain a plurality of initial damaged repaired cultural relic images;

[0010] Perform fragment splicing processing on the plurality of initial damaged repaired cultural relic images to obtain a target damaged repaired cultural relic image.

[0011] Optionally, in the method for repairing heterogeneous cultural relic fragments, the fragment source separation process includes source classification processing and type classification processing; the source separation compressed package includes the source classification result and the type classification result;

[0012] The steps of obtaining a plurality of images of cultural relic fragments to be repaired, and performing fragment source separation processing on the plurality of images of cultural relic fragments to be repaired to obtain a plurality of source separation compressed packages specifically include:

[0013] Obtaining a plurality of images of cultural relic fragments to be repaired, and performing preprocessing on the plurality of images of cultural relic fragments to be repaired to obtain a plurality of global image features;

[0014] Performing source classification processing on the plurality of global image features to obtain a plurality of source classification results;

[0015] Performing type classification processing on the plurality of global image features to obtain a plurality of type classification results.

[0016] Optionally, in the method for repairing heterogeneous cultural relic fragments, the step of performing preprocessing on the plurality of images of cultural relic fragments to be repaired to obtain a plurality of global image features specifically includes:

[0017] Performing object recognition processing and background subtraction processing on the plurality of images of cultural relic fragments to be repaired to obtain a plurality of initial classification images;

[0018] Performing local feature extraction processing on the plurality of initial classification images to obtain a plurality of local image features;

[0019] Performing feature aggregation processing on the plurality of local image features to obtain a plurality of global image features.

[0020] Optionally, in the method for repairing heterogeneous cultural relic fragments, the step of performing source classification processing on the plurality of global image features to obtain a plurality of source classification results specifically includes:

[0021] Determining a support vector machine classifier, and training the model of the support vector machine classifier according to the plurality of global image features;

[0022] Performing classification prediction processing on the plurality of initial classification images through the trained support vector machine classifier to obtain a plurality of source classification results.

[0023] Optionally, in the method for repairing heterogeneous cultural relic fragments, the step of performing type classification processing on the plurality of global image features to obtain a plurality of type classification results specifically includes:

[0024] Determine a pre-trained model, and perform deep feature extraction processing on multiple global features of the images through the pre-trained model to obtain multiple image deep features;

[0025] Use the principal component analysis method to perform dimensionality reduction processing and clustering processing on multiple image deep features to obtain multiple type classification results.

[0026] Optionally, in the method for repairing heterogeneous cultural relic fragments, the step of performing damage repair processing on multiple sub-source compressed packages to obtain multiple initial damaged repaired cultural relic images specifically includes:

[0027] Judge the repair modes corresponding to multiple sub-source compressed packages, where the repair modes include an extrapolation band expansion mode corresponding to multiple separate fragment pictures and an internal mask mode corresponding to a single damaged picture;

[0028] If the repair mode corresponding to multiple sub-source compressed packages is the extrapolation band expansion mode, perform pixel edge extrapolation processing on multiple sub-source compressed packages to obtain an edge extrapolation region map;

[0029] If the repair mode corresponding to multiple sub-source compressed packages is the internal mask mode, perform mask filling processing on multiple sub-source compressed packages to obtain a mask region map;

[0030] Perform grayscale processing and image edge information extraction processing on the edge extrapolation region map or the mask region map to obtain an edge information map;

[0031] Use an edge repair model to perform edge repair processing on the edge information map to obtain an edge repair map;

[0032] Use a content repair model to perform color repair processing and content repair processing on the edge repair map to obtain multiple initial damaged repaired cultural relic images.

[0033] Optionally, in the method for repairing heterogeneous cultural relic fragments, the step of performing fragment splicing processing on multiple initial damaged repaired cultural relic images to obtain a target damaged repaired cultural relic image specifically includes:

[0034] Perform feature matching processing on multiple initial damaged repaired cultural relic images through a feature matching model to obtain feature matching points;

[0035] Perform clustering screening processing and transformation matrix calculation processing on the feature matching points to obtain a target transformation matrix;

[0036] Perform perspective transformation processing and image fusion processing on the target transformation matrix to obtain a target damaged repaired cultural relic image.

[0037] In addition, to achieve the above object, the present invention also provides a restoration system for heterogeneous cultural relic fragments, wherein the restoration system for heterogeneous cultural relic fragments includes:

[0038] A fragment source separation processing module, configured to obtain a plurality of images of cultural relic fragments to be restored of a cultural relic to be restored, and perform fragment source separation processing on the plurality of images of cultural relic fragments to be restored to obtain a plurality of source-separated compressed packages;

[0039] A damage restoration processing module, configured to perform damage restoration processing on the plurality of source-separated compressed packages to obtain a plurality of initial damaged restored cultural relic images;

[0040] A fragment splicing processing module, configured to perform fragment splicing processing on the plurality of initial damaged restored cultural relic images to obtain a target damaged restored cultural relic image.

[0041] In addition, to achieve the above object, the present invention also provides a terminal, wherein the terminal includes: a memory, a processor, and a restoration program for heterogeneous cultural relic fragments stored on the memory and executable on the processor, and when the restoration program for heterogeneous cultural relic fragments is executed by the processor, the steps of the above-mentioned restoration method for heterogeneous cultural relic fragments are implemented.

[0042] In addition, to achieve the above object, the present invention also provides a computer-readable storage medium, wherein the computer-readable storage medium stores a restoration program for heterogeneous cultural relic fragments, and when the restoration program for heterogeneous cultural relic fragments is executed by a processor, the steps of the above-mentioned restoration method for heterogeneous cultural relic fragments are implemented.

[0043] In the present invention, a plurality of images of cultural relic fragments to be restored of a cultural relic to be restored are obtained, and fragment source separation processing is performed on the plurality of images of cultural relic fragments to be restored to obtain a plurality of source-separated compressed packages; damage restoration processing is performed on the plurality of source-separated compressed packages to obtain a plurality of initial damaged restored cultural relic images; fragment splicing processing is performed on the plurality of initial damaged restored cultural relic images to obtain a target damaged restored cultural relic image. By performing fragment source separation processing on the images of cultural relic fragments to be restored, the present invention can effectively avoid splicing errors caused by incorrect source separation, thereby significantly improving the accuracy and efficiency of cultural relic restoration. Further, by performing damage restoration processing on the cultural relic fragments after fragment source separation processing, the present invention can extract the features of the cultural relic fragments, which helps to perform fragment splicing on the cultural relic fragments later, and effectively improves the restoration quality and restoration effect of cultural relic fragments under the conditions of a large number and heterogeneous sources. BRIEF DESCRIPTION OF THE DRAWINGS

[0044] Figure 1 is a flowchart of a preferred embodiment of the restoration method for heterogeneous cultural relic fragments of the present invention;

[0045] Figure 2It is a schematic diagram of the overall process of the preferred embodiment of the method for repairing heterogeneous cultural relic fragments of the present invention;

[0046] Figure 3 It is a schematic diagram of the process of the fragment source separation module of the preferred embodiment of the method for repairing heterogeneous cultural relic fragments of the present invention;

[0047] Figure 4 It is a schematic diagram of the process of the extrapolation band module of the preferred embodiment of the method for repairing heterogeneous cultural relic fragments of the present invention;

[0048] Figure 5 It is a schematic diagram of the repair effect of the preferred embodiment of the method for repairing heterogeneous cultural relic fragments of the present invention;

[0049] Figure 6 It is a schematic diagram of the architecture of the edge repair model of the preferred embodiment of the method for repairing heterogeneous cultural relic fragments of the present invention;

[0050] Figure 7 It is a schematic diagram of the architecture of the content repair model of the preferred embodiment of the method for repairing heterogeneous cultural relic fragments of the present invention;

[0051] Figure 8 It is a schematic diagram of the process of the fragment splicing module of the preferred embodiment of the method for repairing heterogeneous cultural relic fragments of the present invention;

[0052] Figure 9 It is a schematic diagram of the interface for the user to manually adjust the splicing result of the preferred embodiment of the method for repairing heterogeneous cultural relic fragments of the present invention;

[0053] Figure 10 It is a schematic diagram of the structure of the preferred embodiment of the repair system for heterogeneous cultural relic fragments of the present invention;

[0054] Figure 11 It is a schematic diagram of the structure of the preferred embodiment of the terminal of the present invention. Specific embodiments

[0055] To make the objectives, technical solutions and advantages of the present invention clearer and more definite, the following further elaborates on the present invention with reference to the accompanying drawings and by way of examples. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not used to limit the present invention.

[0056] Cultural relics are important witnesses to the development of human civilization, and their protection and restoration work is of great significance for inheriting historical culture. In the professional field of fragment splicing and restoration of unearthed cultural relics, there are two completely different technical paths: traditional manual restoration (physically splicing and restoring cultural relic fragments directly) and digital-assisted restoration (guiding physical restoration through 3D modeling and virtual splicing). In recent years, with the in-depth development of archaeological excavation work, the number of unearthed cultural relics has increased sharply, and the traditional manual restoration method has been difficult to meet the current needs of cultural relic restoration work. However, there are also many deficiencies in the current digital-assisted restoration solutions.

[0057] Manual operations face the following main problems: 1. High labor cost: Professional restorers are scarce and the training cycle is long; 2. Low fault tolerance: Manual operations are prone to cause secondary damage; 3. Time-consuming: Especially when dealing with fragmented cultural relics, the time consumption is more prominent; 4. Low efficiency: A large number of cultural relics need to be restored, but the manual processing speed is limited.

[0058] In the field of fragmented cultural relic restoration, digital technology is gradually becoming the key path to break through the bottleneck of traditional restoration. The virtual splicing prediction system realized through 3D scanning and AI algorithms can provide an accurate digital "construction blueprint" for physical restoration. This technical path decomposes the physical restoration into a scientific process of "digital modeling → virtual splicing → physical verification", effectively avoiding the high risks of direct physical operations. This digital cultural relic splicing and restoration solution is the inevitable solution for future cultural relic restoration.

[0059] Analysis of existing digital technologies and their disadvantages:

[0060] The common computer-aided processing methods on the market currently mainly focus on edge splicing or edge color matching. These methods have the following problems: 1. Limited fragment processing ability: When the number of fragments is too large, the system is difficult to effectively process and manage; 2. Inaccurate edge matching: When the edge similarities of multiple fragments are high, it is easy to cause incorrect matching; 3. Unreliable results: There are often cases of splicing distortion or completely wrong situations.

[0061] Although deep learning models and image generation technologies have been applied in the field of cultural relic restoration, there are still the following problems: 1. Unsatisfactory processing effect: When dealing with fragmented cultural relics, the restoration quality is difficult to guarantee; 2. Limitations of the GAN (Generative Adversarial Networks) model: The effect is not good when dealing with complex fragments and edge damages; 3. Artifact problem: Visual artifacts are easily generated during the restoration process, affecting the restoration effect.

[0062] The current restoration system has the following systematic problems: 1. Single function: mostly concentrated on a single splicing or color matching algorithm; 2. Low integration: lacking an overall solution for multi-functional collaboration; 3. Poor scalability: difficult to adapt to the restoration needs of different types of cultural relics; 4. Efficiency bottleneck: when dealing with a large number of fragments, the system performance drops significantly.

[0063] These problems severely restrict the efficiency and quality of cultural relic restoration work, and there is an urgent need for a new technical solution to break through the limitations of existing technologies.

[0064] To solve the above problems, in view of the problems of high cost, low fault tolerance, and long time consumption in the traditional cultural relic restoration process, especially the technical problems in dealing with a large number of heterogeneous fragmented cultural relics, the present invention proposes an innovative "three-step restoration method", that is, using AI to complete fragment classification, predict and extrapolate images, and feature splicing. The present invention integrates three core modules: fragment source separation, damage repair, and AI splicing, and constructs a set of efficient, accurate, and scalable cultural relic restoration systems. The present invention uses image extrapolation technology to quickly generate repair plans and realizes the intelligent reconstruction of fragments through feature splicing.

[0065] The restoration method for heterogeneous cultural relic fragments according to a preferred embodiment of the present invention, as Figure 1 shown, the restoration method for heterogeneous cultural relic fragments includes the following steps:

[0066] Step S10: Obtain multiple images of cultural relic fragments to be restored of the cultural relic to be restored, and perform fragment source separation processing on the multiple images of cultural relic fragments to be restored to obtain multiple source-separated compressed packages. The fragment source separation processing includes source classification processing and type classification processing; the source-separated compressed package includes source classification results and type classification results.

[0067] As Figure 2 shown, the "three-step restoration method" set in the present invention includes the following three modules: 1. AI fragment source separation module (that is, the fragment source separation module mentioned later): Introduce fragment source separation at the initial stage of the restoration process to avoid incorrect splicing and ensure that the spliced fragments come from the same cultural relic; 2. AI damage repair module (that is, the damage repair module mentioned later): Adopt an improved GAN model and fast Fourier convolution (FFC, Fast Fourier Transform) to improve feature extraction and reconstruction capabilities; 3. AI splicing module (that is, the splicing module mentioned later): Generate predicted content through extrapolation, and perform intelligent splicing by combining feature point recognition and puzzle solving.

[0068] As Figure 2As shown in the figure, the present invention realizes the classification, repair and splicing of fragmented images through AI technology, mainly including three major modules: "AI Fragment Source Classification", "AI Damage Repair" and "AI Fragment Splicing", supplemented by auxiliary modules such as super-resolution models, object recognition models and background classification models. Users can choose to use a certain module alone according to their needs, or perform joint scheduling through the automated workflow mode.

[0069] Among them, the automated workflow mode is applicable to the processing of a large number of fragments, and the system will execute in the following process sequence: 1. AI Fragment Source Classification: Classify the input fragmented images to generate multiple source classification compression packages. 2. AI Damage Repair: Perform edge extrapolation and content repair on the classified fragments. 3. AI Fragment Splicing: Splice the repaired fragments to generate a preliminary splicing result. 4. AI Damage Repair (secondary repair): Perform internal damage repair on the spliced image and output the final result. 5. Super-resolution processing (optional): Users can choose to perform super-resolution processing on the final result to improve the image quality.

[0070] The three major modules set in the present invention are all independent modules, and users can choose to use them according to specific needs.

[0071] Among them, the AI Fragment Source Classification module includes: 1. Function: Classify the input fragmented images to generate multiple source classification compression packages. 2. Input: Multiple individual fragmented pictures (preferably recommended to ensure high resolution and subsequent processing accuracy) or a picture containing multiple fragments. 3. Processing process: Call the object recognition and background deduction API (interface) to only retain the fragments themselves; use the source classification model to classify the fragments. 4. Output: Multiple source classification compression packages, and each compression package contains fragmented images of the same category.

[0072] Specifically, obtain multiple fragmented images of cultural relics to be repaired, and perform object recognition processing and background deduction processing on the multiple fragmented images of cultural relics to be repaired to obtain multiple initial classification images; perform local feature extraction processing on the multiple initial classification images to obtain multiple image local features; perform feature aggregation processing on the multiple image local features to obtain multiple image global features.

[0073] It is understandable that the AI fragment image source classification module in the present invention is divided into two main functions: fragment image traceability and type aggregation. In source classification, the model extracts local features of cultural relic fragments through algorithms such as SIFT (Scale-Invariant Feature Transform) and ORB (Oriented FAST and Rotated BRIEF), aggregates them through VLAD (Vector of Locally Aggregated Descriptors) to generate global features, then uses K-Means clustering to generate a visual word bag, and finally uses an SVM classifier to trace the origin of the fragments to determine whether they come from the same cultural relic. Secondly, in terms of type aggregation, the model uses a pre-trained deep learning model to extract deep features of the image, optimizes the clustering process through PCA (Principal Component Analysis, data dimensionality reduction), and uses the KMeans clustering algorithm to cluster cultural relic fragments to identify the specific regions or types they belong to (such as seals, landscapes, figures, and characters, etc.), providing accurate regional information for subsequent restoration work.

[0074] As Figure 3 shown, the implementation of the fragment image source classification module is mainly divided into two sub-function modules: fragment image traceability and type aggregation. The user only needs to input one picture, and the actual input into the system is the pre-processed picture (pre-processing includes: performing object recognition and background deduction on the input picture, only retaining the cultural relic fragments themselves).

[0075] As Figure 3 shown, in source classification, the fragment source classification module first extracts local features of the fragment image through SIFT and ORB algorithms. SIFT is a method of extracting image features by detecting key points (such as corner points, edges, etc.) in the image. The texture information around each key point will be converted into a 128-dimensional feature vector to represent the local features of the image. ORB first detects corner points in the image through the FAST (Features from Accelerated Segment Test) algorithm. FAST is an efficient corner point detection method that can quickly identify key points in the image. Then, ORB uses the BRIEF (Binary Robust Independent Elementary Features) algorithm to generate a binary feature descriptor for each key point. BRIEF generates a concise binary string by comparing image pixel pairs. The features generated in this way are both efficient and easy to calculate, suitable for large-scale image processing. At the same time, the ORB algorithm calculates a main direction for each key point, making it invariant to image rotation, which is particularly important for processing images at different angles and orientations.

[0076] The local features extracted by SIFT and ORB are then further processed, and the module uses the VLAD method to aggregate local features to generate global features. Specifically, MiniBatchKMeans (an optimized version of K-Means clustering) is first used to cluster local features to generate a set of "cluster centers" (also known as visual vocabulary). Each cluster center represents a "vocabulary" in the visual word bag, and all cluster centers together constitute the visual word bag (i.e., visual vocabulary), that is, each cluster center represents a feature in the image. Next, VLAD maps local features to the visual word bag. The specific processing process is: for each local feature, calculate its residual (difference) with each cluster center, and accumulate these residuals according to the cluster center. Finally, VLAD aggregates all local features into a global feature vector, representing the features of the entire image.

[0077] A vector machine classifier is determined, and a model training is performed on the vector machine classifier according to the global features of the plurality of images; and classification prediction processing is performed on the plurality of initial classification images by the trained vector machine classifier to obtain a plurality of source classification results.

[0078] The fragment source classification module supports classification using support vector machine (SVM) classifier. SVM is a machine learning method that can identify the relationship between image features and categories through training. In source classification, SVM is used to classify fragment images into multiple categories, perform binary or multi-classification, and determine whether these fragments may come from different cultural relics during the training phase. SVM is trained based on global features and corresponding labels (indicating different cultural relics). In the testing phase, the trained model is used to classify and predict the fragments to determine the cultural relics to which the fragments belong.

[0079] A pre-trained model is determined, and a deep feature extraction process is performed on a plurality of the global features of the images through the pre-trained model to obtain a plurality of image deep features; a principal component analysis method is used to perform dimensionality reduction and clustering processing on the plurality of the deep features of the images to obtain a plurality of type classification results.

[0080] like Figure 3As shown, in terms of type classification, the fragment source separation module uses multiple deep learning models (such as VGG16, VGG19, ResNet50, Xception, etc.). Which model to use specifically depends on the dataset. These models are loaded through the Keras framework and use the weights pre-trained on the ImageNet dataset. By using these pre-trained models, deep features can be efficiently extracted from images without having to train the model from scratch, which not only saves computing resources but also improves the classification accuracy. The feature extraction process first loads and resizes the image to a fixed size (224x224 pixels) because deep learning models usually require fixed-size inputs. Further, the image data is normalized by scaling the pixel values to between 0 and 1 to meet the input requirements of the deep learning model. Then, a pre-trained deep learning model (such as VGG19) is used to perform forward propagation on the image to extract the deep features of the image. At this time, the model only extracts the features of the convolutional layer and does not use the fully connected layer for classification. The convolutional layer can capture the spatial features and local patterns in the image and generate high-dimensional vectors, which represent the abstract features of the image. Among them, the extracted deep features are a high-dimensional vector containing the complex patterns and information in the image.

[0081] To reduce the computational complexity and optimize the clustering effect, the fragment source separation module then uses PCA (Principal Component Analysis) to reduce the dimension of the extracted deep features. PCA is a mathematical method that can transform high-dimensional data into low-dimensional data while retaining the main information of the data, reducing the computational complexity. The features after dimensionality reduction will be more conducive to the subsequent clustering process.

[0082] The clustering objects in the clustering process optimized by PCA dimensionality reduction are mainly the results of type classification. Here, the clustering is not for the results of source classification but for the extracted deep features. The specific process is as follows: The code will first perform PCA dimensionality reduction on the extracted deep features, use the PCA model to fit the feature data, and generate the feature representation after dimensionality reduction. The features after dimensionality reduction are more concise and can be used for subsequent K-Means clustering more efficiently.

[0083] In K-Means clustering, the deep features after dimensionality reduction are first clustered. The K-Means algorithm will divide all image features into a specified number of clusters. Each cluster represents a type or region, such as seals, landscapes, figures, and texts, etc., but it will not automatically identify the specific labels. The present invention will automatically classify them into different clusters according to the features of the images. Finally, after the clustering is completed, the code will copy the images to different folders according to the clustering results. Each folder corresponds to a cluster, which can provide a clear regional division for the subsequent repair work and facilitate further analysis.

[0084] Step S20: Perform damage repair processing on multiple sub-source compressed packages to obtain multiple initial damaged repaired cultural relic images.

[0085] The AI damage repair module includes: 1. Function: Perform edge extrapolation and content repair on fragmented or damaged images. 2. Input: Multiple individual fragmented pictures or a picture with internal damage. 3. Processing mode: Mode 1: Extrapolation with expansion mode (applicable to multiple fragments), extrapolate the edge of each fragment by 120 pixels (the extrapolated pixel size can be customized); use the edge repair model and content repair model to repair the extrapolated area and the fragments; output multiple repaired fragment images. Mode 2: Internal mask mode (applicable to a single damaged picture), call the object recognition and background subtraction API to fill the damaged area with a mask; use the edge repair model and content repair model to repair the masked area and the damaged picture. 4. Output: A complete repaired image.

[0086] Among them, the damage repair module adopts an encoder-decoder structure, fuses local and global features through the FFC module, uses dilated convolution and residual blocks in the middle layer to extract deep context information, and the decoder uses transposed convolution to restore the resolution to ensure structural integrity.

[0087] Specifically, determine the repair mode corresponding to multiple sub-source compressed packages, where the repair mode includes the extrapolation with expansion mode corresponding to multiple individual fragmented pictures and the internal mask mode corresponding to a single damaged picture; if the repair mode corresponding to multiple sub-source compressed packages is the extrapolation with expansion mode, perform pixel edge extrapolation processing on multiple sub-source compressed packages to obtain an edge extrapolation area map; if the repair mode corresponding to multiple sub-source compressed packages is the internal mask mode, perform mask filling processing on multiple sub-source compressed packages to obtain a masked area map.

[0088] The extrapolation band module generates an internal mask and an extrapolation band by identifying the damaged area. The mask marks the damaged area, and the extrapolation band extends the pixel information around the damaged area. Then, input the mask and the extrapolation band into the repair model, and the model uses this data for precise regional reconstruction and repair.

[0089] Process of generating the internal mask and the extrapolation band: First, call the object recognition and background subtraction API on the received image to eliminate the background. When the internal mask mode is selected, fill the transparent part inside the object with a white mask and record its position, and transmit the information to the damage repair module for repair. When the extrapolation band mode is selected, perform an operation of extrapolating 120 pixels on the transparent part outside the object, and fill it with a white mask and record its position, and transmit the information to the damage repair module for repair. Among them, the flowchart of the extrapolation band module is as Figure 4 shown.

[0090] Perform grayscale processing and image edge information extraction processing on the edge extrapolation region map or the mask region map to obtain an edge information map; use an edge repair model to perform edge repair processing on the edge information map to obtain an edge repair map; use a content repair model to perform color repair processing and content repair processing on the edge repair map to obtain multiple initial damaged and repaired cultural relic images.

[0091] The input of the AI damage repair module is the picture processed by the extrapolation band module, including the extended picture, the internal mask or the extrapolation band picture. After the AI damage repair module reads the area to be repaired in the picture (extrapolation band area or internal mask area), it first performs preprocessing. The specific process of preprocessing is: first grayscale the image, and then use the Canny edge detector to extract the image edge information. The edge information map will be input into the image of the edge repair model for edge repair. The repaired edge map will be input into the content repair model again for color and content repair. The final output is: an image that has been repaired and restored in the internal mask or extrapolation band area of the input image, and the repair effect is as Figure 5 shown.

[0092] The edge repair model is based on a generative adversarial network, and a part of the traditional convolution of the traditional generative adversarial network is replaced by the FFC fast Fourier convolution. The input of the model is two pictures, namely the image edge information map and the internal mask or extrapolation band area map.

[0093] Among them, the execution process of the encoder is as follows: the size of the input image is H×W×1, which is a single-channel grayscale image. The first layer uses a 7×7 convolution kernel, with a padding of 3 and a stride of 1. This operation keeps the input resolution unchanged (the output is H×W×32), but preliminarily extracts local features (such as edges, textures) through a larger convolution kernel, and at the same time expands the number of channels to 32.

[0094] The second layer uses FFC. This layer realizes downsampling through convolution with a stride of 2, halves the resolution to H / 2×W / 2, and doubles the number of channels to 64 at the same time. This step reduces the spatial dimension while retaining key features and reduces the computational amount.

[0095] The third layer continues to use FFC to further downsample to H / 4×W / 4, and the number of channels increases to 128. At this time, the spatial resolution of the feature map is only one-fourth of the original image, but it contains more abstract high-dimensional semantic information (such as object shape, structure).

[0096] Among them, the specific process of fast Fourier convolution is as follows: FFC (Fast Fourier Transform Convolution) extracts local and global information simultaneously by dividing the input channels into a local branch and a global branch. Specifically, the channels of the input tensor are symmetrically divided into two parts: the local branch receives the first C / 2 channels, while the global branch receives the last C / 2 channels. This division method ensures that the two branches process different levels of information in parallel while maintaining the balance of model parameters.

[0097] The operations of the local branch include two 4×4 convolutional kernels with padding 1 and stride 2 (Conv 4×4) for extracting local features. The global branch includes one 4×4 convolutional kernel with padding 1 and stride 2 (Conv 4×4) and a spectral transform. The core operation of the global branch is the spectral transform, and the specific steps are as follows: 1. Conv-BN-ReLU: First, the input tensor undergoes a conventional convolution operation (Conv), followed by batch normalization (BN) and the ReLU activation function (ReLU). The purpose of this step is to extract the local features of the input. 2. Real FFT2d: Next, the tensor after convolution and activation is applied with the real fast Fourier transform (Real FFT2d) to convert the spatial domain data into frequency domain data, generating a complex spectrum. Since the Fourier transform of a real signal has conjugate symmetry, only half of the spectrum information is retained. 3. Conv-BN-ReLU (frequency domain processing): The signal in the frequency domain is further processed through convolution (Conv), batch normalization (BN), and the ReLU activation function (ReLU) to process the features in the frequency domain. 4. Inv Real FFT2d: The corresponding recovery step for the frequency domain operation is to apply the inverse real fast Fourier transform (Inv Real FFT2d) to convert the frequency domain signal back to the spatial domain, which restores the spatial information of the image. 5. Merging operation: The results after Conv-BN-ReLU and the results after Inverse Real FFT2d will be merged in this step. 6. Conv 1x1: Finally, the merged results are further feature interacted through a 1x1 convolution. The 1x1 convolution is usually used for cross-channel feature learning and fusion, which can effectively improve the expression ability of the model.

[0098] In the merging stage, the results of the local branch and the global branch are merged separately. First, the first convolutional output of the local branch is merged with the 4×4 convolutional result of the global branch to generate Local features; then, the second convolutional output of the local branch is merged with the spectral transform result of the global branch to generate Global features. Both of them undergo a BN-ReLU process after merging, and finally form an image with rich local and global features.

[0099] For the middle layer process: The structure of the middle layer includes dilation convolution and residual blocks, aiming to enhance the feature extraction ability and handle large-scale missing regions. The working principle of dilation convolution is receptive field expansion: The key of dilation convolution lies in the "holes" or "gaps" within the convolutional kernel. In the 3×3 convolutional kernel set in the present invention, standard convolution accesses a 3×3 region during each calculation, while in dilation convolution, the convolutional kernel "skips" some elements during the convolution process according to the specified dilation rate, thereby expanding the receptive field. The dilation rate used in the present invention is 2, and the convolutional kernel skips one pixel between two adjacent elements, only considering the gap between two pixels, thus effectively expanding the receptive field.

[0100] The design of the residual block is to avoid the vanishing gradient problem and accelerate the training process through skip connections. In this structure, the residual block allows the network to learn the residual (i.e., the difference between the input and the output) instead of directly learning the output, making the training easier. As Figure 6 shown, the "residual block" adds the input and the output through skip connections to help the network train more efficiently and improve the expressive ability of features.

[0101] Combining dilation convolution and residual blocks, the network can extract more context information when dealing with large-scale missing regions. Dilation convolution captures larger spatial information by increasing the receptive field, while the residual block enhances the feature learning ability by increasing the depth of the network. Especially when the input data has missing or incomplete information, dilation convolution can restore more context features through the expansion of local information, and the residual block ensures that the network can effectively transmit and learn this information.

[0102] For the decoder process: The decoder part gradually restores the low-resolution features output by the middle layer to the original image size through three layers of transposed convolution. The first layer uses a 4×4 transposed convolutional kernel, padding 1, stride 2, and the input feature size is H / 4×W / 4×128 (the height and width are one-fourth of the original image, and the number of channels is 128). After upsampling, the output size becomes H / 2×W / 2×64 (the height and width are one-half of the original image, and the number of channels is 64). The main function of this step is to increase the resolution of the feature map to half of the original image and reduce the number of channels to reduce the computational complexity.

[0103] The second layer continues to use a 4×4 transposed convolutional kernel, with a padding of 1 and a stride of 2. The input size is H / 2×W / 2×64, and the output size is expanded to H×W×32 (restoring the original image resolution while further compressing the number of channels to 32). This layer completes the conversion from intermediate low-resolution features to the full image size, preparing for the subsequent generation of the final result by gradually adjusting the number of channels.

[0104] The third layer uses a larger 7×7 transposed convolutional kernel, with a padding of 3 and a stride of 1. The input size remains H×W×32, and the number of output channels is adjusted to 1 (generating a single-channel grayscale image). The role of the large-size convolutional kernel is to smooth the output through a wider receptive field, enhancing the consistency of image details (such as edge coherence and texture naturalness), and finally generating a high-quality result with the same resolution as the input image (H×W×1).

[0105] For the discriminator process: The discriminator only functions during the model training process and is not used in the actual production environment. The role of the discriminator is to make the model compete with the generator during training, enabling the generator to continuously improve the quality of the generated images. The specific process is as follows: The unbroken image and the repaired image are each passed through 5 convolutional layers and then compared. The goal is to output a probability value close to 1 for the unbroken image and a probability value close to 0 for the repaired image. The value output by the discriminator will affect the generation quality of the generator, thereby enabling the generator to generate images more similar to the unbroken image. The model architecture is as Figure 6 shown.

[0106] The input to the content repair model is the damaged image and the edge image repaired by the edge repair model. In the rough stage, first, a 5x5 convolution is used, with a padding of 2 and a stride of 1. The output size remains the same while the number of channels increases to H×W×32. Next, an FFC is used (except for some different parameters, the rest is the same as the above explanation of FFC). The convolutional parameter in it is a 3x3 convolution, with a padding of 1 and a stride of 2, which results in downsampling the size to H / 2×W / 2×64. Then, another FFC is used, with a 3x3 convolution, a padding of 1, and a stride of 2, further downsampling the size to H / 4×W / 4×128.

[0107] The middle layer still maintains the dilated convolution and residual block connections. In the refinement stage, the first transposed convolution uses a 3x3 transposed convolution, with a padding of 1, a stride of 2, and an output padding of 1. The output size is H / 2, and the number of channels is reduced from 128 to 64. The second transposed convolution is a 3x3 transposed convolution, with a padding of 1, a stride of 2, and an output padding of 1. The output size is H, and the number of channels is reduced from 64 to 32. Finally, the third transposed convolution is a 3x3 transposed convolution, with a padding of 1, a stride of 1, and the output size is restored to H×W×3, generating an RGB image. The model architecture is as Figure 7 shown.

[0108] Step S30: Perform fragment stitching on multiple of the initial damaged and restored cultural relic images to obtain a target damaged and restored cultural relic image.

[0109] The AI fragment stitching module includes: 1. Function: Stitch the restored fragment images to generate a complete image. 2. Input: Multiple fragment images processed by the AI damage restoration module. 3. Processing flow: Automatically stitch the fragments using the fragment stitching module; the stitching result is returned in JSON format, and the user can manually fine-tune it. 4. Output: The entire stitched image or the stitching result in JSON format for continued stitching use next time.

[0110] Specifically, perform feature matching processing on multiple of the initial damaged and restored cultural relic images through a feature matching model to obtain feature matching points; perform clustering screening processing and transformation matrix calculation processing on the feature matching points to obtain a target transformation matrix; perform perspective transformation processing and image fusion processing on the target transformation matrix to obtain a target damaged and restored cultural relic image.

[0111] Technical implementation of the fragment stitching module: The system first generates predicted content through extrapolation, and then combines the feature point recognition and puzzle-solving module for stitching. After stitching is completed, the user can optimize the result or make manual adjustments to ensure the repair effect is accurate and natural.

[0112] As Figure 8 shown, the fragment stitching module is mainly an improvement and adaptation of the open-source code and the model LoFTR-Stitch. During the entire image stitching process, first, feature points are extracted from two input images, and this process is completed through the LoFTR model. LoFTR is a deep learning-based feature matching model that can extract robust feature points from two images and perform matching. Pass the input images into the LoFTR model, and the present invention will process the images and extract the feature points in the two images, and store these feature points in the form of coordinates.

[0113] After the feature points are extracted and matched, these matching points need to be processed because the feature points in the image may not be completely accurate. Especially when there are incorrect matches, directly using these matching points for subsequent calculations will result in poor stitching effects. To improve the quality of the matching points, the present invention uses the kmeans algorithm to cluster the extracted matching points. Kmeans will divide these matching points into several groups and select the category with the most matching points, considering that the points in this category are the most reliable. Through clustering, some possible incorrect matching points can be filtered out, thus ensuring more accurate subsequent calculations.

[0114] After the feature points are screened, the next step is to estimate the transformation matrix, that is, to calculate the homography matrix. The homography matrix is a matrix that describes the transformation from one image to another image, and it can map the feature points in Image 1 to the corresponding positions in Image 2. In this step, the present invention uses the RANSAC algorithm to estimate the transformation matrix. The core idea of the RANSAC algorithm is to randomly select a small number of matching points (default 4 in the system) to calculate the transformation matrix, and then judge which points are correctly matched by calculating the degree of conformity of other matching points with this matrix (i.e., the reprojection error). Through multiple iterations, RANSAC can find the optimal transformation matrix and effectively remove the influence of mis-matched points.

[0115] When the present invention obtains an accurate transformation matrix through RANSAC, the next step is to perform perspective transformation on the image. The perspective transformation will map all points in Image 1 to the coordinate system of Image 2 according to the transformation matrix, thus achieving the alignment of the two images. In this step, the present invention uses the cv2.warpPerspective function (a function in the opencv library) to perform perspective transformation on the image. Through the transformation, the feature points of Image 1 will be correctly mapped to the corresponding positions in Image 2. At this time, if the input images cannot be stitched, the process of RANSAC estimating the transformation matrix will stop or fail.

[0116] After the image transformation is completed, the present invention obtains two aligned images, and then they need to be fused into a complete image. This process is called image fusion, and the present invention uses the method of weighted average to achieve image fusion. During the fusion process, if there is an overlapping area, the algorithm will calculate the weighted average according to the pixel values of the overlapping area to ensure a smooth transition at the stitching.

[0117] Furthermore, repeat the above process of stitching two fragments, match multiple fragments in pairs until the finally stitched fragments become more and more complete. The position information of the final stitching will be returned in the form of a JSON file and the stitching effect will be displayed on the front end.

[0118] Introduction to the open-source LoFTR (Local Feature Transformer): It is an image matching model based on deep learning, specially designed to extract and match local features in images. It mainly relies on the Transformer architecture, combining convolutional neural networks (CNNs) and self-attention mechanisms.

[0119] First, LoFTR receives two input images and preprocesses these two images. The image preprocessing process includes converting the images into a format suitable for network input and extracting local features in the images through a Convolutional Neural Network (CNN). This CNN part is responsible for extracting low-level feature information from the images, such as edges, corners, and textures, etc., laying the foundation for subsequent feature matching. Then, LoFTR inputs the feature maps of these two images into a shared Transformer module. The Transformer module models the features in the images through the self-attention mechanism. The self-attention mechanism enables each feature to be adjusted through global context information, thereby capturing the long-range dependencies between different features in the images. The core advantage of this mechanism is that it not only relies on local features but also can establish connections between features within the global scope of the image, thus improving the accuracy of image matching.

[0120] In the architecture of LoFTR, processing the features of these two images is completed through a two-stream network. After each image is respectively processed by the Convolutional Neural Network to extract features, the two feature maps will be jointly input into the Transformer module for processing. Through this two-stream network design, the model can process the two images in parallel and use the shared Transformer module to capture the similarity between the two images. This process helps LoFTR find the most matching feature point pairs in the images.

[0121] Next, LoFTR calculates the similarity of each pair of feature points in the images to generate the scores for feature point matching. The score of each pair of matching points reflects the level of similarity between them in the two images. Feature point pairs with higher scores are considered reliable matching points, while those with lower scores are considered incorrect matches. LoFTR filters the most likely correct matching point pairs through these matching scores.

[0122] Finally, the output of LoFTR includes the matching points in the two images and their matching confidence levels. This information can be used for further image stitching tasks. By calculating the transformation matrix for the matching points, LoFTR can help align the two images accurately and finally generate a seamlessly stitched image.

[0123] In addition, the auxiliary modules in the present invention include a super-resolution model, an object recognition model, and a background classification model. Among them, the super-resolution model: is used to perform super-resolution processing on the final result to improve the image quality. The object recognition model: is used to identify fragmented or damaged areas. The background classification model: is used to subtract the background and retain the fragments themselves.

[0124] The result of the splicing will be in the format of a JSON file, which will be used to splice the fragments on the canvas. Users can move, rotate, and scale the position of each fragment. Figure 9 shown.

[0125] Target database technology implementation: A target feature database was established for cultural relics images of different ages, types, and content styles, which significantly improved the model training effect.

[0126] It is understandable that for user usage modes, users can choose the following usage modes according to their needs: 1. Use a certain module alone: ​​Choose to use the "AI Fragment Source Sorting", "AI Damage Repair" or "AI Fragment Stitching" module according to specific needs. 2. Automated workflow mode: The system calls the three modules in sequence to complete the whole process from fragment classification to final splicing. 3. User intervention mode (optional): After the automated workflow outputs the splicing result, the user can manually fine-tune the splicing result (translation, rotation, scaling, etc.). 4. Super-resolution processing (optional): In any mode, the user can choose to perform super-resolution processing on the result.

[0127] Beneficial effects of the present invention:

[0128] 1. By improving the GAN model and introducing fast Fourier convolution (FFC), the inference speed is increased by 40%, and the structural similarity index after repair reaches 93.6%.

[0129] 2. Fragment sources are classified and types are grouped at the early stage of the repair process to effectively avoid splicing errors caused by misclassification of sources and significantly improve the accuracy and efficiency of repair.

[0130] 3. The system adopts a modular design, combining one-click operation with manual supervision. Based on AI matching prediction, it provides experts and technicians with a friendly interactive mode to meet the high-precision requirements of cultural relics splicing.

[0131] Technical effect details:

[0132] The overall system effects include: 1. Stitching accuracy: the interface offset error is controlled within 1.5mm; 2. Repair quality: the structural similarity index is 93.6%, and the variance is only 0.002; 3. Processing speed: single image edge repair is less than 0.5 seconds, and content repair is less than 30 seconds.

[0133] The effects brought by specific technical features: 1. FFC replaces traditional convolution: improves inference speed by 40% and effectively alleviates artifact problems; 2. Pre-classification of fragments: avoids splicing errors caused by misclassification and optimizes the overall repair effect.

[0134] Implementation scheme and variant scheme of the present invention:

[0135] 1. Standard implementation method: Runs based on NVIDIA GeForce GTX 3090Ti GPU, supports common image formats such as PNG and JPG, and the resolution is not less than 300 DPI.

[0136] 2. Possible deformation schemes: Supports the restoration of various types of cultural relics, including murals, paper cultural relics, fabrics, etc.; provides two deployment methods: cloud and local; the system design supports modular expansion, and function modules can be selected according to requirements.

[0137] The present invention proposes a content - led image stitching technology, filling the technical gap in the field of cultural relic restoration; uses FFC to replace traditional convolution, effectively alleviating the artifact problem, and improving the calculation efficiency and restoration effect; breaks through traditional edge matching and achieves high - precision stitching based on semantic content; combines one - key operation with manual supervision to improve the intuitiveness and friendliness of operation.

[0138] The technical features of the present invention are as follows: 1. Performance improvement: The inference speed is increased by 40%, the structural similarity index reaches 93.6%, and the variance is only 0.0020; 2. Precision guarantee: Supports the processing of large - resolution images, and the offset error of the stitching interface is less than 1.5 mm; 3. Strong practicability: Supports various types of cultural relics, such as murals, paper cultural relics, fabrics, etc.; 4. System stability: No performance degradation or crash during long - term operation (>8 hours).

[0139] The present invention classifies fragments at the initial stage of the restoration process, rather than the traditional manual classification, avoiding stitching errors caused by misclassification and significantly improving the accuracy and efficiency of restoration.

[0140] The present invention replaces traditional convolution with FFC to achieve a balance between local details and global styles. Compared with traditional generative adversarial networks, the inference speed is increased by 40%, the structural similarity index after restoration reaches 93.6%, and the variance of the structural similarity index is only 0.0020.

[0141] The present invention breaks through the traditional stitching method that only relies on edge shape and color, and significantly improves the stitching accuracy and stability in the scenario where the edges of fragment images are damaged.

[0142] Possible combination schemes other than the present invention's scheme include:

[0143] 1. Basic combination: Fragment source separation + content - led stitching, which can solve the problem of identifying mixed fragments, and at the same time provide accurate stitching results, and the offset error of the stitching interface is controlled within 1.5 mm.

[0144] 2. Efficient combination: FFC model + content - led stitching, while ensuring the restoration quality, significantly improves the processing efficiency, and the accuracy of the consistency evaluation of the generated result and the original reaches 93%. Figure 1 ​

[0145] 3. Complete combination: Fragment source separation + GAN-FFC model + content-dominated splicing, providing a full-process solution to achieve efficient and high-precision cultural relic restoration, supporting the restoration of multiple types of cultural relics, with the texture and color consistency reaching over 90%, and the system stability reaching 99.9%.

[0146] The alternative technical architecture of the present invention can be:

[0147] 1. Use the traditional U-Net architecture and ResNet pre-trained network to replace the GAN+FFC solution of the present invention, adopt a "margin-first" restoration strategy, first perform edge splicing, and then perform content restoration, avoiding the "three-step restoration method" of the present invention.

[0148] 2. Alternative processing flow: Introduce a two-stage solution of "manual rough classification + AI fine classification", avoid the fully automatic classification process, and adopt a traditional splicing algorithm based on edge shape and color to avoid prediction extrapolation technology.

[0149] 3. System architecture transformation: Split the functions of fragment source separation, damage repair, and AI splicing into independent microservices, and use a pure cloud deployment method to avoid the hardware dependence of local deployment.

[0150] 4. Application scope transformation: Focus on the restoration of a single type of cultural relic (such as only murals or paper cultural relics), avoid the general-purpose restoration solution, develop lightweight restoration equipment, and avoid high-performance computing requirements.

[0151] 5. Interaction method change: Adopt a pure manual supervision mode, reduce the degree of AI automation, provide a simplified version of the function module, and reduce the system complexity.

[0152] Furthermore, as Figure 10 shown, based on the above-mentioned restoration method for heterogeneous cultural relic fragments, the present invention also correspondingly provides a restoration system for heterogeneous cultural relic fragments, wherein the restoration system for heterogeneous cultural relic fragments includes:

[0153] A fragment source separation processing module 51, configured to obtain multiple images of cultural relic fragments to be restored of a cultural relic to be restored, and perform fragment source separation processing on the multiple images of cultural relic fragments to be restored to obtain multiple source-separated compressed packages;

[0154] A damage repair processing module 52, configured to perform damage repair processing on the multiple source-separated compressed packages to obtain multiple initial damaged repaired cultural relic images;

[0155] A fragment splicing processing module 53, configured to perform fragment splicing processing on the multiple initial damaged repaired cultural relic images to obtain a target damaged repaired cultural relic image.

[0156] Furthermore, asFigure 11 As shown, based on the above-mentioned method and system for restoring heterogeneous cultural relic fragments, the present invention also correspondingly provides a terminal, which includes a processor 10, a memory 20, and a display 30. Figure 11 Only some components of the terminal are shown, but it should be understood that it is not required to implement all the shown components, and more or fewer components can be alternatively implemented.

[0157] In some embodiments, the memory 20 may be an internal storage unit of the terminal, such as the hard disk or memory of the terminal. In some other embodiments, the memory 20 may also be an external storage device of the terminal, such as a plug-in hard disk equipped on the terminal, a Smart Media Card (SMC), a Secure Digital (SD) card, a Flash Card, etc. Further, the memory 20 may also include both the internal storage unit and the external storage device of the terminal. The memory 20 is used to store application software installed on the terminal and various types of data, such as the program code for installing the terminal. The memory 20 may also be used to temporarily store data that has been output or will be output. In one embodiment, a program 40 for restoring heterogeneous cultural relic fragments is stored on the memory 20, and this program 40 for restoring heterogeneous cultural relic fragments can be executed by the processor 10, thereby implementing the method for restoring heterogeneous cultural relic fragments in the present application.

[0158] In some embodiments, the processor 10 may be a Central Processing Unit (CPU), a microprocessor, or other data processing chips, and is used to run the program code stored in the memory 20 or process data, such as executing the method for restoring heterogeneous cultural relic fragments, etc.

[0159] In some embodiments, the display 30 may be an LED display, a liquid crystal display, a touch liquid crystal display, and an OLED (Organic Light-Emitting Diode) toucher, etc. The display 30 is used to display information on the terminal and to display a visual user interface.

[0160] In one embodiment, when the processor 10 executes the program 40 for restoring heterogeneous cultural relic fragments in the memory 20, the steps of the method for restoring heterogeneous cultural relic fragments as described above are implemented.

[0161] The present invention also provides a computer-readable storage medium, wherein the computer-readable storage medium stores a program for restoring heterogeneous cultural relic fragments, and when the program for restoring heterogeneous cultural relic fragments is executed by a processor, the steps of the method for restoring heterogeneous cultural relic fragments as described above are implemented.

[0162] In summary, the present invention provides a method, a system, a terminal and a computer-readable storage medium for repairing heterogeneous cultural relic fragments. The method includes: obtaining a plurality of images of cultural relic fragments to be repaired of a cultural relic to be repaired, and performing source separation processing on the plurality of images of cultural relic fragments to be repaired to obtain a plurality of source-separated compressed packages; performing damage repair processing on the plurality of source-separated compressed packages to obtain a plurality of initial damaged-repaired cultural relic images; and performing fragment splicing processing on the plurality of initial damaged-repaired cultural relic images to obtain a target damaged-repaired cultural relic image. By performing source separation processing on the images of cultural relic fragments to be repaired, the present invention can effectively avoid splicing errors caused by incorrect source separation, and thus significantly improve the accuracy and efficiency of cultural relic repair. Further, by performing damage repair processing on the cultural relic fragments after source separation processing, the present invention can extract the features of the cultural relic fragments, which helps to perform fragment splicing on the cultural relic fragments subsequently, and effectively improves the repair quality and repair effect of cultural relic fragments under the conditions of a large number and heterogeneity.

[0163] It should be noted that in this article, the terms "include", "comprise" or any other variant thereof are intended to cover non-exclusive inclusion, so that a process, method, article or terminal including a series of elements not only includes those elements but also includes other elements not expressly listed, or further includes elements inherent to such process, method, article or terminal. Without further limitation, an element defined by the phrase "including one..." does not exclude the existence of additional identical elements in the process, method, article or terminal including that element.

[0164] Of course, those of ordinary skill in the art can understand that all or part of the processes of implementing the above method embodiments can be completed by instructing relevant hardware (such as a processor, a controller, etc.) through a computer program. The program can be stored in a computer-readable storage medium readable by a computer. When the program is executed, it can include the processes of the above method embodiments. The computer-readable storage medium can be a memory, a magnetic disk, an optical disc, etc.

[0165] It should be understood that the application of the present invention is not limited to the above examples. For those of ordinary skill in the art, improvements or transformations can be made according to the above description. All such improvements and transformations should fall within the protection scope of the appended claims of the present invention.

Claims

1. A method for repairing foreign cultural relic fragments, characterized in that: The method for repairing the heterologous cultural relic fragments comprises: Acquire multiple fragment images of the cultural relic to be restored, and perform fragment source separation processing on the multiple fragment images of the cultural relic to be restored to obtain multiple source separation compressed packages; Performing damage repair processing on the plurality of sub-source compressed packages to obtain a plurality of initial damaged and repaired cultural relic images; A plurality of the initial damaged and restored cultural relic images are subjected to fragment splicing processing to obtain a target damaged and restored cultural relic image.

2. The method for repairing foreign cultural relic fragments according to claim 1, characterized in that: The fragment source classification processing includes source classification processing and type classification processing; the source classification compression package includes source classification results and type classification results; The step of obtaining a plurality of fragment images of the cultural relic to be restored and performing fragment source separation processing on the plurality of fragment images of the cultural relic to be restored to obtain a plurality of source separation compressed packages specifically includes: Acquiring a plurality of fragment images of the cultural relic to be restored, and preprocessing the plurality of fragment images of the cultural relic to be restored to obtain a plurality of global features of the images; Performing source classification processing on a plurality of the global features of the images to obtain a plurality of source classification results; Type classification processing is performed on the multiple global features of the image to obtain multiple type classification results.

3. The method for repairing foreign cultural relic fragments according to claim 2, characterized in that: The preprocessing of the plurality of images of the cultural relic fragments to be restored to obtain a plurality of image global features specifically includes: Performing object recognition processing and background subtraction processing on the plurality of images of the cultural relic fragments to be restored to obtain a plurality of initial classification images; Performing local feature extraction processing on the multiple initial classified images to obtain multiple image local features; A feature aggregation process is performed on the multiple local features of the image to obtain multiple global features of the image.

4. The method for repairing foreign cultural relic fragments according to claim 3, characterized in that: The performing source classification processing on the plurality of the global features of the images to obtain a plurality of source classification results specifically includes: Determine a vector machine classifier, and perform model training on the vector machine classifier according to a plurality of the global features of the image; The trained vector machine classifier is used to perform classification prediction processing on the plurality of initial classification images to obtain a plurality of source classification results.

5. The method for repairing foreign cultural relic fragments according to claim 2, characterized in that: The performing type classification processing on the multiple global features of the image to obtain multiple type classification results specifically includes: Determine a pre-trained model, and perform deep feature extraction processing on a plurality of the global features of the image through the pre-trained model to obtain a plurality of image deep features; The principal component analysis method is used to perform dimensionality reduction and clustering processing on the multiple image depth features to obtain multiple type classification results.

6. The method for repairing foreign cultural relic fragments according to claim 1, characterized in that: The damage repair process is performed on the plurality of source compressed packages to obtain a plurality of initial damaged and repaired cultural relic images, specifically comprising: Determine the repair modes corresponding to the plurality of sub-source compressed packages, wherein the repair modes include an extrapolation band extension mode corresponding to a plurality of separate fragmented images and an internal mask mode corresponding to a single damaged image; If the repair mode corresponding to the plurality of sub-source compressed packages is the extrapolation band expansion mode, pixel edge extrapolation processing is performed on the plurality of sub-source compressed packages to obtain an edge extrapolation region map; If the repair mode corresponding to the plurality of sub-source compressed packages is the internal mask mode, performing mask filling processing on the plurality of sub-source compressed packages to obtain a mask area map; Performing grayscale processing and image edge information extraction processing on the edge extrapolation region map or the mask region map to obtain an edge information map; Performing edge repair processing on the edge information graph using an edge repair model to obtain an edge repair graph; The content restoration model is used to perform color restoration processing and content restoration processing on the edge restoration image to obtain multiple initial damaged and restored cultural relics images.

7. The method for repairing foreign cultural relic fragments according to claim 1, characterized in that: The fragment splicing process of the plurality of initial damaged and repaired cultural relic images to obtain a target damaged and repaired cultural relic image specifically includes: Performing feature matching processing on the plurality of initial damaged and repaired cultural relics images through a feature matching model to obtain feature matching points; Performing clustering screening processing and transformation matrix calculation processing on the feature matching points to obtain a target transformation matrix; The target transformation matrix is ​​subjected to perspective transformation processing and image fusion processing to obtain the target damaged and repaired cultural relic image.

8. A system for repairing fragments of cultural relics of different origins, characterized in that: The repair system for the heterologous cultural relic fragments comprises: A fragment source separation processing module is used to obtain a plurality of fragment images of the cultural relic to be restored, and perform fragment source separation processing on the plurality of fragment images of the cultural relic to be restored to obtain a plurality of source separation compressed packages; A damage repair processing module is used to perform damage repair processing on the plurality of sub-source compressed packages to obtain a plurality of initial damage repaired cultural relic images; The fragment splicing processing module is used to perform fragment splicing processing on the multiple initial damaged and restored cultural relic images to obtain a target damaged and restored cultural relic image.

9. A terminal, characterized in that: The terminal includes: a memory, a processor, and a repair program for heterogeneous cultural relic fragments stored in the memory and executable on the processor. When the repair program for heterogeneous cultural relic fragments is executed by the processor, the steps of the method for repairing heterogeneous cultural relic fragments according to any one of claims 1 to 7 are implemented.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a repair program for heterogeneous cultural relic fragments, and when the repair program for heterogeneous cultural relic fragments is executed by a processor, the steps of the method for repairing heterogeneous cultural relic fragments according to any one of claims 1 to 7 are implemented.

Images