An interactive repair method and system for cultural relics

Abstract

The application provides an interactive repair method and system for cultural relics, which comprises the following steps: through an augmented reality terminal, in-situ three-dimensional scanning of a damaged cultural relic is performed to obtain a three-dimensional geometric model, surface texture information and spatial position coordinates of the damaged cultural relic, and a cultural relic prior knowledge base is constructed; constraint conditions corresponding to the damaged cultural relic are generated according to the cultural relic prior knowledge base, and a constraint-driven three-dimensional generative model is adopted, with the three-dimensional geometric model and the surface texture information as input; in the generation process, the constraint conditions are sampled for checking and guiding, a plurality of repair assumption schemes conforming to the constraint conditions are generated in parallel, and a large language model is called to automatically extract the basis literature and reference examples of each repair assumption scheme to generate a repair reason statement; through augmented reality technology, the repair scheme and the repair reason statement are superimposed on the damaged cultural relic based on the spatial position coordinates to complete the repair of the damaged cultural relic. The application can effectively improve the repair efficiency of cultural relics.

Description

Technical Field

[0001] This invention relates to the field of cultural relic restoration technology, and in particular to an interactive method and system for cultural relic restoration. Background Technology

[0002] With the advancement of science and technology and the rapid development of the times, people have made significant progress in the field of archaeology. Among these advancements, people are now able to repair defective cultural relics, thereby effectively restoring their original appearance.

[0003] Cultural relic restoration is a complex task that combines science, art, and history. Traditional restoration processes rely heavily on the personal experience of experts, which means that the quality of the restoration depends entirely on the expert's subjective judgment.

[0004] Furthermore, with the continuous development of computer technology, people are now able to perform digital restoration on computers. However, this method cannot allow experts to intuitively perceive the restoration plan in a real environment from any angle, thus failing to complete the corresponding restoration operations objectively and accurately, which reduces the efficiency of cultural relic restoration. Summary of the Invention

[0005] Based on this, the purpose of this invention is to provide an interactive method and system for the restoration of cultural relics, so as to solve the problem that the existing technology cannot allow staff to intuitively feel the restoration plan in a real environment from any angle, which leads to a reduction in the efficiency of cultural relic restoration.

[0006] The first aspect of the present invention proposes:

[0007] An interactive restoration method for cultural relics, wherein the method includes:

[0008] The damaged cultural relics are scanned in an on-site 3D manner using an augmented reality terminal to obtain the 3D geometric model, surface texture information and spatial coordinates of the damaged cultural relics, and to construct a prior knowledge base of the cultural relics.

[0009] Constraints corresponding to the damaged cultural relics are generated based on the prior knowledge base of cultural relics, and a constraint-driven three-dimensional generative model is adopted, with the three-dimensional geometric model and the surface texture information as input;

[0010] During the generation process, the constraints are sampled for verification and guidance to generate multiple repair hypothesis schemes that meet the constraints in parallel. A large language model is called to automatically extract the supporting literature and reference examples for each repair hypothesis scheme to generate a repair rationale.

[0011] The repair scheme and the repair reason are superimposed on the damaged cultural relic based on the spatial position coordinates through an augmented reality technology to complete the repair of the damaged cultural relic.

[0012] The application has the beneficial effects that: through three-dimensional scanning of the damaged cultural relic, corresponding three-dimensional aggregate models, surface texture information and spatial position coordinates can be obtained, based on which, in order to facilitate subsequent identification and repair, repair hypothesis schemes are generated according to the current three parameters and the obtained constraint conditions for verification and guidance, and adaptive repair reason statements are generated, which are then superimposed on the damaged cultural relic to complete corresponding repair processing, so that the staff can complete the corresponding repair in a real environment, and various repair schemes can be obtained, which correspondingly improve the cultural relic repair efficiency.

[0013] Further, the step of in-situ three-dimensional scanning of the damaged cultural relic through an augmented reality terminal to obtain the three-dimensional geometric model, the surface texture information and the spatial position coordinates of the damaged cultural relic comprises:

[0014] The multi-view image sequence of the damaged cultural relic is collected through a preset camera, and pose tracking is performed in combination with a SLAM algorithm;

[0015] The multi-view image sequence is fused with depth information through a multi-view stereo reconstruction algorithm to generate corresponding target point cloud data, and the three-dimensional geometric model, the surface texture information and the spatial position coordinates are correspondingly generated according to the target point cloud data.

[0016] Further, the step of correspondingly generating the three-dimensional geometric model, the surface texture information and the spatial position coordinates according to the target point cloud data comprises:

[0017] A three-dimensional mesh model corresponding to the damaged cultural relic is generated from the target point cloud data through a Poisson reconstruction algorithm, and the three-dimensional mesh model is texture-mapped to generate the three-dimensional geometric model and the surface texture information;

[0018] Based on a surface integrity analysis and a threshold segmentation algorithm, the damaged area of the damaged cultural relic is automatically identified and the boundary curve is extracted to correspondingly identify the spatial position coordinates of the damaged area.

[0019] Further, the step of sampling the constraint conditions for verification and guidance during the generation process comprises:

[0020] A diffusion model is used as a basic generation architecture, and a constraint embedding vector is introduced in each denoising step during the generation process;

[0021] A constraint consistency verification module is used to verify whether the generation result meets the constraint condition;

[0022] If the constraint consistency check module verifies that the generated result does not meet the constraints, constraint-guided optimization is performed to adjust the generation direction.

[0023] Furthermore, the step of generating multiple repair hypothesis schemes that meet the constraints in parallel includes:

[0024] Parallel sampling using different random seeds is used to generate multiple candidate repair schemes;

[0025] Each candidate repair scheme is subjected to multi-dimensional constraint verification to output a comprehensive score corresponding to each candidate repair scheme, and the repair hypothesis scheme is selected based on the comprehensive score.

[0026] Furthermore, the step of selecting the repair hypothesis scheme based on the comprehensive score includes:

[0027] The candidate repair schemes are sorted according to the comprehensive score, and the candidate repair schemes with the highest preset ranking are selected as the repair hypothesis schemes.

[0028] The large language model is invoked to generate a corresponding explanation of the repair rationale for each repair hypothesis based on the constraint matching results.

[0029] Furthermore, the step of overlaying the repair plan and the rationale for repair onto the damaged cultural relic using augmented reality technology based on the spatial coordinates to complete the repair of the damaged cultural relic includes:

[0030] The three-dimensional geometric model is aligned with the damaged cultural relic based on feature point matching and PnP algorithm according to the spatial coordinates.

[0031] Ambient lighting is estimated using spherical harmonic functions, and realistic rendering effects are achieved by combining them with a physical rendering pipeline. This allows the restoration plan and the rationale for restoration to be superimposed onto the damaged cultural relic, thus completing the corresponding restoration process.

[0032] The second aspect of the present invention proposes:

[0033] An interactive cultural relic restoration system, wherein the system comprises:

[0034] The scanning module is used to perform on-site three-dimensional scanning of the damaged cultural relics through an augmented reality terminal, so as to obtain the three-dimensional geometric model, surface texture information and spatial location coordinates of the damaged cultural relics, and to construct a prior knowledge base of the cultural relics.

[0035] a generating module configured to generate constraint conditions corresponding to the damaged cultural relic according to the prior knowledge base of cultural relics, and to generate a three-dimensional generative model driven by the constraint conditions, with the three-dimensional geometric model and the surface texture information as inputs;

[0036] an extracting module configured to sample the constraint conditions for verification and guidance during the generation process, to generate multiple repair hypothesis schemes in parallel that meet the constraint conditions, and to call a large language model to automatically extract literature and reference examples for each repair hypothesis scheme to generate repair reasons;

[0037] a repairing module configured to superimpose the repair scheme and the repair reasons onto the damaged cultural relic based on the spatial position coordinates through augmented reality technology to complete the repair of the damaged cultural relic.

[0038] Further, the scanning module is specifically configured to:

[0039] acquire a multi-view image sequence of the damaged cultural relic through a preset camera and perform pose tracking in combination with a SLAM algorithm;

[0040] fuse the multi-view image sequence with depth information through a multi-view stereo reconstruction algorithm to generate corresponding target point cloud data, and generate the three-dimensional geometric model, the surface texture information, and the spatial position coordinates according to the target point cloud data.

[0041] Further, the scanning module is specifically configured to:

[0042] generate a three-dimensional mesh model corresponding to the damaged cultural relic according to the target point cloud data through a Poisson reconstruction algorithm, and perform texture mapping on the three-dimensional mesh model to generate the three-dimensional geometric model and the surface texture information;

[0043] automatically identify a damaged area of the damaged cultural relic and extract a boundary curve based on a surface integrity analysis and a threshold segmentation algorithm to correspondingly identify the spatial position coordinates of the damaged area.

[0044] Further, the extracting module is specifically configured to:

[0045] use a diffusion model as a basic generation architecture and introduce a constraint embedding vector at each denoising step during the generation process;

[0046] verify whether the generation result meets the constraint conditions through a constraint consistency verification module;

[0047] if it is verified through the constraint consistency verification module that the generation result does not meet the constraint conditions, perform constraint-guided optimization to adjust the generation direction.

[0048] Further, the extraction module is specifically used for:

[0049] Parallel sampling is performed using different random seeds to generate multiple candidate repair schemes;

[0050] Each of the candidate repair schemes is subjected to multi-dimensional constraint verification to output a comprehensive score corresponding to each of the candidate repair schemes, so that the repair hypothesis scheme is screened out according to the comprehensive score.

[0051] Further, the extraction module is specifically used for:

[0052] According to the comprehensive score, each of the candidate repair schemes is sorted, and a candidate repair scheme with a front preset rank is selected as the repair hypothesis scheme;

[0053] The large language model is called to generate a corresponding repair reason elaboration for each of the repair hypothesis schemes based on the constraint matching result.

[0054] Further, the repair module is specifically used for:

[0055] Feature point matching and PnP algorithm are adopted to align the three-dimensional geometric model with the damaged cultural relics according to the spatial position coordinates;

[0056] The spherical harmonic function is used to estimate the ambient light, and the physical rendering pipeline is used to realize the real rendering effect, so that the repair scheme and the repair reason elaboration are superimposed on the damaged cultural relics to complete the corresponding repair processing.

[0057] The third aspect of the embodiment of the present application provides:

[0058] A computer comprises a memory, a processor, and a computer program stored on the memory and executable on the processor, wherein the processor implements the cultural relic interactive repair method as described above when executing the computer program.

[0059] The fourth aspect of the embodiment of the present application provides:

[0060] A readable storage medium has a computer program stored thereon, wherein the program is executed by a processor to implement the cultural relic interactive repair method as described above.

[0061] Additional aspects and advantages of the present application will be in part apparent and in part pointed out hereinafter. BRIEF DESCRIPTION OF DRAWINGS

[0062] Figure 1 A flowchart of the cultural relic interactive repair method provided by the first embodiment of the present application is shown;

[0063] Figure 2 A structural block diagram of an interactive repair system for cultural relics provided by a third embodiment of the present application is shown in FIG. 3.

[0064] The following detailed description will further describe the present application in conjunction with the above-mentioned drawings. DETAILED DESCRIPTION

[0065] In order to facilitate the understanding of the present application, the present application will be described more fully below with reference to the accompanying drawings. The present application is shown in several embodiments in the drawings. However, the present application can be realized in many different forms and is not limited to the embodiments described herein. Rather, these embodiments are provided so that the disclosure of the present application is more thorough and complete.

[0066] It should be noted that when an element is referred to as being "fixedly attached" to another element, it can be directly on the other element or there can be intervening elements. When an element is referred to as being "connected" or "coupled" to another element, it can be directly connected or coupled to the other element or intervening elements can be present. As used herein the terms "vertical", "horizontal", "left", "right", and the like are merely used for the purpose of illustration.

[0067] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application belongs. The terminology used in the description of the application herein is for the purpose of describing particular embodiments only and is not intended to be limiting of the application. As used herein, the term "and / or" includes any and all combinations of one or more of the associated listed items.

[0068] Referring to FIG. 1, an interactive repair method for cultural relics provided by a first embodiment of the present application is shown. The interactive repair method for cultural relics provided by the present embodiment enables the staff to complete the repair of cultural relics in a real environment, thereby improving the efficiency of cultural relic repair. Figure 1

[0069] Specifically, the present embodiment provides:

[0070] An interactive repair method for cultural relics, wherein the method comprises:

[0071] In step S10, the damaged cultural relic is in-situ three-dimensionally scanned by the augmented reality terminal to obtain a three-dimensional geometric model, surface texture information and spatial position coordinates of the damaged cultural relic, and a cultural relic priori knowledge base is constructed.

[0072] ​It should be noted that, first of all, the damaged cultural relics are scanned by using the augmented reality terminal, specifically, unlike traditional laboratory scanning, "in-situ" means that scanning is completed in the original preservation environment of the cultural relics, avoiding secondary damage caused by moving the cultural relics, while preserving the spatial relationship between the cultural relics and the surrounding environment. The scanning process will obtain three types of key data: three-dimensional geometric model (reflecting the three-dimensional structure of the cultural relics), surface texture information (including decoration, color, material details) and spatial position coordinates (the specific orientation of the cultural relics in the real space). The "cultural relics prior knowledge base" constructed synchronously integrates the historical background of this type of cultural relics, the data of complete cultural relics of the same type, repair specifications and other information, providing a basis for subsequent repair. To facilitate subsequent processing.

[0073] Step S20, generating constraint conditions corresponding to the damaged cultural relics according to the cultural relics prior knowledge base, and using a constraint-driven three-dimensional generative model, taking the three-dimensional geometric model and the surface texture information as input;

[0074] It should be noted that, based on the prior knowledge base, "constraint conditions" corresponding to the damaged cultural relics are generated (for example: the costume decoration style of Tang dynasty pottery figurines, the casting process characteristics of bronze wares, etc.), and a "constraint-driven three-dimensional generative model" is used, specifically, the model takes the three-dimensional geometric model (basic structure of the damaged part) and surface texture information (style reference) obtained by scanning as input, to ensure that the generated repair content is consistent with the original characteristics of the cultural relics. To facilitate subsequent processing.

[0075] Step S30, sampling the constraint conditions for verification and guidance during the generation process, to generate multiple repair hypothesis schemes that meet the constraint conditions in parallel, and calling a large language model to automatically extract the basis literature and reference examples of each repair hypothesis scheme to generate repair reason elaboration;

[0076] It should be noted that, during the model generation process, the constraint conditions are continuously sampled for verification (to determine whether the repair part meets the historical characteristics) and guidance (to correct the generation direction deviating from the constraints), multiple repair hypothesis schemes are generated in parallel (to provide diversified choices); at the same time, a large language model is called to automatically extract the basis literature (such as archaeological reports) and reference examples (such as complete cultural relics of the same type) of each scheme from the prior knowledge base, to generate "repair reason elaboration", enhancing the credibility and explainability of the scheme. To facilitate subsequent processing.

[0077] Step S40, based on the spatial position coordinates, superimposing the repair scheme and the repair reason elaboration onto the damaged cultural relics by using augmented reality technology, to complete the repair of the damaged cultural relics.

[0078] It should be noted that finally, based on the spatial position coordinates obtained by scanning, the repair scheme (virtual complete form) and the repair reason (text or voice explanation) are accurately superimposed on the real damaged cultural relics through augmented reality technology. Specifically, the user can directly see the repair effect on the cultural relic entity, realize the interactive verification of "virtual and real integration", and finally complete the repair decision and implementation. The whole process takes into account the scientificity (based on historical data and constraints), safety (on-site operation) and interactivity (visual verification) of cultural relic repair, and solves the problems of "invisible preview" and "single scheme" in traditional repair. In order to facilitate subsequent processing.

[0079] Second embodiment

[0080] Further, the step of scanning the damaged cultural relics through the augmented reality terminal to obtain the three-dimensional geometric model, surface texture information and spatial position coordinates of the damaged cultural relics comprises:

[0081] The multi-view image sequence of the damaged cultural relics is collected by a preset camera, and pose tracking is performed in combination with a SLAM algorithm.

[0082] The multi-view image sequence is fused with depth information by using a multi-view stereo reconstruction algorithm to generate corresponding target point cloud data, and the three-dimensional geometric model, the surface texture information and the spatial position coordinates are generated according to the target point cloud data.

[0083] It should be noted that first, the "multi-view image sequence" of the damaged cultural relics is collected by a preset camera (such as a depth camera or a multi-lens camera mounted on an AR terminal). Specifically, the cultural relics are photographed from different angles and distances, covering the damaged and complete areas, to provide sufficient visual information for subsequent three-dimensional reconstruction. Since the cultural relics may be in a complex environment (such as a museum showcase or an archaeological site), and the terminal may move during scanning, pose tracking needs to be performed in combination with a "SLAM (simultaneous localization and mapping) algorithm". SLAM can calculate the position and attitude changes of the camera in real time, ensure that the multi-view images are aligned in the same coordinate system, and avoid model distortion caused by shooting angle deviation.

[0084] After acquiring the aligned multi-view images, a multi-view stereo reconstruction algorithm is used to fuse the depth information. Specifically, the algorithm matches the same named points in different images to calculate the three-dimensional spatial coordinates of the pixel points and generate target point cloud data (a discrete representation of the cultural relic surface composed of a large number of three-dimensional points). The point cloud data contains the geometric shape and spatial position information of the cultural relic, based on which a three-dimensional geometric model (a continuous surface structure), surface texture information (color and decoration attached to the geometric model), and spatial position coordinates (positioning of the point cloud in the real world coordinate system) can be further generated. This step is the basis for subsequent repair, and its accuracy directly affects the matching degree of the repair scheme and the cultural relic entity. This facilitates subsequent processing.

[0085] Further, the step of generating the three-dimensional geometric model, the surface texture information, and the spatial position coordinates corresponding to the target point cloud data comprises:

[0086] generating a three-dimensional mesh model corresponding to the damaged cultural relic according to the target point cloud data by a Poisson reconstruction algorithm, and performing texture mapping on the three-dimensional mesh model to generate the three-dimensional geometric model and the surface texture information;

[0087] automatically identifying the damaged area of the damaged cultural relic and extracting the boundary curve based on a surface integrity analysis and a threshold segmentation algorithm to correspondingly identify the spatial position coordinates of the damaged area.

[0088] It should be noted that, first, for the target point cloud data, a Poisson reconstruction algorithm is used to generate a three-dimensional mesh model: this algorithm constructs a continuous surface by solving the Poisson equation, can handle dense point clouds and generate smooth and complete mesh structures, and better preserves the curved surface features of cultural relics (such as the curvature of pottery and the outline of jade). Subsequently, the color, decoration, and other information in the multi-view images are attached to the surface of the mesh model through texture mapping technology to form a three-dimensional geometric model and surface texture information that have both geometric shape and surface details. Specifically, this step ensures that the digital model not only looks like the original, but also restores the visual features of the cultural relic.

[0089] Meanwhile, to accurately locate the part to be repaired, the surface integrity analysis (judging whether the surface of the cultural relic is missing or broken) and the threshold segmentation algorithm (based on the density and curvature of the point cloud) are used to automatically identify the damaged area: for example, the point cloud density at the broken part will suddenly decrease, and this area can be screened by setting a threshold, and the "boundary curve" (the connecting edge between the damaged part and the complete part) is extracted. Based on the spatial coordinates of the boundary curve, the "spatial position coordinates" of the damaged area can be determined, which provides a clear repair target area for the subsequent generative model, ensuring accurate docking of the repair content with the damaged boundary and avoiding over-repair or insufficient repair. To facilitate subsequent processing.

[0090] Further, the step of sampling the constraint condition for verification and guidance during the generation process includes:

[0091] A diffusion model is used as the basic generation architecture, and a constraint embedding vector is introduced at each denoising step during the generation process.

[0092] The generated result is verified by a constraint consistency verification module to determine whether it meets the constraint condition.

[0093] If the generated result does not meet the constraint condition as verified by the constraint consistency verification module, a constraint guidance optimization is performed to adjust the generation direction.

[0094] It should be noted that a diffusion model is used as the basic generation architecture: the diffusion model generates high-quality three-dimensional structures by gradually denoising, which is suitable for the progressive generation requirement of "from damage to completeness" in cultural relic restoration. In each denoising step of the model generation, a "constraint embedding vector" is introduced, which specifically converts the above constraint conditions (such as decorative style, structural proportion) into a vector form that the model can understand, so that the generation process is always centered around the constraints (for example, the symmetry feature of the Tang dynasty lotus pattern will guide the model to generate a symmetrical decorative repair part).

[0095] To verify whether the generated result meets the constraints, a "constraint consistency verification module" is set up: this module compares the generated content with the constraint features (such as the line thickness of the decoration and the mechanical stability of the structure) in the prior knowledge base to determine whether there is a deviation (for example, if the proportion of the bronze handle exceeds the normal range of similar cultural relics, it is determined to be inconsistent). If the verification finds that the generated result does not meet the constraint condition, immediately perform "constraint guidance optimization": adjust the generation parameters of the model (such as denoising intensity and feature weight) to correct the generation direction until the result meets the constraint. This process forms a closed loop of "generation - verification - optimization", avoiding the model generating repair schemes that are not in line with the characteristics of cultural relics. To facilitate subsequent processing.

[0096] Further, the step of generating multiple repair hypothesis schemes in parallel that meet the constraint conditions comprises:

[0097] Parallel sampling using different random seeds to generate multiple candidate repair schemes;

[0098] Each of the candidate repair schemes is subjected to multidimensional constraint verification to output a comprehensive score corresponding to each of the candidate repair schemes, and the repair hypothesis scheme is selected according to the comprehensive score.

[0099] It should be noted that first, multiple candidate repair schemes are generated by "parallel sampling using different random seeds": a random seed is the initial parameter of a generative model, and different seeds will guide the model to explore different detailed possibilities (for example, the same damaged decoration may generate two patterns slightly different but both conforming to the style of the era) under the premise of meeting the constraints, realizing the diversity of the scheme.

[0100] To ensure the quality of the candidate schemes, each scheme needs to be subjected to "multidimensional constraint verification": the dimensions include the matching degree with the prior knowledge base (such as whether it conforms to the characteristics of the cultural relics of the era), the physical rationality of the structure (such as whether the repaired part can stably connect with the damaged boundary), the consistency of the texture (such as whether the color of the repaired area is coordinated with the original surface), etc. According to the verification results of each dimension, a "comprehensive score" is generated by weighted calculation (the higher the score, the better the scheme). Finally, the "repair hypothesis scheme" is selected according to the comprehensive score. Specifically, a certain number of alternative schemes are retained, and each scheme meets the basic repair requirements, providing a reliable basis for subsequent expert decision-making.

[0101] Further, the step of selecting the repair hypothesis scheme according to the comprehensive score comprises:

[0102] According to the comprehensive score, each of the candidate repair schemes is sorted, and the candidate repair schemes in the top pre-set positions are selected as the repair hypothesis schemes;

[0103] The large language model is called to generate a corresponding repair reason statement for each of the repair hypothesis schemes based on the constraint matching results.

[0104] It should be noted that first, the candidate repair schemes are sorted according to the comprehensive score: the scheme with the highest score usually performs best in terms of constraint matching, structural rationality, etc. After sorting, the candidate repair schemes in the "top pre-set positions" (such as the top 3-5) are selected as the repair hypothesis schemes. Specifically, the number ensures the selection space and avoids too many schemes leading to decision confusion.

[0105] To make the basis of the repair plan clearer, a large language model is called upon to generate a repair rationale based on the "constraint matching results": the model extracts relevant literature from the artifact prior knowledge base (such as an archaeological paper on the repair of similar artifacts) and reference examples (such as a complete collection of the same artifact in a museum), and combines the constraints of the plan (such as "the decorative style of Plan A is consistent with the same type of artifact unearthed in a tomb of the Tang Dynasty") to explain the scientific nature and rationality of the repair in natural language. This step solves the problem of "difficulty in explaining the plan" in traditional repair, making the repair decision more transparent and facilitating subsequent expert review and record archiving.

[0106] Further, the step of superimposing the repair plan and the repair rationale onto the damaged artifact based on the spatial position coordinates to complete the repair of the damaged artifact includes:

[0107] Aligning the three-dimensional geometric model with the damaged artifact according to the spatial position coordinates using feature point matching and PnP algorithm;

[0108] Estimating ambient light using spherical harmonics and combining a physical rendering pipeline to achieve realistic rendering effects to superimpose the repair plan and the repair rationale onto the damaged artifact and complete the corresponding repair.

[0109] It should be noted that first, the "feature point matching and PnP (Perspective-n-Point) algorithm" is used for alignment: common feature points (such as the inflection points of the damaged edges and the special decorations on the surface) are extracted from the three-dimensional geometric model obtained by scanning and the real damaged artifact, and the coordinate transformation relationship between the virtual model and the real artifact in three-dimensional space is calculated using the PnP algorithm to ensure that the virtual repair plan (such as the completed damaged part) can be accurately superimposed on the corresponding position of the artifact entity, avoiding misalignment (for example, the repaired clay figurine arm will not deviate from the original shoulder connection point).

[0110] To enhance the realism of the superimposition effect, the "spherical harmonic function estimates ambient light" is used, specifically, the spherical harmonic function can efficiently fit the light direction, intensity and color of the environment where the cultural relics are located (such as the light of the exhibition hall, natural light), and then combined with the "physical rendering pipeline" (simulate the reflection and refraction effect of light on the surface of the cultural relics), so that the light and shadow effect of the virtual repair part is consistent with the real cultural relics (for example, the shadow direction of the repair area matches the shadow of the original cultural relics). Finally, the repair scheme (virtual complete form) and the repair reason (such as the suspended text explanation) are superimposed on the damaged cultural relics through AR technology, so that users can directly see the repair effect and evaluate it combined with the reason, complete the interactive repair decision, and realize the "what you see is what you get" repair experience. So as to facilitate subsequent processing.

[0111] Please refer to Figure 2 The third embodiment of the present application provides:

[0112] An interactive repair system for cultural relics, wherein the system comprises:

[0113] A scanning module for in-situ three-dimensional scanning of a damaged cultural relic through an augmented reality terminal to obtain a three-dimensional geometric model, surface texture information and spatial position coordinates of the damaged cultural relic, and to construct a cultural relic priori knowledge base;

[0114] A generating module for generating constraint conditions corresponding to the damaged cultural relic according to the cultural relic priori knowledge base, and using a constraint-driven three-dimensional generative model with the three-dimensional geometric model and the surface texture information as input;

[0115] An extracting module for sampling the constraint conditions for verification and guidance during the generation process to generate multiple repair hypothesis schemes that meet the constraint conditions in parallel, and calling a large language model to automatically extract literature and reference examples for each repair hypothesis scheme to generate repair reason elaboration;

[0116] A repair module for superimposing the repair scheme and the repair reason elaboration on the damaged cultural relic based on the spatial position coordinates through augmented reality technology to complete the repair of the damaged cultural relic.

[0117] Further, the scanning module is specifically configured to:

[0118] Capture a multi-view image sequence of the damaged cultural relic through a preset camera and combine with a SLAM algorithm for pose tracking;

[0119] Fuse depth information of the multi-view image sequence using a multi-view stereo reconstruction algorithm to generate corresponding target point cloud data, and generate the three-dimensional geometric model, the surface texture information and the spatial position coordinates according to the target point cloud data.

[0120] Further, the scanning module is specifically used for:

[0121] generating a three-dimensional mesh model corresponding to the damaged cultural relic according to the target point cloud data through a Poisson reconstruction algorithm, and performing texture mapping on the three-dimensional mesh model to generate the three-dimensional geometric model and the surface texture information;

[0122] automatically identifying a damaged area of the damaged cultural relic and extracting a boundary curve based on a surface integrity analysis and a threshold segmentation algorithm, to correspondingly identify spatial position coordinates of the damaged area.

[0123] Further, the extraction module is specifically used for:

[0124] adopting a diffusion model as a basic generation architecture, and introducing a constraint embedding vector in each denoising step in the generation process;

[0125] verifying whether the generation result meets the constraint condition through a constraint consistency verification module;

[0126] if it is verified through the constraint consistency verification module that the generation result does not meet the constraint condition, performing constraint guided optimization to adjust the generation direction.

[0127] Further, the extraction module is specifically used for:

[0128] using different random seeds for parallel sampling to generate a plurality of candidate repair schemes;

[0129] performing multi-dimensional constraint verification on each of the candidate repair schemes to output a comprehensive score corresponding to each of the candidate repair schemes, to correspondingly screen the repair hypothesis scheme according to the comprehensive score.

[0130] Further, the extraction module is specifically used for:

[0131] sorting the candidate repair schemes according to the comprehensive score, and selecting a candidate repair scheme with a front preset rank as the repair hypothesis scheme;

[0132] calling the large language model to generate a corresponding repair reason elaboration for each of the repair hypothesis schemes based on the constraint matching result.

[0133] Further, the repair module is specifically used for:

[0134] adopting a feature point matching and PnP algorithm to align the three-dimensional geometric model with the damaged cultural relic according to the spatial position coordinates;

[0135] The environment light is estimated by a spherical harmonic function, a real rendering effect is realized by combining a physical rendering pipeline, the repair scheme and the repair reason are superimposed on the damaged cultural relic, and corresponding repair processing is completed.

[0136] The fourth embodiment of the present application provides a computer, including a memory, a processor and a computer program stored in the memory and executable on the processor, wherein the processor implements the cultural relic interactive repair method as described above when executing the computer program.

[0137] The fifth embodiment of the present application provides a readable storage medium, which stores a computer program, wherein the program is executed by a processor to implement the cultural relic interactive repair method as described above.

[0138] In summary, the cultural relic interactive repair method and system provided by the above embodiments of the present application can enable the staff to complete the repair of cultural relics in a real environment, thereby improving the cultural relic repair efficiency.

[0139] It should be noted that the above-mentioned modules can be functional modules or program modules, which can be implemented by software or hardware. For the modules implemented by hardware, the above-mentioned modules can be located in the same processor, or the above-mentioned modules can be located in different processors in any combination.

[0140] The logic and / or steps represented in the flowchart or otherwise described herein, for example, can be considered a list of executable instructions for implementing logic functions, and can be specifically embodied in any computer-readable medium for use by an instruction execution system, apparatus or device, such as a computer-based system, a system including a processor or other system that can fetch instructions from an instruction execution system, apparatus or device and execute the instructions, or in conjunction with these instruction execution systems, apparatus or devices. For this specification, the "computer-readable medium" can be any device that can contain, store, communicate, propagate or transport programs for use by an instruction execution system, apparatus or device, or in conjunction with these instruction execution systems, apparatus or devices.

[0141] More specific examples (a non-exhaustive list) of the computer-readable medium include the following: an electrical connection (electronic) having one or more wires, a portable computer diskette (magnetic), a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or Flash memory), an optical fiber, and a portable compact disc read-only memory (CDROM). Additionally, the computer-readable medium can also be paper or another suitable medium upon which the program is printed, as the program can be electronically captured, for example, via an optical scanner, then compiled, interpreted, or otherwise processed, and stored in a computer memory in a form that is then employable by a computer. Examples of computer-readable media that are further within the spirit of the present application are a computer program product, a computer readable storage medium, and a computer.

[0142] It should be understood that aspects of the application can be implemented in hardware, software, firmware or combinations thereof. In the embodiments described above, various steps or methods can be implemented, for example, by software or firmware stored in memory and executed by a suitable instruction execution system. For example, if implemented in hardware, and in another embodiment, any of the following techniques can be used to implement the hardware: discrete logic circuits having logic gates for implementing logic functions upon data signals, application specific integrated circuits having logic gates for implementing logic functions upon data signals, programmable gate arrays (PGA), field programmable gate arrays (FPGA), and the like.

[0143] In the description of the specification, the description of the terms "one embodiment", "some embodiments", "example", "specific example", or "some examples" and the like means that the specific features, structures, materials or characteristics described in connection with the embodiment or example are included in at least one embodiment or example of the present application. In the specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described can be combined in any appropriate manner in one or more embodiments or examples.

[0144] The above-described embodiments are merely some embodiments of the present application, which are described in detail and specifically, but should not be understood as limiting the scope of the present application. It should be noted that, for those skilled in the art, several modifications and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the present application should be subject to the appended claims.

Claims

1. An interactive method for the restoration of cultural relics, characterized in that, The method includes: The damaged cultural relics are scanned in an on-site 3D manner using an augmented reality terminal to obtain the 3D geometric model, surface texture information and spatial coordinates of the damaged cultural relics, and to construct a prior knowledge base of the cultural relics. Constraints corresponding to the damaged cultural relics are generated based on the prior knowledge base of cultural relics, and a constraint-driven three-dimensional generative model is adopted, with the three-dimensional geometric model and the surface texture information as input; During the generation process, the constraints are sampled for verification and guidance to generate multiple repair hypothesis schemes that meet the constraints in parallel. A large language model is called to automatically extract the supporting literature and reference examples for each repair hypothesis scheme to generate a repair rationale. Based on the spatial coordinates, augmented reality technology is used to overlay the restoration hypothesis and the rationale for restoration onto the damaged cultural relic, thereby completing the restoration of the damaged cultural relic. The step of performing an on-site 3D scan of the damaged cultural relic using an augmented reality terminal to obtain the 3D geometric model, surface texture information, and spatial coordinates of the damaged cultural relic includes: The damaged cultural relic was captured by a preset camera with multi-view image sequences, and pose tracking was performed using the SLAM algorithm. A multi-view stereo reconstruction algorithm is used to fuse depth information into the multi-view image sequence to generate corresponding target point cloud data, and the three-dimensional geometric model, surface texture information and spatial position coordinates are generated according to the target point cloud data. The step of generating multiple repair hypothesis schemes that meet the constraints in parallel includes: Parallel sampling using different random seeds is used to generate multiple candidate repair schemes; Each candidate repair scheme is subjected to multi-dimensional constraint verification to output a comprehensive score corresponding to each candidate repair scheme, and the repair hypothesis scheme is selected according to the comprehensive score. The step of overlaying the restoration hypothesis and the rationale for restoration onto the damaged cultural relic using augmented reality technology based on the spatial coordinates to complete the restoration of the damaged cultural relic includes: The three-dimensional geometric model is aligned with the damaged cultural relic based on feature point matching and PnP algorithm according to the spatial coordinates. Ambient lighting is estimated by spherical harmonic functions and combined with physical rendering pipeline to achieve realistic rendering effects. The restoration hypothesis and the rationale for restoration are then superimposed onto the damaged cultural relic to complete the corresponding restoration process.

2. The interactive cultural relic restoration method according to claim 1, characterized in that, The step of generating the three-dimensional geometric model, the surface texture information, and the spatial location coordinates based on the target point cloud data includes: The Poisson reconstruction algorithm is used to generate a three-dimensional mesh model corresponding to the damaged cultural relic based on the target point cloud data, and the three-dimensional mesh model is texture-mapped to generate the three-dimensional geometric model and the surface texture information. Based on surface integrity analysis and threshold segmentation algorithm, the damaged areas of the damaged cultural relics are automatically identified and the boundary curves are extracted to identify the spatial coordinates of the damaged areas.

3. The interactive restoration method for cultural relics according to claim 1, characterized in that, The step of sampling the constraints for verification and guidance during the generation process includes: A diffusion model is used as the basic generative architecture, and a constraint embedding vector is introduced in each denoising step of the generation process; The constraint consistency check module verifies whether the generated results meet the constraint conditions. If the constraint consistency check module verifies that the generated result does not meet the constraints, constraint-guided optimization is performed to adjust the generation direction.

4. The interactive cultural relic restoration method according to claim 1, characterized in that, The step of selecting the repair hypothesis scheme based on the comprehensive score includes: The candidate repair schemes are sorted according to the comprehensive score, and the candidate repair schemes with the highest preset ranking are selected as the repair hypothesis schemes. The large language model is invoked to generate a corresponding explanation of the repair rationale for each repair hypothesis based on the constraint matching results.

5. An interactive cultural relic restoration system, characterized in that, The system for implementing the interactive cultural relic restoration method as described in any one of claims 1 to 4, the system comprising: The scanning module is used to perform on-site three-dimensional scanning of the damaged cultural relics through an augmented reality terminal, so as to obtain the three-dimensional geometric model, surface texture information and spatial location coordinates of the damaged cultural relics, and to construct a prior knowledge base of the cultural relics. The generation module is used to generate constraints corresponding to the damaged cultural relic based on the prior knowledge base of the cultural relic, and to use a constraint-driven three-dimensional generative model, with the three-dimensional geometric model and the surface texture information as input; The extraction module is used to sample the constraints during the generation process for verification and guidance, so as to generate multiple repair hypothesis schemes that meet the constraints in parallel, and call a large language model to automatically extract the supporting literature and reference examples for each repair hypothesis scheme to generate a repair rationale. The repair module is used to overlay the repair hypothesis and the explanation of the repair reasons onto the damaged cultural relic based on the spatial location coordinates using augmented reality technology, so as to complete the repair of the damaged cultural relic.

6. A computer comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the interactive cultural relic restoration method as described in any one of claims 1 to 4.

7. A readable storage medium having a computer program stored thereon, characterized in that, When the program is executed by the processor, it implements the interactive cultural relic restoration method as described in any one of claims 1 to 4.

Images