Method and system for repairing cultural relics based on holographic display

Abstract

The invention discloses a holographic display-based cultural relic restoration method and system, and the method creatively integrates an augmented reality (AR) technology, a multi-modal high-precision modeling technology and a real-time virtual-real interaction technology, and constructs a whole-process closed-loop system of data collection, point cloud fusion, virtual-real fusion and restoration guidance. Precise reduction of the microstructure (micron level) and the macrostructure (cm level) of the cultural relics is achieved, the problems of spatial cognition deviation, insufficient material reduction precision, lack of dynamic calibration in the repair process and the like in traditional repair are solved, intelligent technical support is provided for refined repair of multiple types of cultural relics such as ceramics, metals and stone sculptures, repair personnel are assisted in precisely reducing the original appearances of the cultural relics, and the repair efficiency is improved. The quality and efficiency of cultural relic restoration are improved, and high-precision reconstruction, immersive restoration planning and dynamic error calibration of the three-dimensional structure of the cultural relic are truly realized.

Description

Technical Field

[0001] This invention focuses on the intersection of cultural relic restoration and digital conservation, specifically involving a method and system for cultural relic restoration based on holographic display. Background Technology

[0002] In the field of digital preservation of cultural relics, traditional 3D modeling technology faces significant technical bottlenecks when dealing with the complex morphological features of cultural relic surfaces. Taking the multi-level corrosion layer of bronze artifacts as an example, its surface simultaneously contains micron-sized pores (50-200μm), sub-millimeter-sized nodular protrusions (0.3-1.2mm), and a non-uniformly distributed mineralization layer. The lateral resolution of existing monocular structured light scanning equipment is limited to the 0.1mm level, resulting in a loss rate of up to 37% of the gradual features in the corrosion transition area (according to ICOMOS 2022 test data). For nanoscale surface features such as crazing on ceramic glazes (crazing width <10μm), even with white light interferometer scanning, 23% of micro-cracks still cannot be completely modeled, causing permanent omissions of historical process information.

[0003] At the restoration implementation level, the current technical system reveals a dual cognitive gap: on the one hand, restorers can only rely on two-dimensional images for spatial deduction, and research shows that this method leads to an average 1:2.3 misjudgment of the three-dimensional scale (the difference between the actual curvature of the artifact and the screen display); on the other hand, existing dynamic calibration systems lack multi-physics coupling analysis capabilities and cannot synchronously reflect the material stress changes caused by restoration actions (local stresses >15MPa will cause secondary damage to fragile artifacts). Even more serious is the deficiency in spectral resolution of traditional methods in the dimension of material restoration—taking blue and white porcelain glaze as an example, the CIE-Lab color gamut coverage of its cobalt color characteristics is less than 68%, severely restricting the reproduction of historical authenticity.

[0004] The field of cultural relic restoration urgently needs to overcome three major technical barriers:

[0005] (1) Multi-scale morphology reconstruction challenge: How to achieve accurate modeling across all dimensions from macroscopic shape (cm level) to microscopic texture (μm level);

[0006] (2) Spatial cognitive transformation disorder: Construct a three-dimensional repair guidance system that conforms to the principles of human engineering;

[0007] (3) Simulation of time-varying properties of materials: Establish a visual prediction model of the oxidation aging process of repair materials.

[0008] Existing improvement solutions, such as the patent disclosed in CN114202335A, attempt to introduce multi-view photographic modeling, but their point cloud registration error still reaches the 0.3mm level, and they do not solve the problem of abrupt changes in optical parameters across media surfaces (such as the rust layer-body interface of bronze artifacts). While recently developed AR-assisted restoration technology (see Journal of Cultural Heritage 2023) achieves three-dimensional image overlay, its spatial positioning accuracy is limited to ±2mm, which is insufficient to meet the sub-millimeter operational requirements for the restoration of fragile cultural relics.

[0009] Of particular note is the emergence of AR-based artifact reconstruction methods. While these methods achieve high model reconstruction accuracy through the SIFT algorithm and 3D registration, their fundamental approach remains the same: overlaying a virtual model onto a real environment using a 2D screen. This does not alter the screen-based, 2D interactive nature of the process. Restorers still cannot obtain a true sense of 3D space and tactile feedback. Furthermore, these methods lack real-time, automated, and quantitative error detection and guidance capabilities; their function is limited to "display" rather than "guiding restoration." In conclusion, current technologies have failed to create a closed-loop restoration system integrating ultra-high-precision multimodal data acquisition, intelligent fusion modeling, immersive real-time interactive guidance, and multi-dimensional dynamic error calibration. Significant technological gaps exist, particularly in areas such as microscopic morphology reconstruction, cross-media optical modeling, force-visual fusion interaction, and edge-cloud collaborative real-time data processing architecture. Summary of the Invention

[0010] This invention aims to explore the cross-disciplinary integration of technologies to achieve a truly comprehensive, high-precision, and intelligent assisted system for cultural relic restoration. To this end, the technical solution of this invention provides a method and system for cultural relic restoration based on holographic display. This system combines holographic display and 3D modeling technology to assist restorers in accurately restoring the original appearance of cultural relics, thereby improving the quality and efficiency of cultural relic restoration.

[0011] Therefore, the present invention provides the following technical solution:

[0012] On the one hand, the present invention provides a method for cultural relic restoration based on holographic display, comprising the following steps:

[0013] Step 1: Multi-source data acquisition, capturing multi-view images of cultural relics and generating point cloud data of cultural relics by scanning with a scanner, including obtaining micro-morphological point cloud data by laser confocal scanning and obtaining large-scale point cloud data by structured light scanning.

[0014] Step 2: Point cloud data fusion, which involves registering and fusing microscopic topographic point cloud data with large-scale point cloud data;

[0015] Step 3: Generation of 3D cultural relic model. The 3D cultural relic model is obtained by reconstructing the surface using the registered and fused point cloud data. Then, texture mapping technology is used to apply texture mapping to the 3D cultural relic model based on the multi-view images, which adds texture features to the reconstructed 3D cultural relic model.

[0016] Step 4: Holographic display. The 3D cultural relic model is converted into a holographic image and input into the holographic display system to recover the 3D model.

[0017] Step 5: Holographic Restoration;

[0018] Virtual Restoration Guidance: By restoring cultural relics in a holographic virtual space and recording the restoration process, restoration guidance is generated;

[0019] Actual repair: The holographic display shows the real-time repair process.

[0020] Optionally, in step 2, the curvature-weighted ICP algorithm is used to rigidly register the micro-topography point cloud data and the large-scale point cloud data. The micro-topography point cloud data or the large-scale point cloud data is regarded as the source point cloud, and the other type of point cloud data is regarded as the target point cloud. The specific configuration process is as follows.

[0021] For each source point in the source cloud Find the closest point in the target point cloud. ;

[0022] Calculate the curvature of each source point Then, the curvature-based weights are calculated. A point with a greater curvature will have a greater weight; and the curvature of a point is calculated by taking the curvature of local points in the local point cloud neighborhood.

[0023] By minimizing the objective function Calculate the rotation matrix Translation vector ;

[0024] Repeat the above steps until the preset convergence condition is met;

[0025] objective function satisfy:

[0026]

[0027] in, It is the first in the target point cloud One point; It is the first in the source point cloud One point; It is a rotation matrix, representing the rotation transformation from the source point cloud to the target point cloud; It is a translation vector, representing the translation from the source point cloud to the target point cloud; It is Euclidean distance; It is a weight based on the local curvature of the point cloud, where N is the amount of point cloud data.

[0028] Optionally, in step 3, the Poisson reconstruction algorithm is used to reconstruct the surface and obtain the three-dimensional model, specifically as follows:

[0029] First, calculate the normal vector of the fused point cloud data in step 2;

[0030] Then, the normal vector field is obtained by solving the Poisson equation numerically, and a three-dimensional surface is constructed based on the normal vector field.

[0031] The Poisson equation is as follows:

[0032]

[0033] In the formula, It is the divergence of the normal vector field. It refers to the entire three-dimensional space or the area where the point cloud is located. Represents a point. Represents the normal vector field. It is each point in the point cloud The normal vector, Points on a three-dimensional surface The normal vector field.

[0034] This invention fully considers the importance of the subtle textures on the surface of cultural relics for their restoration; therefore, it introduces the Poisson reconstruction algorithm from MeshLab. This algorithm transforms point cloud data into a triangular mesh model, performing surface smoothing and feature preservation during the process, avoiding the shape distortion problems often encountered in traditional methods when dealing with complex textures or cracks. Furthermore, compared to traditional mesh generation methods, Poisson reconstruction can automatically handle data with complex structures (such as holes and cracks), effectively preserving the subtle textures of the cultural relic's surface. The algorithm uses octree resolution to process complex areas, ensuring that the generated mesh is both highly accurate and retains key features.

[0035] Optionally, the process of applying texture mapping technology to the 3D cultural relic model based on the multi-view images in step 3 is as follows:

[0036] First, using the Agisoft Metashape algorithm, a depth map is generated from the multi-view image, where the depth value of each pixel is... This indicates the distance from the pixel to the camera;

[0037] Secondly, using the Agisoft Metashape algorithm, combined with multi-view images and depth maps, dense point cloud data is generated. The three-dimensional spatial coordinates of each point in the point cloud are calculated based on the geometric relationship between the depth map and the image.

[0038] Then, using UV mapping, each 3D point in the dense point cloud data is mapped. Mapping from three-dimensional space to two-dimensional texture space Then, the depth values ​​in the depth map corresponding to the pixels of the two-dimensional texture space and the multi-view image are used. The pixel values ​​of the multi-view image are mapped to corresponding points in the point cloud using image registration technology. The pixel values ​​of the multi-view image are texture information that characterizes color and texture features. u, v are coordinates in a two-dimensional texture space. Image registration is a standard method in the prior art for aligning two or more images (e.g., multi-view image and depth image), thereby mapping the image pixels under different views to the corresponding positions in the three-dimensional point cloud.

[0039] Finally, based on the point matching relationship between the dense point cloud data and the 3D cultural relic model generated in step 3, texture information is assigned to the point cloud data of the 3D cultural relic model, thereby completing the texture mapping.

[0040] Depth maps provide depth information of dense point clouds (i.e., spatial location information of the point cloud, assigning a corresponding depth value to each pixel). Depth information is fundamental to mapping into three-dimensional space. It is the depth values ​​obtained from a depth map. This allows us to determine the spatial location of each point and then match it with the coordinates in the two-dimensional texture space. UV mapping, in particular, is the process of mapping three-dimensional points... Mapping geometric relationships to two-dimensional texture space coordinates The above process, the corresponding UV mapping function It is determined by the geometric relationship between the depth map and the image, without the need for additional complex mathematical models. It can usually be derived through ray projection and spatial geometric relationships, which is achievable with existing technology.

[0041] Optionally, the depth value of the pixel satisfy:

[0042]

[0043] Where f is the focal length of the camera; It is the baseline distance between the left and right cameras; It's parallax. These are pixel coordinates;

[0044] If the surface of the artifact is spherical or approximately spherical, then spherical mapping is used for UV mapping, and the corresponding mapping model is as follows:

[0045]

[0046] in, It is a point Longitude to the center of the globe; It is a point Latitude to the center of the planet, and Used to determine the three-dimensional spatial coordinates of point P .

[0047] Optionally, the virtual repair guidance process described in step 5 is as follows:

[0048] Within a virtual space, a 3D model is virtually repaired using virtual repair tools, and repair data is recorded in real time. Repair guidance is then generated based on the virtually repaired 3D model and the repair errors.

[0049]

[0050] in, To correct the error, It is a virtually repaired 3D model; It is a reference model of cultural relics, that is, the undamaged form of the original cultural relics, and a three-dimensional digital model of the original cultural relics as a benchmark; the restoration guide should at least include the restoration parameters of the restoration area.

[0051] Among them, repair parameters are generated based on the repair records recorded in real time during the virtual repair process, and the repair parameters are adjusted based on the repair error to generate repair guidance.

[0052] Optionally, the actual restoration process described in step 5 is: real-time restoration and real-time monitoring, that is, restoration guidance is provided based on restoration guidelines, and the actual restoration process of the cultural relic is recorded and displayed in real time in the holographic display system, and real-time comparison and monitoring are performed with the cultural relic reference model. The specific process is as follows:

[0053] First, visual algorithms are used to compare the 3D model with the cultural relic reference model to locate the restoration area;

[0054]

[0055] in, This is the current damaged area data, i.e., the repair area; It is a three-dimensional model of the cultural relic after actual restoration, used to guide the actual restoration operation; It is a reference model of cultural relics, that is, the undamaged form of the original cultural relics, and a three-dimensional digital model of the original cultural relics as a benchmark.

[0056] Then, based on the virtual repaired 3D model and error correction By combining inverse modeling technology, a repair scheme for the repaired area is obtained;

[0057] Finally, the restoration area is restored based on the restoration plan, and the restoration process and the restoration area are displayed in real time in the holographic display system. The restoration area is also dynamically updated in real time and compared with the cultural relic reference model in real time. Once the comparison error exceeds the warning value, the priority of the restoration area is adjusted through the dynamic weight update mechanism.

[0058] Furthermore, the present invention also provides a system based on the above method, comprising:

[0059] The multi-source data acquisition module is used to capture multi-view images of cultural relics and generate point cloud data of cultural relics by scanning with a scanner. Among them, laser confocal scanning is used to obtain micro-morphological point cloud data and structured light scanning is used to obtain large-scale point cloud data.

[0060] The point cloud data fusion module is used to register and fuse microscopic topographic point cloud data and large-scale point cloud data;

[0061] The 3D cultural relic model generation module is used to reconstruct the surface using the registered and fused point cloud data to obtain a 3D cultural relic model. Then, based on the multi-view image, texture mapping technology is used to apply texture mapping to the 3D cultural relic model, which adds texture features to the reconstructed 3D cultural relic model.

[0062] The holographic display module is used to convert 3D cultural relic models into holographic images and input them into the holographic display system to recover the 3D model;

[0063] The holographic restoration module is used for virtual restoration guidance: it restores cultural relics through a holographic virtual space, records the restoration process, and generates restoration guidance; it is also used for holographic display of the real-time restoration process.

[0064] The present invention also provides a system based on the above method, comprising:

[0065] Industrial cameras are used to capture multi-view images of cultural relics and transmit them to a data processor;

[0066] A laser confocal scanner is used to scan cultural relics to obtain microscopic topographic point cloud data, which is then transmitted to a data processor.

[0067] Structured light scanners are used to obtain large-scale point cloud data using structured light scanning and then transmit it to a data processor.

[0068] A data processor is used for computer program implementation: point cloud data fusion and 3D cultural relic model generation, that is, registering and fusing micro-morphological point cloud data and large-scale point cloud data; and using the registered and fused point cloud data to reconstruct a surface to obtain a 3D cultural relic model, and then using texture mapping technology to apply texture mapping to the 3D cultural relic model based on the multi-view images, that is, adding texture features to the reconstructed 3D model.

[0069] A holographic display system is used to receive the three-dimensional cultural relic model reconstructed by the data processor, convert it into a holographic image, and input it into the holographic display system to recover the three-dimensional model;

[0070] The virtual restoration system is equipped with virtual restoration tools and a processing module. It is used to restore cultural relics in a holographic virtual space using virtual restoration tools, and the corresponding processing module records the restoration process and generates restoration instructions.

[0071] The restoration monitoring system compares and monitors the cultural relics during the restoration process with a reference model of the cultural relics in real time.

[0072] The present invention also provides a computer-readable storage medium storing a computer program that is invoked by a processor to implement the steps of a method for cultural relic restoration based on holographic display.

[0073] Compared with the prior art, the present invention achieves the following progress and effects:

[0074] This invention provides a holographic display-based method and system for cultural relic restoration. Through a 3D modeling module, it achieves high-precision 3D data acquisition and modeling of cultural relics, providing accurate original data for restoration. Utilizing holographic technology, it allows restorers to intuitively observe the 3D form of the relics, improving the accuracy and efficiency of restoration. By comparing the differences between the restored area and the original model in real time, it helps restorers adjust the restoration plan promptly, ensuring consistency between the restoration effect and the original appearance of the relic. This system combines advanced holographic display technology and 3D modeling technology, providing a novel auxiliary means for cultural relic restoration and possessing broad application prospects.

[0075] In particular, this invention uses two high-precision devices to acquire microscopic topographic data and large-scale point cloud data, respectively. These two scanning technologies provide sub-micron and 50μm level precision, respectively, enabling higher accuracy and detail in capturing fine textures and large-scale damage on artifact surfaces, thus improving restoration accuracy. Furthermore, this invention employs a curvature-weighted ICP algorithm for rigid registration. Its innovation lies in introducing a curvature weighting factor, which automatically adjusts the matching weights based on the local curvature of the point cloud surface during data registration. Compared to traditional ICP algorithms, this invention reduces errors when processing complex surfaces, especially significantly improving registration accuracy and detail preservation in handling microscopic textures and cracks on artifact surfaces. Moreover, this invention uses MeshLab's Poisson reconstruction algorithm for surface reconstruction and optimizes it. The Poisson reconstruction algorithm effectively handles complex cracks, holes, and other structures, automatically smoothing and preserving the fine textures of artifact surfaces. Based on this, this invention uses octree resolution and optimizes the mesh generation process, ensuring that the generated 3D mesh not only has high precision but also retains more complex details of the artifact surface. In the texture mapping process, traditional methods typically generate texture maps by simply aligning images with point cloud data, which easily leads to texture distortion and stretching issues. This invention introduces a UV unwrapping algorithm and multi-view image matching technology to ensure the accuracy and consistency of the texture maps. Attached Figure Description

[0076] Figure 1 This is a schematic diagram of a cultural relic restoration system based on holographic display;

[0077] Figure 2 This is a flowchart illustrating a method for cultural relic restoration based on holographic display, provided in an embodiment of the present invention. Detailed Implementation

[0078] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the invention and are not intended to limit the invention. The technical features involved in the various embodiments of the invention described below can be combined with each other as long as they do not conflict with each other.

[0079] It should be noted that although functional modules are divided in the device schematic diagram and a logical order is shown in the flowchart, in some cases, the steps shown or described may be performed in a different order than the module division in the device or the order in the flowchart. The terms "first," "second," etc., in the specification, claims, and the aforementioned drawings are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence.

[0080] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application belongs. The terminology used herein is for the purpose of describing embodiments of this application only and is not intended to limit this application.

[0081] This invention provides a method and system for cultural relic restoration based on holographic display. It introduces 3D modeling and holographic technology into cultural relic restoration applications. Through multimodal data fusion and intelligent algorithms, it achieves high-precision reconstruction of the 3D structure of cultural relics, immersive restoration planning, and dynamic error calibration. This provides comprehensive auxiliary means for the precise restoration of ceramic, metal, and other types of cultural relics, and offers a complete, high-precision, and intelligent closed-loop system for cultural relic restoration. Specifically, the technical concept of the holographic display-based cultural relic restoration method provided in this embodiment is as follows:

[0082] Step 1: Multi-source data acquisition, capturing multi-view images of cultural relics and generating point cloud data of cultural relics by scanning with a scanner, including obtaining micro-morphological point cloud data by laser confocal scanning and obtaining large-scale point cloud data by structured light scanning.

[0083] Step 2: Point cloud data fusion, which involves registering and fusing microscopic topographic point cloud data with large-scale point cloud data;

[0084] Step 3: Generation of 3D cultural relic model. The 3D cultural relic model is obtained by reconstructing the surface using the registered and fused point cloud data. Then, texture mapping technology is used to apply texture maps to the 3D cultural relic model based on multi-view images, which adds texture features to the reconstructed 3D cultural relic model.

[0085] Step 4: Holographic display. The 3D cultural relic model is converted into a holographic image and input into the holographic display system to recover the 3D model.

[0086] Step 5: Holographic restoration and real-time monitoring;

[0087] Virtual Restoration Guidance: By restoring cultural relics in a holographic virtual space and recording the restoration process, restoration guidance is generated;

[0088] Real-time restoration monitoring: The actual restoration process of cultural relics is recorded in real time in the holographic display system, and the results are compared and monitored in real time with the cultural relic reference model, which is a three-dimensional model of the original cultural relic in its undamaged state.

[0089] In some embodiments, to achieve higher precision in artifact restoration, this invention combines two high-precision scanning technologies: a laser confocal scanner and a structured light scanner, to acquire microscopic morphological data and large-scale point cloud data of the artifact, respectively. Specifically, for microscopic morphological data: the laser confocal scanner provides sub-micron level surface height data, capable of capturing minute textures and cracks on the artifact surface, particularly suitable for the fine structures of ceramic and metal artifacts. For large-scale point cloud data: the structured light scanner provides high-density point cloud data with a precision of up to 50 μm, suitable for capturing the macroscopic morphology of artifacts, such as the shape of objects and large-scale cracks.

[0090] Specifically:

[0091] 1) Structured light scanner: By combining an industrial camera with a structured light projector, high-density point cloud data of the artifact surface can be obtained with an accuracy of up to 50μm. This point cloud data can completely record the large-scale geometry of the artifact, such as cracks and large-scale damage on the outer surface.

[0092] 2) Laser confocal scanner: Utilizing laser scanning technology, it accurately captures the minute textures and details on the surface of cultural relics, with a resolution of up to 0.1μm. It is particularly suitable for the fine structures of surfaces such as ceramics and metals, such as textures, microcracks and corrosion layers.

[0093] To ensure detailed reconstruction, this invention simultaneously fuses multi-view photographic images with point cloud data. Through texture mapping technology, it restores the delicate colors and textures of the artifact's surface while maintaining geometric accuracy. This will be explained in detail below.

[0094] In some embodiments, step 2 employs a curvature-weighted ICP algorithm to rigidly register the micro-topography point cloud data and the large-scale point cloud data. Compared to traditional methods, rigid registration algorithms based on surface matching, such as the curvature-weighted ICP algorithm (Iterative Closest Point Algorithm), can significantly reduce registration errors, especially when dealing with complex surfaces, effectively avoiding information loss. Specifically, the micro-topography point cloud data or the large-scale point cloud data is considered the source point cloud, and the other type of point cloud data is considered the target point cloud. The specific configuration process is as follows:

[0095] For each source point in the source cloud Find the closest point in the target point cloud. ;

[0096] Calculate the curvature of each source point Then, the curvature-based weights are calculated. The greater the curvature of a point, the greater its corresponding weight; and the curvature of a point is calculated from the curvature of local points within the local point cloud neighborhood.

[0097] By minimizing the objective function Calculate the rotation matrix Translation vector ;

[0098] Repeat the above steps until the preset convergence condition is met;

[0099] objective function satisfy:

[0100]

[0101] in, It is the first in the target point cloud One point; It is the first in the source point cloud One point; It is a rotation matrix, representing the rotation transformation from the source point cloud to the target point cloud; It is a translation vector, representing the translation from the source point cloud to the target point cloud; It is Euclidean distance; It is a weight based on the local curvature of the point cloud, where N is the amount of point cloud data.

[0102] Regarding curvature calculation, in some embodiments, curvature Curvature is typically calculated from the neighborhood of a local point cloud, for example, by using principal component analysis (PCA) to estimate the curvature of each point.

[0103]

[0104] in, It is the principal feature value of the neighborhood of each point in the point cloud, representing the curvature of the local point.

[0105] The registration and fusion operation described in this embodiment enables precise registration of data from the two scanning techniques in geometric space. The fused dataset retains both large-scale shape and microscopic details. Traditional multi-source data fusion often faces insufficient registration accuracy, especially on complex surfaces (such as irregular cracks and wrinkles on artifacts). The method of this invention incorporates a curvature weighting factor during data registration, which can more accurately handle surface feature details, reduce errors, and retain more minute surface features.

[0106] In some embodiments, the data fusion generates a point cloud containing millions of data points. To transform these point clouds into 3D models with clear geometric shapes, this invention uses MeshLab's Poisson reconstruction algorithm. This algorithm transforms the point cloud data into a triangular mesh model, performing surface smoothing and feature preservation in the process, avoiding the shape distortion problem often encountered in traditional methods when dealing with complex textures or cracks. Compared to traditional mesh generation methods, Poisson reconstruction can automatically handle data with complex structures (such as holes, cracks, etc.) while preserving the subtle textures of the artifact's surface, which is crucial for artifact restoration. This invention uses octree resolution in the algorithm to process complex areas, ensuring that the generated mesh is both highly accurate and retains key features.

[0107] Therefore, in step 3, the Poisson reconstruction algorithm is used to reconstruct the surface and obtain the three-dimensional model, specifically as follows:

[0108] First, calculate the normal vector of the fused point cloud data in step 2;

[0109] Then, the normal vector field is obtained by solving the Poisson equation numerically, and a three-dimensional surface is constructed based on the normal vector field.

[0110] The Poisson equation is as follows:

[0111]

[0112] In the formula, It is the divergence of the normal vector field. It refers to the entire three-dimensional space or the area where the point cloud is located. Represents a point. Represents the normal vector field. It is each point in the point cloud The normal vector, Points on a three-dimensional surface The normal vector field.

[0113] It should be understood that during the surface reconstruction process, MeshLab's Poisson reconstruction algorithm was used to generate triangular meshes, which contain the following information:

[0114] Geometry: Accurately represents the geometry of the artifact's surface and its local geometric features, including details such as cracks, damaged areas, and textures.

[0115] Normals: The Poisson reconstruction algorithm uses normal vectors to optimize the smoothness of the mesh and ensure high precision in surface generation. It can accurately restore the microscopic features of the artifact surface (such as cracks, surface irregularities, etc.) and maintain accurate surface details.

[0116] Topology of the Mesh: The generated triangular mesh has a clear topology, representing each region on the surface of the artifact and the complete geometry of the artifact surface.

[0117] Surface Damage and Features: The algorithm automatically repairs damaged areas while preserving details. The generated triangular mesh not only shows the shape of the artifact but also accurately restores surface details (such as cracks, missing parts, etc.).

[0118] In some embodiments, after generating the triangular mesh, the next step is to add texture to the reconstructed 3D model. This invention employs Agisoft Metashape's dense point cloud matching method to obtain detailed textures of the artifact's surface from high-resolution images taken from different perspectives. Specifically, these image data are precisely matched with the generated 3D model to generate a high-quality 8192×8192 resolution texture map, ensuring perfect restoration of the artifact's surface color and material. Considering that traditional texture mapping techniques often suffer from texture stretching and distortion, this invention uses a UV unwrapping algorithm, combined with multi-view image data, to ensure the accuracy and consistency of the texture map. During the mapping process, adaptive texture compression technology is incorporated, enabling the generated texture to maintain high quality while reducing processing time and improving restoration efficiency.

[0119] Therefore, the process of applying texture mapping technology to the 3D cultural relic model based on multi-view images in step 3 is as follows:

[0120] First, using the Agisoft Metashape algorithm, a depth map is generated from the multi-view image, where the depth value of each pixel is... This indicates the distance from the pixel to the camera;

[0121] Given the pixel positions of the left and right images. The formula for calculating parallax is:

[0122]

[0123] in: It refers to the pixel position in the left image; This refers to the corresponding pixel position in the image on the right. This is the parallax at that location.

[0124] Through parallax It can calculate the depth value of each pixel. satisfy:

[0125]

[0126] Where f is the focal length of the camera; It is the baseline distance between the left and right cameras; It's parallax. These are pixel coordinates.

[0127] Secondly, the Agisoft Metashape algorithm is used to generate dense point cloud data by combining multi-view images and depth maps. The three-dimensional spatial coordinates of each point in the point cloud are calculated based on the geometric relationship between the depth map and the image. This is achievable with existing technology and will not be described in detail.

[0128] Then, using UV mapping, each 3D point in the dense point cloud data is mapped. Mapping from three-dimensional space to two-dimensional texture space ; and then utilize the two-dimensional texture space The depth values ​​in the depth map corresponding to the pixels of the multi-view image. Image registration technology maps the pixel values ​​of the multi-view images to corresponding points in the point cloud. The pixel values ​​of the multi-view images represent texture information that characterizes color and texture features. Assume each point in the point cloud... In three-dimensional space, the UV unwrapping algorithm maps it to two-dimensional texture coordinates. :

[0129]

[0130] in, These are the three-dimensional coordinates of points in point cloud data; These are the coordinates in the texture image. If the surface of the artifact is spherical or approximately spherical, then spherical mapping is used for UV mapping, and the corresponding mapping model is:

[0131]

[0132] in, It is a point Longitude to the center of the globe; It is a point Latitude to the center of the planet, and Used to determine the three-dimensional coordinates of point P .

[0133] Finally, based on the point matching relationship between the dense point cloud data and the 3D model generated in step 3, texture information is assigned to the point cloud data of the 3D model, thereby completing the texture mapping.

[0134] The technical solution of this invention uses Agisoft Metashape's image matching algorithm to map image data from different perspectives onto each point cloud point, generating a texture map. The texture map combines image information from multiple perspectives with point cloud data to generate a high-resolution two-dimensional texture map, typically with a high resolution of 8192×8192, ensuring realistic detail reproduction.

[0135] This process incorporates UV unwrapping technology, which prevents texture images from being stretched or distorted, thus accurately reflecting the color and texture characteristics of the artifact's surface. After the UV unwrapping technology transforms the artifact's surface into a two-dimensional plane, a precise correspondence is established between each point cloud point and the pixel on its texture image, thereby achieving high-quality texture mapping.

[0136] It should be understood that step 4, holographic display, involves converting the 3D artifact model into a holographic image and inputting it into the holographic display system to reconstruct the 3D model. Specifically, this includes:

[0137] Holographic Image Generation: In this embodiment of the invention, a holographic image generation method based on light field display technology is adopted to convert the three-dimensional model of the cultural relic into a holographic image. First, the three-dimensional digital data of the cultural relic (i.e., the three-dimensional cultural relic model in step 3) is acquired, and then converted into a hologram using digital holography. Digital holography uses interference patterns to record the wavefront information of the object. By simulating light waves on the three-dimensional model, a corresponding interference pattern is generated. Its mathematical model is as follows:

[0138]

[0139] In the formula, It is a holographic image; It is the light wave information of the object; It is a reference wave information.

[0140] Holographic Image Display: The generated holographic image is input into a holographic display system, which uses wavefront reconstruction technology to reconstruct the object's three-dimensional image. Wavefront reconstruction technology is based on interferometry patterns and reconstructs the three-dimensional shape of the artifact through optical diffraction and inversion calculations. Wavefront Reconstruction Formula:

[0141]

[0142] in, It is the reconstructed three-dimensional light wave information; It is the current holographic image that is being input; It is the distance that light waves travel; Wave number represents the wavelength and propagation characteristics of light. The coordinates of the current holographic image are given. This formula describes the transformation process from an interference pattern to a three-dimensional object image, recovering the three-dimensional shape of the artifact by calculating the contribution of each pixel. Therefore, step 4 is technically feasible.

[0143] In some embodiments, step 5 involves combining holographic images with a three-dimensional digital model of the artifact, displaying the artifact restoration process in real time through a holographic display system, and providing virtual restoration guidance.

[0144] The virtual repair guidance process is as follows:

[0145] Within a virtual space, a 3D model is virtually repaired using virtual repair tools, and repair data is recorded in real time. Repair guidance is then generated based on the virtually repaired 3D model and the repair errors.

[0146]

[0147] in, To correct the error, It is a virtually repaired 3D model; It is a reference model of cultural relics, that is, the undamaged form of the original cultural relics, and a three-dimensional digital model of the original cultural relics as a benchmark; the restoration guide includes at least the restoration parameters of the restoration area; thus, restoration parameters are generated based on the restoration records recorded in real time during the virtual restoration process, and the restoration parameters are adjusted based on the restoration error to generate restoration suggestions.

[0148] Virtually repaired 3D model This refers to a completely new 3D model generated after restoring an original 3D model using virtual restoration tools. These tools simulate the actual restoration process to optimize and repair the artifact model; restoration errors... : Represents the virtual repaired 3D model With the lossless three-dimensional model Differences between them; based on the 3D model after virtual repair and error correction This allows for a clear understanding of the changes and errors that occur during the repair process.

[0149] Correcting errors It is mainly used to guide repair operations during actual repair processes. Specific applications are as follows:

[0150] 1. Error Analysis and Guidance: Repair errors reflect potential problems during the virtual repair process, such as dimensional deviations and shape inaccuracies. Therefore, during actual repair, repair personnel can adjust their repair methods based on these errors. For example, if errors are significant in certain areas, prompts can be generated to encourage further repair measures, ensuring that the actual repair results closely approximate the virtual repaired model. These rules can be pre-programmed into a programming language based on experience in the field.

[0151] 2. Model Adjustment: Repair errors can also be used to further adjust the original 3D model. For example, by comparing the differences between the virtual restoration model and the actual artifact, restorers can fine-tune the 3D model of the actual artifact to make it closer to the ideal restoration result.

[0152] In some embodiments, the real-time repair monitoring process in step 5 is as follows:

[0153] First, visual algorithms are used to compare the 3D model with the cultural relic reference model to locate the restoration area;

[0154]

[0155] in, It is data on the damaged area; It refers to the three-dimensional cultural relic model currently being restored, used to guide actual restoration operations. It is the three-dimensional cultural relic model obtained after the restoration process. It can be a digital model generated through virtual restoration technology; in some applications, it may be presented through a holographic display system. It is a digital restoration model primarily used to guide actual restoration operations.

[0156] Then, based on the virtual repaired 3D model and error correction Combined with inverse modeling technology, a repair plan for the repair area is generated;

[0157] In some embodiments, the process of generating a repair scheme for the repair area is as follows:

[0158] 1. Repair Error Calculation: First, using the virtual repaired 3D model... With the lossless three-dimensional model By comparing the results, the repair error was calculated. That is, the virtual repaired 3D model With the lossless three-dimensional model The differences between them. This process calculates the repair error for each repair area, reflecting potential problems during the virtual repair process, such as dimensional deviations and shape inaccuracies.

[0159] 2. Inverse Modeling Technique: Based on error repair ( The virtual restoration process employs reverse modeling technology to generate restoration plans. Reverse modeling involves analyzing restoration errors to deduce the areas requiring restoration and their corresponding strategies. The greater the error in the restoration area, the more refined the restoration strategy will be, ensuring the post-restoration effect matches the virtual restoration model. It should be understood that in practical applications, based on the actual cultural relic, preset optional restoration strategies are used to automate strategy modification suggestions and other operations.

[0160] 3. Repair Area Identification and Priority Ranking: Based on error calculation and the error distribution of the repair areas, the repair system automatically identifies the areas that need repair and prioritizes them according to the magnitude of the error. Areas with larger errors will be repaired first to ensure repair accuracy. In practical applications, the system can use algorithms (such as least squares method, weighted algorithm, etc.) to optimize the repair order.

[0161] 4. Repair Suggestion Generation: Based on the repair error and the priority of the repair area, the system will generate specific repair suggestions, including:

[0162] Recommended restoration methods: For areas with large errors, it is recommended to use higher precision restoration tools or different restoration materials in order to restore the original appearance of the cultural relic to the greatest extent possible.

[0163] Repair operation parameters: These include adjusting the repair tools, the thickness and material of the repair materials, etc., to ensure that the repaired effect is as close as possible to the virtual repaired 3D model.

[0164] 5. Automation and Optimization: During the restoration process, the system dynamically adjusts the restoration plan based on real-time feedback data. For example, by monitoring changes in the restoration area in real time, the system can automatically update the restoration plan to ensure that the alignment error between the restoration area and the original artifact is minimized.

[0165] 6. Optimized Repair Results: Repair suggestions are continuously optimized during the actual repair process. The system dynamically adjusts repair strategies and operating parameters based on the repair progress and changes in error in the repaired area, thereby maximizing the repair results.

[0166] The repair solution of this invention is a repair strategy generated through data analysis and error back-calculation. Utilizing reverse modeling technology, a repair solution can be automatically generated based on errors. This solution is "non-subjective," entirely based on error calculation and automatic generation of the repair model, which can greatly improve repair efficiency and accuracy, providing repair personnel with more precise repair suggestions.

[0167] Finally, the restoration area is restored based on the restoration plan, and the restoration process is displayed in real time in the holographic display system. The system also compares and monitors the restoration process with the cultural relic reference model in real time. Once an error is found, the system will update the restoration area in real time and adjust the restoration priority through a dynamic weight update mechanism.

[0168] The dynamic weight update mechanism proposed in this invention adjusts the restoration priority based on the magnitude of the error in each restoration area during the restoration process. During restoration, areas with larger errors are dynamically given more restoration weight, ensuring these areas receive more attention and restoration. This mechanism continuously updates the restoration strategy and the weights of the restoration areas by monitoring the differences between the restoration areas and the original artifact in real time, ensuring the accuracy of the restoration process. In practical applications, the system generates specific restoration suggestions for areas with larger errors, covering adjustments to restoration parameters such as size, shape, and depth. These restoration suggestions constitute the specific content of the restoration plan, and they are dynamically generated through restoration error analysis and real-time data, ensuring the accuracy and effectiveness of the restoration process.

[0169] Furthermore, in some embodiments, the present invention ensures that the holographic display system and the restoration system can accurately match the three-dimensional model and displayed image of the cultural relic. It also preferably performs: real-time calculation of display errors and optical calibration of the holographic display device to ensure that the displayed holographic image and the three-dimensional model are correctly aligned in space, thereby minimizing the error between the virtual restoration model and the original cultural relic model during the holographic restoration process. The display error is used to dynamically adjust the restoration error of the display system during the restoration process, ensuring that each restoration area is aligned with the shape of the original cultural relic.

[0170] Calibration formula:

[0171]

[0172] in, It indicates a display error; It displays the location in the image; It refers to the location of the cultural relic in the 3D model; It is the focal length of the display system.

[0173] In practical applications, display errors are not only used for monitoring comparison errors, but also play an important role in the repair process. Specific applications are as follows:

[0174] 1. Comparison Error Monitoring: Displays errors used to calculate the virtual repaired 3D model. ) and cultural relic reference model ( The system compares the differences between the repair progress and accuracy in real time by comparing the errors, ensuring that the priority of the repair area matches the repair accuracy.

[0175] 2. Dynamic Adjustment of Repair Plan: By monitoring and displaying errors in real time, the system automatically adjusts the alignment and priority of the repair areas. Repair areas with larger errors will have their repair weight dynamically increased, guiding repair personnel to take more repair measures for these areas. The dynamic updating of display errors ensures the accuracy and precision of the repair process.

[0176] 3. Generate Repair Suggestions: The error display not only provides comparative data but also generates specific repair suggestions. These suggestions include adjustments to repair parameters such as size, shape, and depth to ensure that the actual repaired result closely approximates the virtual repair model.

[0177] 4. Repair guidance: By displaying error analysis, the system can provide repair personnel with precise repair guidance, including specific operations such as adjusting the model, repair order, and using repair tools.

[0178] Therefore, displaying errors is not only used to monitor errors, but it also directly affects the repair process through dynamic updates and repair suggestions, ensuring the maximization of repair results.

[0179] In some embodiments, a repair process calibration is also performed: during the repair process, the matching degree between the repair model and the 3D model is calibrated to ensure that the repair data accurately corresponds to the virtual repair area in the holographic display image. The calibration during the repair process dynamically adjusts the relative positions of the repair area and the model, ensuring that the repair operation always matches the actual effect.

[0180] Calibration formula:

[0181]

[0182] in, It is a calibration error; This is the current repair model in the holographic display; It is a 3D model of the original artifact (a 3D model of its current form). Through calibration, the artifact model in the restoration and display system is kept consistent throughout all steps, helping to ensure the accuracy and consistency of the restoration tools, and that each restoration operation can be accurately presented in the holographic image.

[0183] Example:

[0184] 1. Preprocessing stage

[0185] ① After cleaning the surface of the cultural relic, an orthogonal 9-view photo was taken using a 5-megapixel industrial camera for subsequent texture mapping.

[0186] ② The structured light scanner performs a spiral scan on the cultural relic with a layer spacing of 0.5mm. The scanning time for a single workpiece is ≤15 minutes, generating raw point cloud data (density ≥200 points / cm²).

[0187] ③ Laser confocal scanner, used to scan cultural relics to obtain microscopic morphological point cloud data.

[0188] 2. Data Processing Stage

[0189] Point cloud denoising: Outliers are removed using a statistical filter (k=20, σ=2), retaining ≥95% of the valid point cloud.

[0190] Multi-source data fusion: The microscopic topography data (sampling interval 0.05 mm) acquired by laser confocal scanning is rigidly registered with the structured light point cloud (RMSD < 0.08 mm).

[0191] 3. Model Generation Stage

[0192] Surface reconstruction: Triangular meshes were generated using MeshLab’s Poisson reconstruction algorithm (depth value 8), with an average side length of 0.3 mm and feature edges were automatically preserved.

[0193] Texture mapping: Dense point cloud matching is performed using Agisoft Metashape to generate texture maps with a resolution of 8192×8192, with a mapping error of <0.1 pixels.

[0194] Holographic Restoration Operation Process

[0195] 1. Restoration staff wear force feedback gloves and adjust the perspective of the holographic model through intuitive gestures. Opening and closing the palm allows for zooming in and out of the model, while sliding the fingers controls its rotation. This natural interaction method allows restoration staff to observe the holographic model in detail from any angle.

[0196] 2. Within the virtual space, repair personnel can use pre-set repair tools, such as virtual scrapers to remove imperfections and putty brushes to fill missing parts, to conduct repair simulation work. The system possesses powerful real-time computing capabilities, accurately calculating volume changes due to material additions or subtractions with an accuracy of ±0.3 cm³, providing reliable data support for repair operations.

[0197] Real-time repair calibration

[0198] During the restoration process, three sets of RGB-D cameras continuously captured the real-time status of the artifact at a high frequency of 30fps. Using the ICP algorithm, the data was quickly registered with the original model, with each frame processed in less than 50ms to ensure timely data processing. A difference heatmap was simultaneously overlaid on the holographic image. If a restoration deviation in a certain area exceeded 0.5mm, the system would immediately trigger an audible alarm, and the deviation value would be displayed as a floating numerical label in the corresponding position. The font size dynamically adjusted according to the degree of deviation; the larger the deviation, the larger the font, facilitating quick detection and location of the problem area by restoration personnel.

[0199] Material texture reproduction

[0200] For the restoration of painted cultural relics, the system utilizes multispectral imaging technology, covering a wavelength range of 380-1000nm, to accurately acquire the original pigment spectral data, and then construct a CIELAB color space matching model. When restorers mix pigments, the holographic interface displays the ΔE*ab value between the current color and the target color in real time, with an accuracy of 0.5, providing restorers with precise color matching guidance and helping them accurately adjust the pigment ratio to achieve a high degree of restoration of material texture.

[0201] In some embodiments, the present invention also provides a system based on the above method, including a multi-source data acquisition module, a point cloud data fusion module, a three-dimensional cultural relic model generation module, a holographic display module, and a holographic restoration and real-time monitoring module that are connected in sequence or interconnected.

[0202] The multi-source data acquisition module is used to capture multi-view images of cultural relics and generate point cloud data of cultural relics by scanning with a scanner. Among them, laser confocal scanning is used to obtain micro-morphological point cloud data and structured light scanning is used to obtain large-scale point cloud data.

[0203] The point cloud data fusion module is used to register and fuse microscopic topographic point cloud data and large-scale point cloud data;

[0204] The 3D cultural relic model generation module is used to reconstruct a surface using the registered and fused point cloud data to obtain a 3D model. Then, based on the multi-view image, texture mapping technology is used to perform texture mapping, which adds texture features to the reconstructed 3D model.

[0205] The holographic display module is used to convert 3D cultural relic models into holographic images and input them into the holographic display system to recover the 3D model;

[0206] The holographic restoration and real-time monitoring modules are used for virtual restoration guidance: restoring cultural relics through a holographic virtual space and recording the restoration process to generate restoration guidance; and for real-time restoration monitoring: recording and displaying the cultural relic restoration process in real time in the holographic display system, and comparing and monitoring it with the cultural relic reference model in real time.

[0207] It should also be understood that the specific implementation process of each module is described in the above method. This invention will not repeat it here. The above division of functional modules is only for illustrative purposes. In some embodiments, some functional modules can be combined and some functional modules can be separated. Each functional module can be implemented in software, hardware, or a combination of software and hardware. The software and hardware devices include, but are not limited to, general-purpose computer equipment, programmable gate arrays, digital signal processors, microprocessors and their corresponding programming or burning software.

[0208] In some embodiments, the present invention also provides a system based on the above method, including: an industrial camera, a laser confocal scanner, a structured light scanner, a data processor, a holographic display system, a virtual repair system, and a repair monitoring system.

[0209] Industrial cameras are used to capture multi-view images of cultural relics and transmit them to a data processor.

[0210] A laser confocal scanner is used to scan cultural relics to obtain microscopic topographic point cloud data, which is then transmitted to a data processor.

[0211] Structured light scanners are used to obtain large-scale point cloud data using structured light scanning and then transmit it to a data processor.

[0212] The data processor is used for computer program implementation: point cloud data fusion and 3D cultural relic model generation, which involves registering and fusing microscopic and large-scale point cloud data; and using the registered and fused point cloud data to reconstruct surfaces to obtain a 3D model, and then using texture mapping technology based on the multi-view images to add texture features to the reconstructed 3D model. In practical applications, the data processor is used to process and store data from various modules, such as controlling each module according to the operation instructions of the restoration personnel. It can use high-performance computers or servers with sufficient storage capacity and computing power.

[0213] A holographic display system receives a reconstructed 3D model from a data processor, converts it into a holographic image, and inputs it into the holographic display system to reconstruct the 3D model. In some embodiments, the holographic display system includes a holographic projector, a holographic imaging medium, and an interactive device. The holographic projector projects the processed 3D data onto the holographic imaging medium to form a realistic holographic image. Restoration personnel can interact with the holographic image through the interactive device, such as zooming in, zooming out, and rotating, to observe the details of the artifact more closely. The interactive device can be a gesture recognition device or a touch control device, etc.

[0214] The virtual restoration system is equipped with virtual restoration tools and processing modules. The virtual restoration tools are used to restore cultural relics in a holographic virtual space, while the corresponding processing module records the restoration process and generates restoration instructions.

[0215] The restoration monitoring system compares and monitors the artifact during the restoration process against a reference model in real time. In some embodiments, it includes an image acquisition device and a comparison algorithm module. The image acquisition device, which can be a high-definition camera, acquires images of the artifact in real time during the restoration process. This camera is installed at a suitable location within the restoration work area to ensure clear imaging of the restoration area. The comparison algorithm module matches and compares the real-time images with the original 3D model, displaying the differences between the restored area and its original state. The comparison algorithm module employs a feature point matching method. First, it extracts feature points from the original 3D model and the real-time images. Then, it determines the correspondence between the restored area and the original model through feature point matching, calculates the difference information, and displays it visually, such as using different colors to mark the areas of difference.

[0216] In some embodiments, the present invention also provides a computer-readable storage medium storing a computer program that is invoked by a processor to implement the steps of a method for restoring cultural relics based on holographic display.

[0217] That is, specific execution:

[0218] Step 1: Multi-source data acquisition, capturing multi-view images of cultural relics and generating point cloud data of cultural relics by scanning with a scanner, including obtaining micro-morphological point cloud data by laser confocal scanning and obtaining large-scale point cloud data by structured light scanning.

[0219] Step 2: Point cloud data fusion, which involves registering and fusing microscopic topographic point cloud data with large-scale point cloud data;

[0220] Step 3: Generation of 3D cultural relic model. The 3D model is obtained by reconstructing the surface using the registered and fused point cloud data. Then, texture mapping technology is used to apply texture maps to the 3D cultural relic model based on multi-view images, which adds texture features to the reconstructed 3D model.

[0221] Step 4: Holographic display. The 3D cultural relic model is converted into a holographic image and input into the holographic display system to recover the 3D model.

[0222] Step 5: Holographic restoration and real-time monitoring;

[0223] Virtual Restoration Guidance: By restoring cultural relics in a holographic virtual space and recording the restoration process, restoration guidance is generated;

[0224] Real-time restoration monitoring: The actual restoration process of cultural relics is recorded and displayed in real time in the holographic display system, and is compared and monitored in real time with the cultural relic reference model.

[0225] For details on the implementation of each step, please refer to the description of the control method embodiment above.

[0226] The readable storage medium is a computer-readable storage medium, which can be an internal storage unit of the hardware and software device described in any of the foregoing embodiments, such as the hard drive or memory of the controller. The readable storage medium can also be an external storage device of the controller, such as a plug-in hard drive, Smart MediaCard (SMC), Secure Digital (SD) card, or Flash Card equipped on the controller. Further, the readable storage medium can include both internal storage units and external storage devices of the controller. The readable storage medium is used to store the computer program and other programs and data required by the controller. The readable storage medium can also be used to temporarily store data that has been output or will be output.

[0227] Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention. The aforementioned readable storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0228] Those skilled in the art will understand that embodiments of this application can be provided as methods, systems, or computer program products. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application can take the form of a computer program product embodied on one or more computer-readable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code. This application refers to flowchart illustrations and / or instructions executed by a processor of a method, apparatus (system), and computer program product according to embodiments of this application to create means for implementing the functions specified in one or more flowchart illustrations and / or one or more block diagrams. These computer program instructions may also be stored in a computer-readable storage medium capable of directing a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means that implement the functions specified in one or more flowchart illustrations and / or one or more block diagrams. These computer program instructions may also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer-implemented process, such that the instructions, which execute on the computer or other programmable apparatus, provide steps for implementing the functions specified in one or more flowcharts and / or one or more blocks of a block diagram.

[0229] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

Claims

1. A method for cultural relic restoration based on holographic display, characterized in that: Includes the following steps: Step 1: Multi-source data acquisition, capturing multi-view images of cultural relics and generating point cloud data of cultural relics by scanning with a scanner, including obtaining micro-morphological point cloud data by laser confocal scanning and obtaining large-scale point cloud data by structured light scanning. Step 2: Point cloud data fusion, which involves registering and fusing microscopic topographic point cloud data with large-scale point cloud data; Step 3: Generation of 3D cultural relic model. The 3D cultural relic model is obtained by reconstructing the surface using the registered and fused point cloud data. Then, texture mapping technology is used to apply texture mapping to the 3D cultural relic model based on the multi-view images, which adds texture features to the reconstructed 3D cultural relic model. Step 4: Holographic display. The 3D cultural relic model is converted into a holographic image and input into the holographic display system to recover the 3D model. Step 5: Holographic Restoration; Virtual Restoration Guidance: By restoring cultural relics in a holographic virtual space and recording the restoration process, restoration guidance is generated; Actual repair: The holographic display shows the real-time repair process.

2. The method according to claim 1, characterized in that: In step 2, the curvature-weighted ICP algorithm is used to rigidly register the micro-topography point cloud data and the large-scale point cloud data. The micro-topography point cloud data or the large-scale point cloud data is regarded as the source point cloud, and the other type of point cloud data is regarded as the target point cloud. The specific configuration process is as follows. For each source point in the source cloud Find the closest point in the target point cloud. ; Calculate the curvature of each source point Then, the curvature-based weights are calculated. A point with a larger curvature has a larger weight; and the curvature of a point is calculated by the curvature of local points in the local point cloud neighborhood. By minimizing the objective function Calculate the rotation matrix Translation vector ; Repeat the above steps until the preset convergence condition is met; objective function satisfy: ； in, It is the first in the target point cloud One point; It is the first in the source point cloud One point; It is a rotation matrix, representing the rotation transformation from the source point cloud to the target point cloud; It is a translation vector, representing the translation from the source point cloud to the target point cloud; It is Euclidean distance; It is a weight based on the local curvature of the point cloud, where N is the amount of point cloud data.

3. The method according to claim 1, characterized in that: Step 3 uses the Poisson reconstruction algorithm to reconstruct the surface and obtain the 3D model, specifically: First, calculate the normal vector of the fused point cloud data in step 2; Then, the normal vector field is obtained by solving the Poisson equation numerically, and a three-dimensional surface is constructed based on the normal vector field. The Poisson equation is as follows: ； In the formula, It is the divergence of the normal vector field. It refers to the entire three-dimensional space or the area where the point cloud is located. Represents a point. Represents the normal vector field. It is each point in the point cloud The normal vector, Points on a three-dimensional surface The normal vector field.

4. The method according to claim 1, characterized in that: The process of applying texture mapping technology to the 3D cultural relic model based on the multi-view images in step 3 is as follows: First, using the Agisoft Metashape algorithm, a depth map is generated from the multi-view image, where the depth value of each pixel is... This indicates the distance from the pixel to the camera; Secondly, using the Agisoft Metashape algorithm, combined with multi-view images and depth maps, dense point cloud data is generated. The three-dimensional spatial coordinates of each point in the point cloud are calculated based on the geometric relationship between the depth map and the image. Then, using UV mapping, each point in the dense point cloud data is mapped. Mapping from three-dimensional space to two-dimensional texture space Then, using the two-dimensional texture space coordinates and the depth values ​​of the pixels in the multi-view image in the depth map... The pixel values ​​of the multi-view image are mapped to the corresponding points of the point cloud through image registration technology. The pixel values ​​of the multi-view image are texture information that characterizes color and texture features, and u,v are coordinates in a two-dimensional texture space. Finally, based on the point matching relationship between the dense point cloud data and the 3D cultural relic model in step 3, texture information is assigned to the point cloud data of the 3D cultural relic model, thereby completing the texture mapping.

5. The method according to claim 4, characterized in that: Pixel depth value satisfy: Where f is the focal length of the camera; It is the baseline distance between the left and right cameras; It's parallax. These are pixel coordinates; If the surface of the artifact is spherical or approximately spherical, then spherical mapping is used for UV mapping, and the corresponding mapping model is as follows: in, It is a point Longitude to the center of the globe; It is a point Latitude to the center of the planet, and Used to determine the three-dimensional spatial coordinates of point P .

6. The method according to claim 1, characterized in that: The virtual repair guidance process described in step 5 is as follows: Within a virtual space, a 3D model is virtually repaired using virtual repair tools, and repair data is recorded in real time. Repair guidance is then generated based on the virtually repaired 3D model and the repair errors. in, To correct the error, It is a virtually repaired 3D model; It is a reference model of cultural relics, that is, the undamaged form of the original cultural relics, and a three-dimensional digital model of the original cultural relics as a benchmark; the restoration guide should at least include the restoration parameters of the restoration area. Among them, repair parameters are generated based on the repair records recorded in real time during the virtual repair process, and the repair parameters are adjusted based on the repair error to generate repair guidance.

7. The method according to claim 6, characterized in that: The actual restoration process described in step 5 is as follows: real-time restoration and real-time monitoring, that is, restoration guidance is provided based on restoration guidelines, and the actual restoration process of the cultural relic is recorded and displayed in real time in the holographic display system, and real-time comparison and monitoring are performed with the cultural relic reference model. The specific process is as follows: First, visual algorithms are used to compare the 3D model with the cultural relic reference model to locate the restoration area; in, This is the current damaged area data, i.e., the repair area; It is a three-dimensional model of the cultural relic after actual restoration, used to guide the actual restoration operation; It is a reference model for cultural relics, a three-dimensional digital model of the original cultural relics as a benchmark; Then, based on the virtual repaired 3D model and error correction By combining inverse modeling technology, a repair scheme for the repaired area is obtained; Finally, the restoration area is restored based on the restoration plan, and the restoration process and the restoration area are displayed in real time in the holographic display system. The restoration area is also dynamically updated in real time and compared with the cultural relic reference model in real time. Once the comparison error exceeds the warning value, the priority of the restoration area is adjusted through the dynamic weight update mechanism.

8. A system based on the method of any one of claims 1-7, characterized in that: include: The multi-source data acquisition module is used to capture multi-view images of cultural relics and generate point cloud data of cultural relics by scanning with a scanner. Among them, laser confocal scanning is used to obtain micro-morphological point cloud data and structured light scanning is used to obtain large-scale point cloud data. The point cloud data fusion module is used to register and fuse microscopic topographic point cloud data and large-scale point cloud data; The 3D cultural relic model generation module is used to reconstruct the surface using the registered and fused point cloud data to obtain a 3D cultural relic model. Then, based on the multi-view image, texture mapping technology is used to apply texture mapping to the 3D cultural relic model, which adds texture features to the reconstructed 3D cultural relic model. The holographic display module is used to convert 3D cultural relic models into holographic images and input them into the holographic display system to recover the 3D model; The holographic restoration module is used for virtual restoration guidance: it restores cultural relics through a holographic virtual space, records the restoration process, and generates restoration guidance; it is also used for holographic display of the real-time restoration process.

9. A system based on the method of any one of claims 1-7, characterized in that: include: Industrial cameras are used to capture multi-view images of cultural relics and transmit them to a data processor; A laser confocal scanner is used to scan cultural relics to obtain microscopic topographic point cloud data, which is then transmitted to a data processor. Structured light scanners are used to obtain large-scale point cloud data using structured light scanning and then transmit it to a data processor. A data processor is used for computer program implementation: point cloud data fusion and 3D cultural relic model generation, that is, registering and fusing micro-morphological point cloud data and large-scale point cloud data; and using the registered and fused point cloud data to reconstruct a surface to obtain a 3D cultural relic model, and then using texture mapping technology to apply texture mapping to the 3D cultural relic model based on the multi-view images, that is, adding texture features to the reconstructed 3D model. A holographic display system is used to receive the three-dimensional cultural relic model reconstructed by the data processor, convert it into a holographic image, and input it into the holographic display system to recover the three-dimensional model; The virtual restoration system is equipped with virtual restoration tools and a processing module. It is used to restore cultural relics in a holographic virtual space using virtual restoration tools, and the corresponding processing module records the restoration process and generates restoration instructions. The restoration monitoring system compares and monitors the cultural relics during the restoration process with a reference model of the cultural relics in real time.

10. A computer-readable storage medium, characterized in that: The computer program is stored and is invoked by the processor to implement: The steps of the method according to any one of claims 1-7.

Images