Method and system for establishing three-dimensional model of immovable cultural relics based on point cloud

Abstract

The application discloses a kind of based on point cloud's immovable cultural relic three-dimensional model establishment method and system, it is related to three-dimensional modeling technical field.A kind of based on point cloud's immovable cultural relic three-dimensional model establishment system, including have: data acquisition module, regional division module, point cloud acquisition module, point cloud processing module, model optimization module and model display module.The application is finely divided by generating processing area chart and collecting area chart to cultural relic surface;Processing area chart is labeled complexity, for density filtering and data optimization;Collecting area chart is used for dynamically adjusting point cloud acquisition density;More details are obtained by increasing acquisition density to complex area, and acquisition density is appropriately reduced in simple area, so that the problem of redundant data collection or detail missing in traditional method is effectively avoided.

Description

Technical Field

[0001] This invention relates to the field of 3D modeling technology, and in particular to a method and system for establishing 3D models of immovable cultural relics based on point clouds. Background Technology

[0002] In recent years, 3D modeling technology has been widely used in the field of cultural heritage protection and digitization. Immovable cultural relics, as important carriers of history and culture, are complex and diverse in form, distributed in open or complex natural environments, and long-term exposure may lead to the loss of details. Traditional manual measurement and modeling methods, due to their low efficiency, poor accuracy, and insufficient capture of surface features, are no longer sufficient to meet the high requirements of digital preservation of cultural relics.

[0003] Point cloud technology, as an efficient means of acquiring spatial three-dimensional data, provides a new solution for the detailed modeling of immovable cultural relics. However, existing point cloud modeling technologies still have the following problems when dealing with the complex geometry and surface texture of cultural relics: on the one hand, uniform acquisition of point cloud data can easily lead to data redundancy in simple areas and insufficient data in complex areas, thus affecting the detail representation of the overall model; on the other hand, the lack of intelligent data processing and optimization methods results in a large workload and low efficiency in subsequent model processing, and is prone to introducing errors.

[0004] To address the aforementioned issues, a technical solution is urgently needed that can dynamically adjust the point cloud data acquisition and processing strategy based on the surface characteristics of cultural relics. This solution combines the geometric shape and texture features of the relics to intelligently control the acquisition density for different areas, while simultaneously refining the point cloud data to ensure the integrity and accuracy of the 3D model. Summary of the Invention

[0005] This invention proposes a method and system for establishing 3D models of immovable cultural relics based on point clouds. By introducing a complexity analysis model and a region segmentation strategy, the complex areas of the cultural relics can be accurately labeled, the point cloud acquisition density and processing parameters can be dynamically adjusted, and a high-precision, high-fidelity 3D model of the cultural relics can be generated through multi-scale optimization and detail restoration techniques.

[0006] A method for establishing a 3D model of an immovable cultural relic based on point clouds includes the following steps:

[0007] Step S1: Obtain the size data of the cultural relic and construct the limitation range of the cultural relic model based on the size data; collect the color and depth image data of the cultural relic and construct the initial three-dimensional model of the cultural relic, and set the alignment and registration points for subsequent point cloud data registration.

[0008] Step S2: Input the initial 3D model and color and depth image data into the complexity analysis model for calculation to obtain the processing area map and acquisition area map of the initial 3D model; the processing area map is marked with complexity for subsequent density filtering of point cloud data; the acquisition area map is marked with average complexity for adjusting the point cloud acquisition density related parameters of the acquisition device.

[0009] For processing areas within the acquisition area map whose complexity exceeds a set proportion compared to the average complexity, mark them for re-acquisition and set re-acquisition registration points;

[0010] Step S3: Initially collect point cloud data of cultural relics using a laser scanner or other 3D scanning equipment. During the collection process, the point cloud collection density is intelligently controlled through the collection area map. For the processing area marked with recollection, point cloud data of the corresponding density is collected again. The recollected point cloud data is fitted with the initially collected point cloud data through recollection registration points to obtain reconstructed point cloud data. The reconstructed point cloud data is processed using the processing area map, and the density of the reconstructed point cloud data is filtered according to the complexity.

[0011] Step S4: Register the density-filtered point cloud data with the initial 3D model using its registration points to generate a registered 3D model;

[0012] Step S5: Perform integrity detection on the registered 3D model and process any missing parts that do not meet the integrity requirements; perform detail processing on the registered 3D model after missing part processing to obtain the final 3D model; the detail processing includes reconstruction of the initial state of the cultural relic, texture mapping, geometric enhancement, surface detail repair, multi-scale detail processing, and model smoothing and denoising;

[0013] Step S6: Export the final 3D model as a 3D file format and visualize it.

[0014] As a preferred embodiment of the present invention, the complexity analysis model includes the following structure:

[0015] The input layer receives the initial 3D model and color and depth image data, and preprocesses the image data, converting it into a format processed by a deep convolutional neural network. Color image preprocessing includes denoising, normalization, and feature extraction. Denoising is performed using Gaussian filtering, normalization normalizes pixel values ​​to [0,1], and feature extraction uses a convolutional neural network to extract texture and geometric features. Depth image preprocessing includes denoising, normalization, and inpainting, with inpainting performed using interpolation. The processed color and depth image data are then merged into a single image data set via channel concatenation. The merged image data is a four-dimensional tensor with dimensions 4*H*W, where the first three channels are the color image and the last channel is the depth image.

[0016] The region segmentation layer is used to divide the cultural relics into regions. It adopts an image segmentation algorithm based on U-Net, which divides the image data fused from the input layer into multiple regions based on texture and geometric features. The segmentation process is based on the following objective function:

[0017] L seg =αL dice +βL CE L seg L represents the total loss value. dice For Dice's loss, L CE The cross-entropy loss is represented by α and β, which are the corresponding weighting coefficients, respectively.

[0018] A sub-complexity calculation layer is used to calculate the sub-complexity of each region, wherein the sub-complexity includes:

[0019] Geometric complexity C geo Based on the curvature distribution of the region, the calculation formula is as follows: Where K i Let A represent the curvature and A represent the area of ​​the region.

[0020] Texture complexity C tex Based on the image grayscale gradient distribution, the calculation formula is as follows: in This represents the gradient intensity of a pixel, where N represents the number of pixels;

[0021] Structural complexity C str It is obtained through the density of edges and contours within the region, and the calculation formula is:

[0022] Where K max and K min The maximum and minimum curvatures within the region;

[0023] The data fusion and adjustment layer is used to fuse the various sub-complexities. It employs an adaptive weighting method, adjusting the weights based on learned dynamic weights to obtain the comprehensive complexity C of each region. tot The formula is:

[0024] C tot =W geo (X)C geo +W tex (X)C tex +W str (X)C str W geo (X), W geo (X), W str(X) are adaptive weighting coefficients based on input features, corresponding to geometric complexity, texture complexity, and structural complexity, respectively;

[0025] The graph generation layer is used to generate a processing region map and a collection region map based on the complexity of the fusion. The processing region map is a set of processing region maps marked with complexity for density filtering. The collection region map is a set of non-overlapping region segmentation maps, which contains processing region information obtained by matching the comprehensive complexity of the regions.

[0026] The re-sampling area marking layer is used to mark areas with complexity exceeding a set ratio as re-sampling areas based on the acquired area map, and to set re-sampling registration points. The point cloud density P of the re-sampling area is set. d Adjustments are made based on the overall complexity of the region, and the formula is: P d =P0(1+kC) avg ), where P0 is the base point cloud density, k is the adjustment coefficient, and C avg For average complexity;

[0027] The output layer is used to output the complexity analysis results, including a processing area map that marks the complexity of each region and a collection area map that contains resampling and registration points and processing area information.

[0028] As a preferred embodiment of the present invention, the acquisition of adaptive weighting coefficients includes the following steps:

[0029] Multidimensional features are extracted from color images and depth images, including texture features, geometric features, and depth information.

[0030] Features from color and depth images are fused into a unified feature vector. A deep neural network is then used to process the fused feature vector to learn weighting coefficients for different features. The deep neural network dynamically calculates the corresponding weighting coefficients based on the features and complexity of each region, using the following formula:

[0031] W geo (X), W tex (X), W str (X)=NN(f input ), where NN() represents a deep neural network, f input W is the input feature vector. geo (X), W geo (X), W str (X) represents the adaptive weighting coefficients of the output, which are the weighting coefficients of the geometric complexity, texture complexity and structural complexity of the region, respectively;

[0032] The weighting coefficients are optimized using the backpropagation algorithm, resulting in a final synthesis complexity of C0. totIt matches the actual complexity of the surface features of cultural relics; the optimization objective of the backpropagation algorithm is the mean squared error loss function.

[0033] As a preferred embodiment of the present invention, the acquisition area map is a non-overlapping region segmentation map set, including the following:

[0034] The initial 3D model is unfolded into a 2D layer using 3D model deconstruction technology, and the structural technology maps the 3D surface onto a 2D plane; the size of the minimum acquisition block is set according to the parameters of the acquisition device.

[0035] Starting from a corner of the 2D layer, select a starting point and construct the first collection region from that point. The initial size of the collection region is determined by the smallest collection block. Subsequently, expand the collection region around its perimeter sequentially. During the expansion process, calculate the proportion of the impact of the complexity of the added region on the average complexity of the collection region, using the following formula: Where ΔC avg This represents the change in the average complexity of the expanded acquisition area, where "new area" represents the newly added expanded area, and X... i C represents all areas within the data collection area. total (X i The value represents the average complexity of the new region. During the expansion process, if the complexity of the newly added region has less than a set proportion of the impact on the average complexity of the acquisition region, the expansion is accepted; otherwise, the current expansion is stopped. The minimum acquisition block is obtained through the parameters of the acquisition device.

[0036] The expansion process continues until there are no further expansion areas around the perimeter; for all undivided areas, the above steps are repeated until the entire initial 3D model is divided into non-overlapping areas; for each acquisition area, the point cloud acquisition density is matched according to the average complexity as the processing area information.

[0037] As a preferred embodiment of the present invention, the training of the complexity analysis model includes:

[0038] The collected cultural relic dataset is divided into a training set, a validation set, and a test set. The training set is used to train the complexity analysis model, the validation set is used to tune parameters and evaluate model performance, and the test set is used for final model evaluation and performance verification.

[0039] Preprocessing and feature annotation are performed on the divided training set, validation set, and test set; the feature annotation is the complexity label of the cultural relic, including geometric complexity, texture complexity, and structural complexity;

[0040] Using a deep convolutional neural network architecture pre-trained based on image recognition tasks, we initialize the various network layers and parameters of the model, and initialize the weights of the model to obtain the initial complexity analysis model.

[0041] The initial complexity analysis model is trained using a training set. The training process includes forward propagation, loss calculation, backpropagation, and iterative training. The model is evaluated using a validation set. If the loss function value in the evaluation results is not up to standard, the hyperparameters of the model are adjusted or regularization is added. After training is completed, the optimized complexity analysis model is obtained.

[0042] The optimization complexity analysis model is evaluated using a test set. If the evaluation result meets the standard, the final complexity analysis model is obtained; otherwise, the training steps are repeated to obtain the optimization complexity analysis model. The evaluation is performed using the F1 score and mean squared error of the optimization complexity analysis model.

[0043] As a preferred embodiment of the present invention, step S3 further includes:

[0044] Point cloud registration and fusion: The iterative nearest point algorithm is used to fit and align the reacquired point cloud data with the initially acquired point cloud data; the registration algorithm matches the point cloud data based on the reacquired registration points, minimizes the spatial distance between the two sets of point clouds, and obtains the reconstructed point cloud data;

[0045] Density filtering involves performing local point cloud density analysis on the merged point cloud data to evaluate the point cloud distribution in each processing region. The evaluation is conducted by calculating the number of points per unit area within the processing region. A point cloud density threshold set is obtained based on the accuracy requirements of the processing region map and the 3D model. This threshold set is then used to perform density filtering on each processing region in the initial 3D model.

[0046] As a preferred embodiment of the present invention, generating the registered 3D model includes:

[0047] The density-optimized point cloud data is registered with the initial 3D model. The point cloud data is then aligned using the iterative nearest point algorithm based on the registration points, and the registration accuracy is optimized through repeated iterations to minimize the registration error. The registered point cloud and the 3D model are then synthesized to generate a registered 3D model.

[0048] As a preferred embodiment of the present invention, the integrity detection includes:

[0049] By calculating the difference between the surface normal of the registered 3D model and the expected surface normal, missing regions are identified, the identified missing regions are labeled, and the labeled missing regions are processed, including data supplementation or re-acquisition.

[0050] Repeat the above steps until there are no missing areas in the registered 3D model, thus completing the integrity check of the registered 3D model.

[0051] As a preferred embodiment of the present invention, the detailed processing of the resampled and supplemented registered 3D model to obtain the final 3D model includes the following steps:

[0052] The initial state reconstruction of the registered 3D model is selectively performed according to the needs to restore the original form of the cultural relic before collection. The initial state reconstruction is performed by combining the texture, geometric features and historical documents in the collected data to reconstruct the historical state of the cultural relic.

[0053] Texture mapping is performed on the registered 3D model. The texture mapping includes matching color image data with the 3D surface to enhance the texture details of the artifact surface. The enhancement includes high-resolution texture generation, color restoration, lighting adjustment, and noise removal.

[0054] The registered 3D model is geometrically enhanced, which repairs and strengthens the surface details of the artifact through 3D reconstruction algorithms; its geometry is restored using surface smoothing or polygon reconstruction techniques.

[0055] The details of the registered 3D model are restored, including restoring the texture, carvings or cracks on the surface of the artifact;

[0056] The registered 3D model undergoes multi-scale detail processing, which involves optimizing the model's geometric shape and texture details layer by layer.

[0057] The registered 3D model is smoothed and denoised. The smoothing and denoising are performed by removing noise and irregular geometric shapes using Laplacian smoothing technology.

[0058] A system for creating 3D models of immovable cultural relics based on point clouds includes the following modules:

[0059] Data acquisition module: used to acquire the size data, color image and depth image data of cultural relics, and to construct the limitation range of the cultural relic model based on the size data; to build the initial 3D model; and to set the alignment and registration points to prepare for subsequent point cloud data registration.

[0060] Region segmentation module: This module is used to input the initial 3D model and color and depth image data into the complexity analysis model for calculation, obtain the processing region map and the acquisition region map, mark the complexity and generate the corresponding acquisition density parameters; based on the region complexity analysis results, it marks the regions that need to be resampled and sets the resampled registration points;

[0061] Point cloud acquisition module: used to initially acquire point cloud data of cultural relics using a laser scanner or other 3D scanning equipment. During the acquisition process, the point cloud density is adjusted according to the acquisition area map. Point cloud data with the corresponding density is acquired again for areas marked for reacquisition. The reacquisitioned point cloud data is then fitted with the initially acquired point cloud data through registration points to obtain reconstructed point cloud data.

[0062] Point cloud processing module: Used to process the reconstructed point cloud data using the processing area map, and to perform density filtering of the point cloud based on complexity; and to register the density-filtered point cloud data with the initial 3D model by aligning registration points to generate a registered 3D model;

[0063] Model optimization module: Used to perform integrity detection on the registered 3D model and handle missing parts that do not meet the integrity requirements; it also performs detail repair on the 3D model after missing parts are handled, including reconstruction of the initial state of the cultural relic, texture mapping, geometric enhancement, surface detail repair and multi-scale detail processing;

[0064] Model Display Module: Used to export the final 3D model as a standard 3D file format and visualize it, providing 3D data support for cultural relic protection, display or research.

[0065] The present invention has the following advantages:

[0066] This invention performs fine-grained subdivision of the artifact surface by generating a processing area map and a collection area map. The processing area map is marked with high complexity and is used for density filtering and data optimization. The collection area map is used to dynamically adjust the point cloud collection density. More details are obtained by increasing the collection density in areas with high complexity, while the collection density is appropriately reduced in simple areas. This effectively avoids the problems of data redundancy or lack of detail in traditional methods and greatly improves the efficiency and accuracy of data collection.

[0067] This invention dynamically adjusts the point cloud acquisition density based on multidimensional complexity, enabling intelligent registration and density filtering of point cloud data for heavily acquired areas. By processing the complexity data in the region map, it dynamically optimizes the density distribution of point cloud data, ensuring the detail representation of high-complexity areas while reducing the amount of redundant data in simple areas, significantly improving the efficiency and accuracy of point cloud data processing.

[0068] This invention introduces a deep learning-driven complexity analysis model to reduce manual intervention and achieve automation and intelligence in the data collection and modeling process. This data-driven strategy enables the method to adaptively adjust the modeling process when faced with different forms of cultural relics, greatly improving efficiency and accuracy. Attached Figure Description

[0069] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings in the following description are only schematic diagrams of the present invention. For those skilled in the art, other drawings can be obtained based on the provided drawings without creative effort.

[0070] Figure 1 This is a schematic diagram of a point cloud-based system for creating a 3D model of an immovable cultural relic, as used in an embodiment of the present invention. Detailed Implementation

[0071] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this invention, and not all embodiments. Based on the embodiments of this invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this invention.

[0072] Example 1: A method for establishing a 3D model of an immovable cultural relic based on point clouds, comprising the following steps:

[0073] Step S1: Obtain the size data of the cultural relic and construct the limitation range of the cultural relic model based on the size data; collect the color and depth image data of the cultural relic and construct the initial three-dimensional model of the cultural relic, and set the alignment and registration points for subsequent point cloud data registration.

[0074] In this step, the first step is to perform preliminary dimensional measurements and spatial constraint analysis on the artifact. Specifically, high-precision laser rangefinders, photogrammetry, or other measuring equipment are used to acquire the artifact's physical dimensional data. This dimensional data includes geometric information such as the artifact's length, width, height, and surface curvature. This data is used to construct the artifact's constraint range, providing spatial reference and boundary conditions for subsequent 3D modeling and point cloud data acquisition.

[0075] Based on the acquired dimensional data, the system can automatically generate a preliminary geometric model framework for the cultural relic and determine the constraints on the relic's dimensions according to its specific form (such as planar, curved, or complex structures). These constraints include not only physical dimensions but also the complexity analysis of the relic's surface, which is used for subsequent point cloud density adjustment and processing area division.

[0076] Based on the preliminary geometric model of the artifact, the next step is to acquire color and depth image data. High-resolution photographic equipment is used to acquire color images, recording the true texture and color information of the artifact's surface. Simultaneously, depth images are acquired using depth cameras or laser scanners to obtain three-dimensional spatial information of the artifact's surface. Depth images, by measuring the distance between each pixel and the camera, provide detailed data on the spatial location of the artifact's surface.

[0077] These color and depth image data will serve as input. After image preprocessing, they will be used together with the preliminary geometric model of the artifact to construct an initial 3D model. To ensure accurate alignment of data obtained from different acquisition devices, alignment registration points are set. These registration points are key points with distinct features on the artifact's surface, serving as a benchmark for subsequent point cloud data registration and fusion. These registration points are generally distributed in areas of significant geometric variation on the artifact's surface, ensuring accurate docking between the point cloud data and the 3D model.

[0078] Furthermore, to facilitate subsequent point cloud data processing and optimization, the system intelligently plans the subsequent point cloud acquisition areas based on the preliminary model of the artifact and the collected image data. By analyzing the complexity and surface features of the artifact, the system can automatically mark the areas that need to be prioritized for acquisition, as well as the low-complexity areas that can be ignored, thereby improving data acquisition efficiency and accuracy.

[0079] Step S2: Input the initial 3D model and color and depth image data into the complexity analysis model for calculation to obtain the processing area map and acquisition area map of the initial 3D model; the processing area map is marked with complexity for subsequent density filtering of point cloud data; the acquisition area map is marked with average complexity for adjusting the point cloud acquisition density related parameters of the acquisition device.

[0080] For processing areas within the acquisition area map whose complexity exceeds a set proportion compared to the average complexity, mark them for re-acquisition and set re-acquisition registration points;

[0081] The complexity analysis model includes the following structure:

[0082] The input layer receives the initial 3D model and color and depth image data, and preprocesses the image data, converting it into a format processed by a deep convolutional neural network. Color image preprocessing includes denoising, normalization, and feature extraction. Denoising is performed using Gaussian filtering, normalization normalizes pixel values ​​to [0,1], and feature extraction uses a convolutional neural network to extract texture and geometric features. Specifically: texture features (such as local texture direction, color contrast, and local image gradient) are extracted from the color image. Geometric features (such as surface curvature, normal direction, and local structure) are extracted from the depth image. Global information about the artifact area, such as overall surface smoothness and point cloud sparsity, is extracted.

[0083] Preprocessing of depth images includes denoising, normalization, and inpainting, with inpainting performed using interpolation. The processed color image data and depth image data are then merged into a single image data by channel concatenation. The merged image data is a four-dimensional tensor with dimensions 4*H*W, where the first three channels are the color image and the last channel is the depth image.

[0084] The region segmentation layer is used to divide the cultural relics into regions. It adopts an image segmentation algorithm based on U-Net, which divides the image data fused from the input layer into multiple regions based on texture and geometric features. The segmentation process is based on the following objective function:

[0085] L seg =αL dice +βL CE L seg L represents the total loss value. dice For Dice's loss, L CE The cross-entropy loss is represented by α and β, which are the corresponding weighting coefficients, respectively.

[0086] A sub-complexity calculation layer is used to calculate the sub-complexity of each region, wherein the sub-complexity includes:

[0087] Geometric complexity C geo Based on the curvature distribution of the region, the calculation formula is as follows: Where K i Let A represent the curvature and A represent the area of ​​the region.

[0088] Texture complexity C tex Based on the image grayscale gradient distribution, the calculation formula is as follows: in This represents the gradient intensity of a pixel, where N represents the number of pixels;

[0089] Structural complexity C str It is obtained through the density of edges and contours within the region, and the calculation formula is:

[0090] Where K max and K min The maximum and minimum curvatures within the region;

[0091] The data fusion and adjustment layer is used to fuse the various sub-complexities. It employs an adaptive weighting method, adjusting the weights based on learned dynamic weights to obtain the comprehensive complexity C of each region. tot The formula is:

[0092] C tot =W geo (X)C geo +W tex (X)C tex +W str (X)C str W geo (X), W geo (X), W str (X) are adaptive weighting coefficients based on input features, corresponding to geometric complexity, texture complexity, and structural complexity, respectively;

[0093] The graph generation layer is used to generate a processing region map and a collection region map based on the complexity of the fusion. The processing region map is a set of processing region maps marked with complexity for density filtering. The collection region map is a set of non-overlapping region segmentation maps, which contains processing region information obtained by matching the comprehensive complexity of the regions.

[0094] The re-sampling area marking layer is used to mark areas with complexity exceeding a set ratio as re-sampling areas based on the acquired area map, and to set re-sampling registration points. The point cloud density P of the re-sampling area is set. d Adjustments are made based on the overall complexity of the region, using the following formula:

[0095] P d =P0(1+kC) avg ), where P0 is the base point cloud density, k is the adjustment coefficient, and C avg For average complexity;

[0096] The output layer is used to output the complexity analysis results, including a processing area map that marks the complexity of each region and a collection area map that contains resampling and registration points and processing area information.

[0097] Obtaining the adaptive weighting coefficients includes the following steps:

[0098] Multidimensional features are extracted from color images and depth images, including texture features, geometric features, and depth information.

[0099] Features from color and depth images are fused into a unified feature vector. A deep neural network is then used to process the fused feature vector to learn weighting coefficients for different features. The deep neural network dynamically calculates the corresponding weighting coefficients based on the features and complexity of each region, using the following formula:

[0100] W geo (X), W tex (X), W str (X)=NN(f input ), where NN() represents a deep neural network, f input W is the input feature vector. geo (X), W geo (X), W str (X) represents the adaptive weighting coefficients of the output, which are the weighting coefficients of the geometric complexity, texture complexity and structural complexity of the region, respectively;

[0101] The weighting coefficients are optimized using the backpropagation algorithm, resulting in a final synthesis complexity of C0. tot It matches the actual complexity of the surface features of cultural relics; the optimization objective of the backpropagation algorithm is the mean squared error loss function.

[0102] The collected area map is a non-overlapping region segmentation map set, including the following:

[0103] The initial 3D model is unfolded into a 2D layer using 3D model deconstruction technology, and the structural technology maps the 3D surface onto a 2D plane; the size of the minimum acquisition block is set according to the parameters of the acquisition device.

[0104] Starting from a corner of the 2D layer, select a starting point and construct the first collection region from that point. The initial size of the collection region is determined by the smallest collection block. Subsequently, expand the collection region around its perimeter sequentially. During the expansion process, calculate the proportion of the impact of the complexity of the added region on the average complexity of the collection region, using the following formula: Where ΔC avg This represents the change in the average complexity of the expanded acquisition area, where "new area" represents the newly added expanded area, and X... i C represents all areas within the data collection area. total (X i The value represents the average complexity of the new region. During the expansion process, if the complexity of the newly added region has less than a set proportion of the impact on the average complexity of the acquisition region, the expansion is accepted; otherwise, the current expansion is stopped. The minimum acquisition block is obtained through the parameters of the acquisition device.

[0105] The expansion process continues until there are no further expansion areas around the perimeter; for all undivided areas, the above steps are repeated until the entire initial 3D model is divided into non-overlapping areas; for each acquisition area, the point cloud acquisition density is matched according to the average complexity as the processing area information.

[0106] Training the complexity analysis model includes:

[0107] The collected cultural relic dataset is divided into a training set, a validation set, and a test set. The training set is used to train the complexity analysis model, the validation set is used to tune parameters and evaluate model performance, and the test set is used for final model evaluation and performance verification.

[0108] Preprocessing and feature annotation are performed on the divided training set, validation set, and test set; the feature annotation is the complexity label of the cultural relic, including geometric complexity, texture complexity, and structural complexity;

[0109] Using a deep convolutional neural network architecture pre-trained based on image recognition tasks, we initialize the various network layers and parameters of the model, and initialize the weights of the model to obtain the initial complexity analysis model.

[0110] The initial complexity analysis model is trained using a training set. The training process includes forward propagation, loss calculation, backpropagation, and iterative training. The model is evaluated using a validation set. If the loss function value in the evaluation results is not up to standard, the hyperparameters of the model are adjusted or regularization is added. After training is completed, the optimized complexity analysis model is obtained.

[0111] The optimization complexity analysis model is evaluated using a test set. If the evaluation result meets the standard, the final complexity analysis model is obtained; otherwise, the training steps are repeated to obtain the optimization complexity analysis model. The evaluation is performed using the F1 score and mean squared error of the optimization complexity analysis model.

[0112] Step S3: Initially collect point cloud data of cultural relics using a laser scanner or other 3D scanning equipment. During the collection process, the point cloud collection density is intelligently controlled through the collection area map. For the processing area marked with recollection, point cloud data of the corresponding density is collected again. The recollected point cloud data is fitted with the initially collected point cloud data through recollection registration points to obtain reconstructed point cloud data. The reconstructed point cloud data is processed using the processing area map, and the density of the reconstructed point cloud data is filtered according to the complexity.

[0113] Point cloud registration and fusion: The iterative nearest point algorithm is used to fit and align the reacquired point cloud data with the initially acquired point cloud data; the registration algorithm matches the point cloud data based on the reacquired registration points, minimizes the spatial distance between the two sets of point clouds, and obtains the reconstructed point cloud data;

[0114] Density filtering involves performing local point cloud density analysis on the merged point cloud data to evaluate the point cloud distribution in each processing region. The evaluation is conducted by calculating the number of points per unit area within the processing region. A point cloud density threshold set is obtained based on the accuracy requirements of the processing region map and the 3D model. This threshold set is then used to perform density filtering on each processing region in the initial 3D model.

[0115] Step S4: Register the density-filtered point cloud data with the initial 3D model using its registration points to generate a registered 3D model;

[0116] Generating the registered 3D model includes:

[0117] The density-optimized point cloud data is registered with the initial 3D model. The point cloud data is then aligned using the iterative nearest point algorithm based on the registration points, and the registration accuracy is optimized through repeated iterations to minimize the registration error. The registered point cloud and the 3D model are then synthesized to generate a registered 3D model.

[0118] Step S5: Perform integrity detection on the registered 3D model and process any missing parts that do not meet the integrity requirements; perform detail processing on the registered 3D model after missing part processing to obtain the final 3D model; the detail processing includes reconstruction of the initial state of the cultural relic, texture mapping, geometric enhancement, surface detail repair, multi-scale detail processing, and model smoothing and denoising;

[0119] Integrity testing includes:

[0120] By calculating the difference between the surface normal of the registered 3D model and the expected surface normal, missing regions are identified, the identified missing regions are labeled, and the labeled missing regions are processed, including data supplementation or re-acquisition.

[0121] Repeat the above steps until there are no missing areas in the registered 3D model, thus completing the integrity check of the registered 3D model.

[0122] The detailed processing of the resampled and supplemented registered 3D model to obtain the final 3D model includes the following steps:

[0123] The initial state reconstruction of the registered 3D model is selectively performed according to the needs to restore the original form of the cultural relic before collection. The initial state reconstruction is performed by combining the texture, geometric features and historical documents in the collected data to reconstruct the historical state of the cultural relic.

[0124] Texture mapping is performed on the registered 3D model. The texture mapping includes matching color image data with the 3D surface to enhance the texture details of the artifact surface. The enhancement includes high-resolution texture generation, color restoration, lighting adjustment, and noise removal.

[0125] The registered 3D model is geometrically enhanced, which repairs and strengthens the surface details of the artifact through 3D reconstruction algorithms; its geometry is restored using surface smoothing or polygon reconstruction techniques.

[0126] The details of the registered 3D model are restored, including restoring the texture, carvings or cracks on the surface of the artifact;

[0127] The registered 3D model undergoes multi-scale detail processing, which involves optimizing the model's geometric shape and texture details layer by layer.

[0128] The registered 3D model is smoothed and denoised, and the smoothing and denoising is performed by removing noise and irregular geometric shapes using Laplacian smoothing technology.

[0129] Step S6: Export the final 3D model as a 3D file format and visualize it.

[0130] Example 2: A system for creating 3D models of immovable cultural relics based on point clouds. (See also...) Figure 1 As shown, it includes the following modules:

[0131] Data acquisition module: used to acquire the size data, color image and depth image data of cultural relics, and to construct the limitation range of the cultural relic model based on the size data; to build the initial 3D model; and to set the alignment and registration points to prepare for subsequent point cloud data registration.

[0132] Region segmentation module: This module is used to input the initial 3D model and color and depth image data into the complexity analysis model for calculation, obtain the processing region map and the acquisition region map, mark the complexity and generate the corresponding acquisition density parameters; based on the region complexity analysis results, it marks the regions that need to be resampled and sets the resampled registration points;

[0133] Point cloud acquisition module: used to initially acquire point cloud data of cultural relics using a laser scanner or other 3D scanning equipment. During the acquisition process, the point cloud density is adjusted according to the acquisition area map. Point cloud data with the corresponding density is acquired again for areas marked for reacquisition. The reacquisitioned point cloud data is then fitted with the initially acquired point cloud data through registration points to obtain reconstructed point cloud data.

[0134] Point cloud processing module: Used to process the reconstructed point cloud data using the processing area map, and to perform density filtering of the point cloud based on complexity; and to register the density-filtered point cloud data with the initial 3D model by aligning registration points to generate a registered 3D model;

[0135] Model optimization module: Used to perform integrity detection on the registered 3D model and handle missing parts that do not meet the integrity requirements; it also performs detail repair on the 3D model after missing parts are handled, including reconstruction of the initial state of the cultural relic, texture mapping, geometric enhancement, surface detail repair and multi-scale detail processing;

[0136] Model Display Module: Used to export the final 3D model as a standard 3D file format and visualize it, providing 3D data support for cultural relic protection, display or research.

[0137] The specific embodiments described above further illustrate the purpose, technical solution, and beneficial effects of the present invention. It should be understood that the above description is only a specific embodiment of the present invention and is not intended to limit the scope of protection of the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A method for establishing a 3D model of an immovable cultural relic based on point clouds, characterized in that, Includes the following steps: Step S1: Obtain the size data of the cultural relic and construct the limitation range of the cultural relic model based on the size data; collect the color and depth image data of the cultural relic and construct the initial three-dimensional model of the cultural relic, and set the alignment and registration points for subsequent point cloud data registration. Step S2: Input the initial 3D model and color and depth image data into the complexity analysis model for calculation to obtain the processing area map and acquisition area map of the initial 3D model; the processing area map is marked with complexity for subsequent density filtering of point cloud data; the acquisition area map is marked with average complexity for adjusting the point cloud acquisition density related parameters of the acquisition device. For processing areas within the acquisition area map whose complexity exceeds a set proportion compared to the average complexity, mark them for re-acquisition and set re-acquisition registration points; Step S3: Initially collect point cloud data of cultural relics using a laser scanner or other 3D scanning equipment. During the collection process, the point cloud collection density is intelligently controlled through the collection area map. Point cloud data of the corresponding density is collected again for the processing area marked with resampling; the resampling point cloud data is fitted with the initially collected point cloud data through resampling registration points to obtain reconstructed point cloud data; the reconstructed point cloud data is processed using the processing area map, and density filtering of the reconstructed point cloud data is performed based on complexity. Step S4: Register the density-filtered point cloud data with the initial 3D model using its registration points to generate a registered 3D model; Step S5: Perform integrity detection on the registered 3D model and process any missing parts that do not meet the integrity requirements; perform detail processing on the registered 3D model after missing part processing to obtain the final 3D model; the detail processing includes reconstruction of the initial state of the cultural relic, texture mapping, geometric enhancement, surface detail repair, multi-scale detail processing, and model smoothing and denoising; Step S6: Export the final 3D model as a 3D file format and visualize it.

2. The method for establishing a 3D model of an immovable cultural relic based on point cloud as described in claim 1, characterized in that, The complexity analysis model includes the following structure: The input layer receives the initial 3D model and color and depth image data, and preprocesses the image data, converting it into a format processed by a deep convolutional neural network. Color image preprocessing includes denoising, normalization, and feature extraction. Denoising is performed using Gaussian filtering, normalization normalizes pixel values ​​to [0,1], and feature extraction uses a convolutional neural network to extract texture and geometric features. Depth image preprocessing includes denoising, normalization, and inpainting, with inpainting performed using interpolation. The processed color and depth image data are then merged into a single image data set via channel concatenation. The merged image data is a four-dimensional tensor with dimensions 4*H*W, where the first three channels are the color image and the last channel is the depth image. The region segmentation layer is used to divide the cultural relics into regions. It adopts an image segmentation algorithm based on U-Net, which divides the image data fused from the input layer into multiple regions based on texture and geometric features. The segmentation process is based on the following objective function: L seg =αL dice +βL CE L seg L represents the total loss value. dice For Dice's loss, L CE The cross-entropy loss is represented by α and β, which are the corresponding weighting coefficients, respectively. A sub-complexity calculation layer is used to calculate the sub-complexity of each region, wherein the sub-complexity includes: Geometric complexity C geo Based on the curvature distribution of the region, the calculation formula is as follows: Where K i Let A represent the curvature and A represent the area of ​​the region. Texture complexity C tex Based on the image grayscale gradient distribution, the calculation formula is as follows: in This represents the gradient intensity of a pixel, where N represents the number of pixels; Structural complexity C str It is obtained through the density of edges and contours within the region, and the calculation formula is: Where K max and K min The maximum and minimum curvatures within the region; The data fusion and adjustment layer is used to fuse the various sub-complexities. It employs an adaptive weighting method, adjusting the weights based on learned dynamic weights to obtain the comprehensive complexity C of each region. tot The formula is: C tot =W geo (X)C geo +W tex (X)C tex +W str (X)C str W geo (X), W geo (X), W str (X) are adaptive weighting coefficients based on input features, corresponding to geometric complexity, texture complexity, and structural complexity, respectively; The graph generation layer is used to generate a processing region map and a collection region map based on the complexity of the fusion. The processing region map is a set of processing region maps marked with complexity for density filtering. The collection region map is a set of non-overlapping region segmentation maps, which contains processing region information obtained by matching the comprehensive complexity of the regions. The re-sampling area marking layer is used to mark areas with complexity exceeding a set ratio as re-sampling areas based on the acquired area map, and to set re-sampling registration points. The point cloud density P of the re-sampling area is set. d Adjustments are made based on the overall complexity of the region; the formula is as follows: P d =P0(1+kC) avg ), where P0 is the base point cloud density, k is the adjustment coefficient, and C avg For average complexity; The output layer is used to output the complexity analysis results, including a processing area map that marks the complexity of each region and a collection area map that contains resampling registration points and processing area information.

3. The method for establishing a 3D model of an immovable cultural relic based on point cloud as described in claim 2, characterized in that, Obtaining the adaptive weighting coefficients includes the following steps: Multidimensional features are extracted from color images and depth images, including texture features, geometric features, and depth information. Features from color and depth images are fused into a unified feature vector. A deep neural network is then used to process the fused feature vector to learn weighting coefficients for different features. The deep neural network dynamically calculates the corresponding weighting coefficients based on the features and complexity of each region, using the following formula: W geo (X), W tex (X), W str (X)=NN(f input ), where NN() represents a deep neural network, f input W is the input feature vector. geo (X), W geo (X), W str (X) represents the adaptive weighting coefficients of the output, which are the weighting coefficients of the geometric complexity, texture complexity and structural complexity of the region, respectively; The weighting coefficients are optimized using the backpropagation algorithm, resulting in a final synthesis complexity of C0. tot It matches the actual complexity of the surface features of cultural relics; the optimization objective of the backpropagation algorithm is the mean squared error loss function.

4. The method for establishing a three-dimensional model of an immovable cultural relic based on point cloud as described in claim 2, characterized in that, The collected area map is a non-overlapping region segmentation map set, including the following: The initial 3D model is unfolded into a 2D layer using 3D model deconstruction technology, and the structural technology maps the 3D surface onto a 2D plane; the size of the minimum acquisition block is set according to the parameters of the acquisition device. Select a starting point from the corner of the 2D layer, and start building the first collection area from that point; The initial size of the acquisition area is determined by the smallest acquisition block. Subsequently, the area is expanded sequentially around its perimeter. During the expansion process, the impact of the complexity of the added area on the average complexity of the acquisition area is calculated using the following formula: Where ΔC avg This represents the change in the average complexity of the expanded acquisition area, where "new area" represents the newly added expanded area, and X... i C represents all areas within the data collection area. total (X i ) represents the average complexity of the new region; During the expansion process, if the complexity of the newly added region has less than a set proportion of the impact on the average complexity of the acquisition region, the expansion is accepted; otherwise, the current expansion is stopped. The minimum acquisition block is obtained through the parameters of the acquisition device. The expansion process continues until there are no further expansion areas around the perimeter; for all undivided areas, the above steps are repeated until the entire initial 3D model is divided into non-overlapping areas; for each acquisition area, the point cloud acquisition density is matched according to the average complexity as the processing area information.

5. The method for establishing a three-dimensional model of an immovable cultural relic based on point clouds according to claim 2, characterized in that, Training the complexity analysis model includes: The collected cultural relic dataset is divided into a training set, a validation set, and a test set. The training set is used to train the complexity analysis model, the validation set is used to tune parameters and evaluate model performance, and the test set is used for final model evaluation and performance verification. Preprocessing and feature annotation are performed on the divided training set, validation set, and test set; the feature annotation is the complexity label of the cultural relic, including geometric complexity, texture complexity, and structural complexity; Using a deep convolutional neural network architecture pre-trained based on image recognition tasks, we initialize the various network layers and parameters of the model, and initialize the weights of the model to obtain the initial complexity analysis model. The initial complexity analysis model is trained using a training set. The training process includes forward propagation, loss calculation, backpropagation, and iterative training. The model is evaluated using a validation set. If the loss function value in the evaluation results is not up to standard, the hyperparameters of the model are adjusted or regularization is added. After training is completed, the optimized complexity analysis model is obtained. The optimization complexity analysis model is evaluated using a test set. If the evaluation result meets the standard, the final complexity analysis model is obtained; otherwise, the training steps are repeated to obtain the optimization complexity analysis model. The evaluation is performed using the F1 score and mean squared error of the optimization complexity analysis model.

6. The method for establishing a three-dimensional model of an immovable cultural relic based on point cloud as described in claim 1, characterized in that, Step S3 further includes: Point cloud registration and fusion: The iterative nearest point algorithm is used to fit and align the reacquired point cloud data with the initially acquired point cloud data; the registration algorithm matches the point cloud data based on the reacquired registration points, minimizes the spatial distance between the two sets of point clouds, and obtains the reconstructed point cloud data; Density filtering involves performing local point cloud density analysis on the merged point cloud data to evaluate the point cloud distribution in each processing region. The evaluation is conducted by calculating the number of points per unit area within the processing region. A point cloud density threshold set is obtained based on the accuracy requirements of the processing region map and the 3D model. This threshold set is then used to perform density filtering on each processing region in the initial 3D model.

7. The method for establishing a three-dimensional model of an immovable cultural relic based on point cloud as described in claim 1, characterized in that, Generating the registered 3D model includes: The density-optimized point cloud data is registered with the initial 3D model. The point cloud data is then aligned using the iterative nearest point algorithm based on the registration points, and the registration accuracy is optimized through repeated iterations to minimize the registration error. The registered point cloud and the 3D model are then synthesized to generate a registered 3D model.

8. The method for establishing a 3D model of an immovable cultural relic based on point cloud as described in claim 1, characterized in that, Integrity testing includes: By calculating the difference between the surface normal of the registered 3D model and the expected surface normal, missing regions are identified, the identified missing regions are labeled, and the labeled missing regions are processed, including data supplementation or re-acquisition. Repeat the above steps until there are no missing areas in the registered 3D model, thus completing the integrity check of the registered 3D model.

9. The method for establishing a three-dimensional model of an immovable cultural relic based on point cloud as described in claim 1, characterized in that, The detailed processing of the resampled and supplemented registered 3D model to obtain the final 3D model includes the following steps: The initial state reconstruction of the registered 3D model is selectively performed according to the needs to restore the original form of the cultural relic before collection. The initial state reconstruction is performed by combining the texture, geometric features and historical documents in the collected data to reconstruct the historical state of the cultural relic. Texture mapping is performed on the registered 3D model. The texture mapping includes matching color image data with the 3D surface to enhance the texture details of the artifact surface. The enhancement includes high-resolution texture generation, color restoration, lighting adjustment, and noise removal. The registered 3D model is geometrically enhanced, which repairs and strengthens the surface details of the artifact through 3D reconstruction algorithms; its geometry is restored using surface smoothing or polygon reconstruction techniques. The details of the registered 3D model are restored, including restoring the texture, carvings or cracks on the surface of the artifact; The registered 3D model undergoes multi-scale detail processing, which involves optimizing the model's geometric shape and texture details layer by layer. The registered 3D model is smoothed and denoised. The smoothing and denoising are performed by removing noise and irregular geometric shapes using Laplacian smoothing technology.

10. A system for establishing a 3D model of immovable cultural relics based on point clouds, characterized in that, The system employs a point cloud-based method for establishing a 3D model of an immovable cultural relic, according to any one of claims 1 to 9, and includes the following modules: Data acquisition module: used to acquire the size data, color image and depth image data of cultural relics, and to construct the limitation range of the cultural relic model based on the size data; Construct an initial 3D model; simultaneously set alignment and registration points to prepare for subsequent point cloud data registration; Region segmentation module: This module is used to input the initial 3D model and color and depth image data into the complexity analysis model for calculation, obtain the processing region map and the acquisition region map, mark the complexity and generate the corresponding acquisition density parameters; based on the region complexity analysis results, it marks the regions that need to be resampled and sets the resampled registration points; Point cloud acquisition module: used to initially acquire point cloud data of cultural relics using a laser scanner or other 3D scanning equipment, and adjust the point cloud density according to the acquisition area map during the acquisition process; Point cloud data of the corresponding density is collected again for the area marked for recollection, and the recollected point cloud data is fitted with the initially collected point cloud data through registration points to obtain reconstructed point cloud data. Point cloud processing module: Used to process the reconstructed point cloud data using the processing area map, and to perform density filtering on the point cloud based on complexity; and to register the density-filtered point cloud data with the initial 3D model by aligning registration points to generate a registered 3D model; Model optimization module: Used to perform integrity detection on the registered 3D model and handle missing parts that do not meet the integrity requirements; it also performs detail repair on the 3D model after missing parts are handled, including reconstruction of the initial state of the cultural relic, texture mapping, geometric enhancement, surface detail repair and multi-scale detail processing; Model Display Module: Used to export the final 3D model as a standard 3D file format and visualize it, providing 3D data support for cultural relic protection, display or research.

Images