Method for processing cultural relic data by using three-dimensional scanner

Abstract

The invention relates to the technical field of cultural relic protection, in particular to a method for processing cultural relic data by using a three-dimensional scanner, and the method comprises the steps: obtaining scanning data and photogrammetric data of a cultural relic; acquiring environmental parameters of an environment where the cultural relics are located in real time, establishing a mapping relation between the environmental parameters and cultural relic deformation, and dynamically correcting scanning data; the environment parameters comprise temperature, humidity and vibration data; acquiring multispectral data of the cultural relic in real time, and establishing cultural relic material data matched with the surface structure of the cultural relic in combination with the scanning data; identifying the sensitivity of the cultural relic material to the laser wavelength, dynamically adjusting scanning parameters according to a sensitivity identification result, and correcting scanning data; and constructing a multi-modal data fusion framework for fusion of scanning data, photogrammetric data and multispectral data to realize digitization of cultural relics.

Description

Technical Field

[0001] The present invention relates to the technical field of cultural relics protection, and particularly relates to a method for processing cultural relics data using a 3D scanner. Background Art

[0002] Digitizing cultural relics is a new method of cultural relics protection. However, many cultural relics have irregular shapes and may have fine carvings or textures, which require high scanning resolution and accuracy. For example, the patterns on bronze wares or the cracks on pottery, and cultural relics of different materials have different adaptabilities to scanning technologies. For example, metals may reflect light and may deform under infrared rays, while ceramics or murals may have large surface color differences, resulting in noise or data loss during scanning and color changes under ultraviolet rays.

[0003] During the scanning process, due to changes in environmental factors (such as temperature and humidity), some cultural relics will deform. For example, wooden cultural relics are prone to moisture absorption and expansion, which will lead to inaccurate scanning data. The data generated by 3D scanning is very large, especially for high-precision models, and multiple types of data need to be integrated. Summary of the Invention

[0004] The present invention can be used for processing scanning data in a dynamic environment, combining multiple types of data to correct the scanning data of cultural relics. Considering that the laser wavelength will cause deformation of the cultural relics surface and may be superimposed with the influence of environmental factors, causing errors in the scanning data, a method of dynamically adjusting the wavelength and power of the scanning laser by identifying the wavelength sensitivity of the cultural relics is adopted.

[0005] The technical solution proposed by the present invention is: a method for processing cultural relics data using a 3D scanner, the method comprising: obtaining the scanning data and photogrammetry data of the cultural relics;

[0006] Obtaining the environmental parameters of the environment where the cultural relics are located in real time, establishing a mapping relationship between the environmental parameters and the deformation of the cultural relics, and dynamically correcting the scanning data; the environmental parameters include temperature, humidity and vibration data;

[0007] Obtaining the multi-spectral data of the cultural relics in real time, combining with the scanning data, and establishing the cultural relics material data that matches the surface structure of the cultural relics;

[0008] Identifying the sensitivity of the cultural relics material to the laser wavelength, and dynamically adjusting the scanning parameters according to the sensitivity identification result to correct the scanning data;

[0009] Constructing a multi-modal data fusion framework for the fusion of scanning data, photogrammetry data and multi-spectral data to realize the digitization of cultural relics.

[0010] Preferably, the obtaining the scanning data and photogrammetry data of the cultural relics includes:

[0011] Obtain the laser point cloud data of the cultural relics through a 3D scanner;

[0012] Obtain RGB images of the cultural relics from multiple angles through a camera device, and add angle recognition labels to the RGB images of each angle.

[0013] Preferably, the environmental parameters of the environment where the cultural relics are located are obtained in real time, the mapping relationship between the environmental parameters and the deformation of the cultural relics is established, and the scanned data is dynamically corrected, including:

[0014] Construct an environmental deformation model: ΔL = L 0 ·α·ΔT + L 0 ·β·ΔH; where, ΔL represents the cultural relic deformation compensation amount, L 0 represents the original size of the cultural relic, α represents the coefficient of thermal expansion, β represents the coefficient of humidity expansion, ΔH represents the humidity change amount, and ΔT represents the temperature change amount;

[0015] During scanning, by calculating the deformation compensation amount, the laser point cloud coordinates are corrected in reverse;

[0016] Construct a state equation, and use Kalman filtering or particle filtering, taking the environmental parameters as state variables, to dynamically predict and eliminate deformation noise;

[0017] The state equation is: x k = Fx k-1 + Bu k + w k , where, F and B respectively represent the state transition matrix and the input control matrix, x k represents the current state variable, that is, the deformation amount; x k-1 represents the state variable at the previous moment, u k represents the environmental parameter vector, that is, the temperature and humidity vector; w k represents the process noise.

[0018] Preferably, the environmental parameters of the environment where the cultural relics are located are obtained in real time, the mapping relationship between the environmental parameters and the deformation of the cultural relics is established, and the scanned data is dynamically corrected, further including:

[0019] Using incremental SLAM technology, combined with pose graph optimization to dynamically adjust local deformation;

[0020] Specifically, it includes the following steps:

[0021] Obtain scanned data, obtain 10 - 30 frames of laser point cloud data of the cultural relics per second, and form a laser point cloud data set;

[0022] At the same time, obtain temperature data and humidity data, and form a temperature and humidity vector set; the acquisition frequency of the temperature data and the humidity data is not less than 1Hz;

[0023] Obtain the pose of the 3D scanner;

[0024] Extract point cloud feature data and temperature-humidity feature vectors from the laser point cloud dataset and the temperature-humidity dataset respectively, and form a point cloud feature dataset and a temperature-humidity feature vector set respectively;

[0025] Construct a pose graph, specifically including:

[0026] Use the pose T′ corresponding to the laser point cloud data of each frame i As nodes, initialize the motion trajectory of the 3D scanner; use the relative change between nodes during two scans as edges;

[0027] Construct an environment-deformation constraint model, specifically including:

[0028] Introduce environmental parameters as deformation constraints, and calculate the local deformation matrix according to the changes in temperature and humidity: d = α·ΔT + β·ΔH; where, ΔT env Represents the expected deformation of the cultural relic caused by environmental parameters, I represents the identity matrix, and d represents the deformation translation vector;

[0029] Take ΔT env As a priori edges to connect adjacent nodes, minimize the Mahalanobis distance of all edges, and adjust the relationship of the edges by using the covariance of environmental parameters as the constraint of deformation;

[0030] Use the nonlinear least squares method to optimize the pose graph, and adjust the laser point cloud data obtained by scanning according to the optimized pose of the 3D scanner to compensate for deformation.

[0031] Preferably, the method for obtaining the multi-spectral data of the cultural relic in real time and combining the scanning data to establish the cultural relic material data matching the surface structure of the cultural relic includes:

[0032] According to the physical form, structure, and decorative features of the cultural relic, divide the surface or interior of the cultural relic into multiple different cultural relic regions;

[0033] Assign a unique number Num to each divided region;

[0034] Collect data for each cultural relic region at all set spectral bands, and obtain the reflectance vector of region i at m spectral bands where, r ij Represents the reflectance of the i-th region at the j-th spectral band;

[0035] Multiple reflectance vectors form a multi-spectral feature matrix R = (r ij ), j ≤ m

[0036] Obtain the characteristic data of a variety of known materials from the material characteristic database, and extract the reflectivity values under all set spectral bands therefrom to form a known material reflectivity vector Multiple known material reflectivity vectors form a known material characteristic matrix Y=(y lj );

[0037] Calculate the Euclidean distance between the known material characteristic matrix and the multispectral characteristic matrix. If the calculated Euclidean distance is less than the preset matching threshold, it is determined that the material in the corresponding area of the cultural relic is the corresponding known material;

[0038] Obtain the structural characteristics of the cultural relic area, specifically:

[0039] Obtain the RGB image of the cultural relic, divide the RGB image into multiple image areas, and associate the multiple image areas with the cultural relic area;

[0040] Through the image analysis algorithm, obtain the structural characteristics of the cultural relics in the cultural relic area, and all the cultural relic characteristics form a cultural relic structural characteristic matrix;

[0041] Match the cultural relic characteristic matrix with the known material characteristic matrix, so that the elements at different positions in each cultural relic characteristic matrix respectively correspond to the elements at the same positions in the known material characteristic matrix.

[0042] Preferably, the sensitivity of the identified cultural relic material to the laser wavelength includes:

[0043] Use the preset laser emission power, and select lasers of multiple wavelengths to pre-scan different parts of the cultural relic;

[0044] Obtain the microscopic deformation data and color change data of the corresponding scanned parts of the cultural relic after scanning with different wavelengths in real time, specifically including:

[0045] Record the reflection spectrum data one of multiple wavelength lasers in different bands and the temperature data one at the scanning position during the pre-scanning stage;

[0046] Record the reflection spectrum data two of multiple wavelength lasers in different bands and the temperature data two at the scanning position S seconds after the pre-scanning;

[0047] Calculate the change rate one of the reflection spectrum data two relative to the reflection spectrum data one and the change rate two between the temperature data two and the temperature data one;

[0048] If the change rate one is greater than the preset safety spectrum threshold or the change rate two is greater than the preset safety temperature threshold, it is determined that the cultural relic is sensitive to the laser of the corresponding wavelength;

[0049] Record all the wavelengths to which the cultural relic is sensitive to form a sensitive wavelength map.

[0050] Preferably, dynamically adjusting the scanning parameters according to the sensitivity recognition result includes:

[0051] Adjusting the wavelength of the laser used for scanning to use a laser with a wavelength outside the sensitive wavelength spectrum; or, constructing a laser wavelength adjustment function and fusing the laser wavelength adjustment function with the environmental deformation model to adjust the scanning parameters.

[0052] Preferably, constructing a laser wavelength adjustment function, fusing the laser wavelength adjustment function with the environmental deformation model, and adjusting the scanning parameters includes:

[0053] Obtaining the environmental deformation model ΔL = L 0 ·α·ΔT + L 0 ·β·ΔH;

[0054] Introducing the viscoelastic effect and constructing a time-dependent deformation model, where J(t) represents the creep compliance function, σ env represents the environmental stress, σ env = k T ΔT + k H ΔH, where k T 、k H represent the temperature stress coefficient and the humidity stress coefficient respectively, and η represents the material viscosity coefficient;

[0055] Constructing the laser wavelength adjustment function specifically includes the following steps:

[0056] Constructing the material optical response function R(λ, T, H) = R 0 (λ)·[1 + β T (λ)·ΔT + β H (λ)·ΔH]; where R 0 (λ) represents the reference reflectance spectrum, β T (λ) represents the temperature sensitivity coefficient, β H (λ) represents the humidity sensitivity coefficient;

[0057] Constructing a multi-objective optimization equation:

[0058] where P max (λ) represents the material damage threshold power, SNR(λ, P) represents the signal-to-noise ratio, SNR min represents the signal-to-noise ratio threshold; ΔL scan represents the amount of deformation after scanning;

[0059] Constructing the state equation includes:

[0060] Defining the state vector where R(λ) represents the real-time reflectance spectrum;

[0061] Then, the state equation is: where W represents the state noise;

[0062] The laser ranging equation is: where h(x xt ) represents the theoretical ranging function, c represents the speed of light, N(λ, T, H) represents the refractive index, L 0 represents the original size of the cultural relic, t TOF represents the time from the laser pulse emission to reception, and v represents the measurement noise;

[0063] Based on the extended Kalman filter, the deformation of the cultural relic is estimated in real time. According to the estimated deformation of the cultural relic, the target wavelength and power value that satisfy the multi-objective optimization equation are calculated. The 3D scanner controls the wavelength and power of the output laser according to the target wavelength and power value.

[0064] Preferably, the 3D scanner controls the wavelength and power of the output laser according to the target wavelength and power value, including:

[0065] Construct a drive matrix

[0066] where λ z represents the z-th wavelength, f rfz is the laser frequency corresponding to the z-th wavelength, and V z represents the drive voltage corresponding to the z-th wavelength;

[0067] Query from the drive matrix according to the target wavelength to obtain the corresponding laser frequency and drive voltage;

[0068] Fine-tune the wavelength through the linear interpolation algorithm;

[0069] Construct a reflectivity-power adjustment matrix: where R(λ z ) represents the reflectivity spectrum corresponding to the z-th wavelength; P zmax represents the maximum safe power corresponding to the z-th wavelength, and P zopt represents the target power corresponding to the z-th wavelength;

[0070] Query from the drive matrix according to the target wavelength to obtain the corresponding reflectivity spectrum, maximum safe power and target power; use a PID controller to dynamically adjust the power.

[0071] Preferably, the construction of the multi-modal data fusion framework for the fusion of scanning data, photogrammetry data and multi-spectral data to realize the digitization of cultural relics includes:

[0072] Construct a cross-modal data container, encapsulate the lidar point cloud, RGB image and hyperspectral data into a hierarchical structure, and retain the original data, where the original data includes device parameters and timestamps; specifically, it includes the following steps:

[0073] Through adaptive coding, convert data in different formats into an intermediate representation in a shared latent space;

[0074] Reduce the data volume through the lightweight compression algorithms Draco or LZ4, while maintaining the multi-modal correlation;

[0075] Extract cross-modal common semantic features through self-supervised learning and establish a feature mapping relationship, specifically including the following steps:

[0076] Use a trained two-tower neural network to process point cloud data and RGB image data respectively; align the feature vectors of the point cloud data and the RGB image through contrastive learning;

[0077] Introduce an attention mechanism to dynamically allocate weights for multi-modal features, where the multi-modal features include the feature vectors of point cloud data, RGB image feature vectors and hyperspectral data feature vectors.

[0078] Advantages of the present invention:

[0079] In the present invention, the influence of the dynamic environment on the deformation of cultural relics is considered. By establishing a mapping relationship between environmental parameters and the deformation of cultural relics, the scanned data is dynamically corrected; the surface material of the cultural relic is inferred using hyperspectral data, and the surface material is corresponded to the surface structure of the cultural relic one by one. Combining the material and structure of the cultural relic, after identifying the sensitivity of the cultural relic to the laser wavelength, the wavelength and frequency of the laser used for scanning are dynamically adjusted to correct the scanned data, avoiding the deformation of the cultural relic caused by the wavelength of a specific laser and superimposing the influence of environmental parameters. Description of the Drawings

[0080] Figure 1 It is a flowchart of a method for processing cultural relic data using a 3D scanner according to the present invention. Detailed Embodiments

[0081] The following description is used to disclose the present invention so that those skilled in the art can implement the present invention. The preferred embodiments described below are only examples, and those skilled in the art can think of other obvious deformations. The basic principles defined in the following description can be applied to other implementation schemes, deformation schemes, improvement schemes, equivalent schemes, and other technical schemes that do not deviate from the spirit and scope of the present invention.

[0082] It is understood that the term "a" should be construed as "at least one" or "one or more". That is, in one embodiment, the number of an element may be one, while in other embodiments, the number of the element may be multiple. The term "a" should not be construed as a limitation on the number.

[0083] Embodiment 1:

[0084] Reference Figure 1 , the technical solution provided by the present invention is: a method for processing cultural relic data using a three-dimensional scanner, comprising the following steps:

[0085] Step 1, obtaining the scanning data and photogrammetry data of the cultural relic; including: obtaining the laser point cloud data of the cultural relic through a three-dimensional scanner, and obtaining RGB images of the cultural relic from multiple angles through a camera device; and adding angle recognition labels to the RGB images of each angle.

[0086] Step 2, obtaining the environmental parameters of the environment where the cultural relic is located in real time, establishing a mapping relationship between the environmental parameters and the deformation of the cultural relic, and dynamically correcting the scanning data; the environmental parameters include temperature, humidity and vibration data; specifically including the following steps:

[0087] 2.1, constructing an environmental deformation model: ΔL = L 0 ·α·ΔT + L 0 ·β·ΔH; where, ΔL represents the cultural relic deformation compensation amount, L 0 represents the original size of the cultural relic, α represents the coefficient of thermal expansion, β represents the coefficient of humidity expansion, ΔH represents the humidity change amount, and ΔT represents the temperature change amount;

[0088] 2.2, during scanning, by calculating the deformation compensation amount, reversely correcting the laser point cloud coordinates;

[0089] 2.3, constructing a state equation, using Kalman filtering or particle filtering, taking the environmental parameters as state variables, and dynamically predicting and eliminating deformation noise;

[0090] The state equation is: x k = Fx k-1 + Bu k + w k , where, F and B respectively represent the state transition matrix and the input control matrix, x k represents the current state variable, that is, the amount of deformation; x k-1 represents the state variable at the previous moment, u k represents the environmental parameter vector, that is, the temperature and humidity vector; w k represents the process noise.

[0091] Step 3: Obtain the multispectral data of the cultural relics in real time, and combine with the scanning data to establish the cultural relic material data that matches the surface structure of the cultural relics. The steps include:

[0092] Divide the surface or interior of the cultural relic into multiple different cultural relic areas according to the physical form, structure, and decorative features of the cultural relic.

[0093] Assign a unique number Num to each divided area.

[0094] Collect data for each cultural relic area under all set spectral bands to obtain the reflectance vector of area i under m spectral bands where r ij represents the reflectance of the i-th area under the j-th spectral band;

[0095] Multiple reflectance vectors form the multispectral feature matrix R=(r ij ), j≤m

[0096] Obtain the characteristic data of multiple known materials from the material property database, and extract the reflectance values under all set spectral bands to form the known material reflectance vector Multiple known material reflectance vectors form the known material feature matrix Y=(y lj );

[0097] Calculate the Euclidean distance between the known material feature matrix and the multispectral feature matrix. If the calculated Euclidean distance is less than the preset matching threshold, it is determined that the material of the corresponding area of the cultural relic is the corresponding known material.

[0098] Obtain the structural characteristics of the cultural relic area, specifically:

[0099] Obtain the RGB image of the cultural relic, divide the RGB image into multiple image areas, and associate the multiple image areas with the cultural relic areas.

[0100] Through the image analysis algorithm, obtain the structural characteristics of the cultural relic in the cultural relic area, and all the cultural relic characteristics form the cultural relic structure feature matrix.

[0101] Match the cultural relic feature matrix with the known material feature matrix, so that the elements at different positions in each cultural relic feature matrix respectively correspond to the elements at the same positions in the known material feature matrix.

[0102] Step 4: Identify the sensitivity of the cultural relic material to the laser wavelength, and dynamically adjust the scanning parameters according to the sensitivity identification result to correct the scanning data.

[0103] Among them, identifying the sensitivity of the cultural relic material to the laser wavelength includes the following steps:

[0104] Using a preset laser emission power, select lasers of multiple wavelengths to pre-scan different parts of the cultural relic;

[0105] Obtain in real time the microscopic deformation data and color change data of the corresponding scanned parts of the cultural relic after being scanned with different wavelengths, specifically including:

[0106] Record the reflection spectrum data one of multiple wavelengths of lasers in different bands and the temperature data one at the scanning position during the pre-scanning stage;

[0107] Record the reflection spectrum data two of multiple wavelengths of lasers in different bands and the temperature data two at the scanning position S seconds after the pre-scanning;

[0108] Calculate the change rate one of the reflection spectrum data two relative to the reflection spectrum data one and the change rate two between the temperature data two and the temperature data one;

[0109] If the change rate one is greater than the preset safe spectrum threshold or the change rate two is greater than the preset safe temperature threshold, it is determined that the cultural relic is sensitive to the laser of the corresponding wavelength;

[0110] Record all the wavelengths to which the cultural relic is sensitive to form a sensitive wavelength map.

[0111] Among them, according to the sensitivity recognition result, dynamically adjust the scanning parameters, including the following steps:

[0112] Adjust the wavelength of the laser used for scanning, and use a laser with a wavelength outside the sensitive wavelength map; or, construct a laser wavelength adjustment function, and fuse the laser wavelength adjustment function with the environmental deformation model to adjust the scanning parameters.

[0113] Among them, constructing a laser wavelength adjustment function and fusing the laser wavelength adjustment function with the environmental deformation model to adjust the scanning parameters includes the following steps:

[0114] 1. Obtain the environmental deformation model ΔL = L 0 ·α·ΔT + L 0 ·β·ΔH;

[0115] Introduce the viscoelastic effect and construct a time-dependent deformation model, where J(t) represents the creep compliance function, and σ env represents the environmental stress, and σ env = k T ΔT + k H ΔH, where k T and k H represent the temperature stress coefficient and the humidity stress coefficient respectively, and η represents the material viscosity coefficient;

[0116] 2. Determine the optical properties of the material and construct a relationship function between wavelength and reflectivity, specifically:

[0117] Construct the optical response function of the material \(R(\lambda, T, H)=R\) 0 (\(\lambda)\cdot[1 + \beta\) T (\(\lambda)\cdot\Delta T+\beta\) H (\(\lambda)\cdot\Delta H]\); where \(R\) 0 (\(\lambda)\) represents the reference reflectance spectrum, \(\beta\) T (\(\lambda)\) represents the temperature sensitivity coefficient, \(\beta\) H (\(\lambda)\) represents the humidity sensitivity coefficient;

[0118] 3. Construct the wavelength adjustment function, and select the optimal power and wavelength according to the environmental parameters and material response, including:

[0119] Construct the wavelength adjustment function, which specifically includes the following steps:

[0120] Construct the multi-objective optimization equation:

[0121] where \(P\) max (\(\lambda)\) represents the material damage threshold power, \(SNR(\lambda, P)\) represents the signal-to-noise ratio, \(SNR\) min represents the signal-to-noise ratio threshold; \(\Delta L\) scan represents the deformation amount after scanning;

[0122] 4. Combine the environmental deformation model with the wavelength adjustment function to form a comprehensive correction algorithm, so that it can use the PID controller in control theory; specifically:

[0123] Construct the state equation, including:

[0124] Define the state vector where \(R(\lambda)\) represents the real-time reflectance spectrum;

[0125] Then, the state equation is: where \(W\) represents the state noise;

[0126] The laser ranging equation is: where \(h(x\) xt ) represents the theoretical ranging function, \(c\) represents the speed of light, \(N(\lambda, T, H)\) represents the refractive index, \(L\) 0 represents the original size of the cultural relic, \(t\) TOF represents the time from laser pulse emission to reception, and \(v\) represents the measurement noise;

[0127] Based on the extended Kalman filter, the deformation amount of the cultural relic is estimated in real time. According to the estimated deformation amount of the cultural relic, the target wavelength and power values that satisfy the multi-objective optimization equation are calculated. The 3D scanner controls the wavelength and power of the output laser according to the target wavelength and power values.

[0128] For example, a standard material, the INVAR alloy, can be used to verify the accuracy of the environmental deformation model in a temperature control box, and the target accuracy can be set to 0.5 μm / °C; by gradually increasing the power value, a reflectivity change greater than 1% is detected to determine the P max (λ) curve.

[0129] In this embodiment, the 3D scanner controls the wavelength and power of the output laser according to the target wavelength and power value, which specifically includes the following steps:

[0130] Construct a driving matrix

[0131] where λ z represents the z-th wavelength, f rfz is the laser frequency corresponding to the z-th wavelength, and V z represents the driving voltage corresponding to the z-th wavelength;

[0132] Query from the driving matrix according to the target wavelength to obtain the corresponding laser frequency and driving voltage;

[0133] Fine-tune the wavelength through a linear interpolation algorithm;

[0134] Construct a reflectivity-power adjustment matrix: where R(λ z ) represents the reflectivity spectrum corresponding to the z-th wavelength; P zmax represents the maximum safe power corresponding to the z-th wavelength, and P zopt represents the target power corresponding to the z-th wavelength;

[0135] Query from the driving matrix according to the target wavelength to obtain the corresponding reflectivity spectrum, maximum safe power, and target power; use a PID controller to dynamically adjust the power.

[0136] Step 5: Construct a multi-modal data fusion framework for fusing scanning data, photogrammetry data, and multi-spectral data to digitize cultural relics. Specifically, it includes the following steps:

[0137] The construction of the multi-modal data fusion framework for fusing scanning data, photogrammetry data, and multi-spectral data to digitize cultural relics includes:

[0138] Construct a cross-modal data container to encapsulate the laser point cloud, RGB image, and multi-spectral data into a hierarchical structure and retain the original data, where the original data includes device parameters and timestamps; specifically, it includes the following steps:

[0139] Convert data in different formats into an intermediate representation in a shared latent space through adaptive encoding;

[0140] Reduce the data volume through the lightweight compression algorithms Draco or LZ4 while maintaining the multi-modal relevance;

[0141] Extract the semantic features common to cross-modalities through self-supervised learning and establish a feature mapping relationship, which specifically includes the following steps:

[0142] Use the trained two-tower neural network to process the point cloud data and RGB image data respectively; align the feature vectors of the point cloud data and the RGB image through contrastive learning;

[0143] Introduce an attention mechanism to dynamically allocate weights for multi-modal features, where the multi-modal features include the feature vectors of point cloud data, RGB image feature vectors, and multi-spectral data feature vectors.

[0144] Example 2:

[0145] The difference between this example and Example 1 is:

[0146] Apply the SLAM technology to the 3D scanning of cultural relics. The traditional SLAM technology is used when a robot moves in an unknown environment, simultaneously constructing a map and positioning itself. In the scanning of cultural relics, although the scanner itself cannot move, due to environmental changes, the shape of the cultural relic itself changes, which is equivalent to the environment changing dynamically. Therefore, the scanner can be regarded as a mobile sensor, and the deformation of the cultural relic surface is equivalent to the environmental change;

[0147] The specific steps are as follows:

[0148] Obtain scanning data, and obtain 10 - 30 frames of laser point cloud data of cultural relics per second to form a laser point cloud data set;

[0149] Simultaneously obtain temperature data and humidity data to form a temperature-humidity vector set; the acquisition frequency of the temperature data and humidity data is not less than 1Hz;

[0150] Obtain the pose of the 3D scanner;

[0151] Extract point cloud feature data and temperature-humidity feature vectors from the laser point cloud data set and the temperature-humidity data set respectively to form a point cloud feature data set and a temperature-humidity feature vector set;

[0152] Construct a pose graph, which specifically includes:

[0153] Use the pose T i ′ corresponding to each frame of laser point cloud data as a node to initialize the motion trajectory of the 3D scanner; use the relative change between the nodes during two scans as an edge;

[0154] Construct an environment-deformation constraint model, which specifically includes:

[0155] Introduce environmental parameters as deformation constraints, and calculate the local deformation matrix according to the changes in temperature and humidity: d = α·ΔT + β·ΔH; where, ΔT env represents the expected deformation of the cultural relic caused by environmental parameters, I represents the identity matrix, and d represents the deformation translation vector;

[0156] Take ΔT env as a priori edges to connect adjacent nodes, minimize the Mahalanobis distance of all edges, and adjust the edge relationship by using the covariance of environmental parameters as the deformation constraint;

[0157] Use the nonlinear least squares method to optimize the pose graph, and adjust the laser point cloud data obtained by scanning according to the optimized pose of the 3D scanner to compensate for deformation.

[0158] In this embodiment, the problem of pose graph optimization is defined as the problem of minimizing the Mahalanobis distance of all edges ;

[0159] Obtain the optimized pose {T′ i *}, where ∑ env represents the covariance of environmental parameters, that is, the covariance of temperature and humidity data; through ∑ env reflects the confidence of environmental parameters, and ∑ ij represents the pose covariance;

[0160] Transform the laser point cloud data obtained by scanning through the optimized pose. Specifically:

[0161] Transform the original point cloud according to the optimized pose {T′ i *}, where, is the original point cloud vector. In addition, for high-deformation regions, non-rigid ICP can be applied for further refinement.

[0162] For example, when scanning wooden cultural relics, the environmental changes are temperature fluctuations of ±5°C and humidity fluctuations of ±15%RH;

[0163] Operation process: Add deformation constraint edges every 5 seconds (α T = 5×10 -5 / °C), and the error of the optimized model is reduced from 2.1 mm to 0.3 mm. The verification method can verify the position accuracy of key points through a laser tracker.

[0164] Embodiments disclosed in the present invention, the processes described above with reference to the flowcharts can be implemented as computer software programs. Embodiments disclosed in the present invention include a computer program product, which includes a computer program carried on a computer-readable medium, and the computer program contains program codes for performing the methods shown in the flowcharts. In such an embodiment, the computer program can be downloaded and installed from the network through the communication part, and / or installed from a removable medium. When the computer program is executed by a central processing unit (CPU), the above-mentioned functions defined in the methods of the present invention are performed. It should be noted that the computer-readable medium in the present invention can be a computer-readable signal medium, a computer-readable storage medium, or any combination of the two. A computer-readable storage medium can, for example, but not be limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination of the above. More specific examples of the computer-readable storage medium can include, but not be limited to: an electrical connection having one or more wire segments, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In the present invention, the computer-readable storage medium can be any tangible medium that contains or stores a program, and the program can be used by or combined with an instruction execution system, apparatus, or device. In the present invention, a computer-readable signal medium can include a data signal propagated in a baseband or as part of a carrier wave, in which computer-readable program codes are carried. Such a propagated data signal can take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. The computer-readable signal medium can also be any computer-readable medium other than the computer-readable storage medium, and the computer-readable medium can send, propagate, or transmit a program for use by or combined with an instruction execution system, apparatus, or device. The program codes contained on the computer-readable medium can be transmitted by any suitable medium, including but not limited to: a wireless segment, a wire segment, an optical cable, RF, etc., or any suitable combination of the above.

[0165] The flowcharts in the accompanying drawings illustrate the architecture, functions, and operations that the methods according to various embodiments of the present invention may implement. In this regard, each block in the flowchart or block diagram may represent a module, a segment of a program, or a part of code that contains one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions marked in the blocks may occur in a different order from that marked in the accompanying drawings. For example, two consecutive blocks shown may actually be executed substantially in parallel, and they may sometimes be executed in the reverse order, depending on the functions involved. It should also be noted that each block in the block diagram and / or flowchart, and combinations of blocks in the block diagram and / or flowchart, may be implemented by a dedicated hardware-based system that performs the specified functions or operations, or may be implemented by a combination of dedicated hardware and computer instructions.

[0166] Those skilled in the art should understand that the embodiments of the present invention described above and shown in the accompanying drawings are only examples and do not limit the present invention. The objectives of the present invention have been fully and effectively achieved. The functions and structural principles of the present invention have been demonstrated and illustrated in the embodiments. Without departing from the said principles, any changes or modifications can be made to the embodiments of the present invention.

Claims

1. A method for processing cultural relics data using a three-dimensional scanner, characterized in that: The method comprises: Obtain scanning data and photogrammetric data of cultural relics; Real-time acquisition of environmental parameters of the environment where the cultural relics are located, establishment of a mapping relationship between environmental parameters and cultural relics deformation, and dynamic correction of scanning data; the environmental parameters include temperature, humidity and vibration data; Acquire multispectral data of cultural relics in real time, combine it with scanning data, and establish material data of cultural relics that matches the surface structure of cultural relics; Identify the sensitivity of the cultural relic material to the laser wavelength, dynamically adjust the scanning parameters based on the sensitivity identification results, and correct the scanning data; Construct a multimodal data fusion framework for the fusion of scanning data, photogrammetry data, and multispectral data to achieve the digitization of cultural relics.

2. The method for processing cultural relics data using a three-dimensional scanner according to claim 1, characterized in that: The acquisition of scanning data and photogrammetric data of cultural relics includes: Obtain laser point cloud data of cultural relics through 3D scanner; The RGB images of the cultural relics at multiple angles are acquired through the camera equipment, and angle identification labels are added to the RGB images of each angle.

3. The method for processing cultural relics data using a three-dimensional scanner according to claim 2, characterized in that: The real-time acquisition of environmental parameters of the environment where the cultural relics are located, the establishment of a mapping relationship between the environmental parameters and the deformation of the cultural relics, and the dynamic correction of the scanning data include: Construct the environmental deformation model: ΔL = L0·α·ΔT+L0·β·ΔH; where ΔL represents the deformation compensation of the cultural relic, L0 represents the original size of the cultural relic, α represents the thermal expansion coefficient, β represents the humidity expansion coefficient, ΔH represents the humidity change, and ΔT represents the temperature change; During scanning, the laser point cloud coordinates are reversely corrected by calculating the deformation compensation amount; Construct state equations, use Kalman filtering or particle filtering, take environmental parameters as state variables, dynamically predict and eliminate deformation noise; The state equation is: k =Fx k-1 +Bu k +w k , where F and B represent the state transfer matrix and input control matrix respectively, x k represents the current state variable, i.e., the shape variable; x k-1 Represents the state variable at the previous moment, u k represents the environmental parameter vector, namely the temperature and humidity vector; w k represents process noise.

4. The method for processing cultural relics data using a three-dimensional scanner according to claim 3, characterized in that: The real-time acquisition of environmental parameters of the environment where the cultural relics are located, the establishment of a mapping relationship between the environmental parameters and the deformation of the cultural relics, and the dynamic correction of the scanning data also include: Using incremental SLAM technology combined with pose graph optimization to dynamically adjust local deformation; The specific steps include: Obtain scanning data, obtain 10-30 frames of laser point cloud data of cultural relics per second, and form a laser point cloud data set; Acquire temperature data and humidity data simultaneously to form a temperature and humidity vector set; the acquisition frequency of the temperature data and humidity data is not less than 1 Hz; Get the position and posture of the 3D scanner; Extracting point cloud feature data and temperature and humidity feature vectors from the laser point cloud dataset and the temperature and humidity dataset respectively, forming a point cloud feature dataset and a temperature and humidity feature vector set respectively; Construct a pose graph, including: The pose T corresponding to each frame of laser point cloud data i ' is used as a node to initialize the motion trajectory of the 3D scanner; the relative change of the node between two scans is used as an edge; Construct an environment-deformation constraint model, including: Environmental parameters are introduced as deformation constraints, and the local deformation matrix is ​​calculated according to the changes in temperature and humidity: Where, ΔT env represents the expected deformation of the cultural relic caused by environmental parameters, I represents the unit matrix, and d represents the deformation translation vector; ΔT env Connect adjacent nodes as a priori edges, minimize the Mahalanobis distance of all edges, and use the covariance of environmental parameters as a deformation constraint to adjust the relationship between edges; The pose graph is optimized using the nonlinear least squares method, and the laser point cloud data obtained by scanning is adjusted according to the optimized pose of the 3D scanner to compensate for the deformation.

5. The method for processing cultural relics data using a three-dimensional scanner according to claim 4, characterized in that: The real-time acquisition of multispectral data of cultural relics, combined with scanning data, to establish cultural relic material data matching the surface structure of the cultural relic, includes: According to the physical form, structure and decorative features of the cultural relics, the surface or interior of the cultural relics is divided into multiple different cultural relics areas; Assign a unique number Num to each divided area; Data is collected for each cultural relic area in all set spectral bands to obtain the reflectivity vector of area i in m spectral bands. Among them, r ij represents the reflectance of the i-th region in the j-th spectral band; Multiple reflectivity vectors constitute a multispectral feature matrix R = (r ij ),j≤m Obtain the characteristic data of various known materials from the material characteristic database, extract the reflectance values ​​in all set spectral bands, and construct the reflectance vector of known materials A plurality of known material reflectivity vectors constitute a known material characteristic matrix Y = (y lj ); Calculate the Euclidean distance between the known material feature matrix and the multi-spectral feature matrix. If the calculated Euclidean distance is less than a preset matching threshold, determine that the material in the corresponding area of ​​the cultural relic is the corresponding known material. Get the structural characteristics of the cultural relics area, specifically: Acquire an RGB image of the cultural relic, divide the RGB image into a plurality of image regions, and associate the plurality of image regions with the cultural relic region; The structural features of the cultural relics in the cultural relics area are obtained through image analysis algorithms, and all the cultural relics features constitute a cultural relics structural feature matrix; The cultural relic feature matrix is ​​matched with the known material feature matrix so that the elements at different positions in each cultural relic feature matrix correspond to the elements at the same position in the known material feature matrix.

6. The method for processing cultural relics data using a three-dimensional scanner according to claim 5, characterized in that: The sensitivity of the identified cultural relic material to the laser wavelength includes: Use the preset laser emission power and select lasers of multiple wavelengths to pre-scan different parts of the cultural relics; Real-time acquisition of microscopic deformation data and color change data of the corresponding scanned parts of the cultural relics after scanning at different wavelengths, including: Recording reflection spectrum data 1 of multiple wavelength lasers in different bands in the pre-scanning stage and temperature data 1 at the scanning position; Recording reflection spectrum data 2 of multiple wavelength lasers in different bands after pre-scanning for S seconds and temperature data 2 at the scanning position; Calculate a change rate one of the reflectance spectrum data two relative to the reflectance spectrum data one and a change rate two of the temperature data two relative to the temperature data one; If the change rate 1 is greater than the preset safety spectrum threshold or the change rate 2 is greater than the preset safety temperature threshold, it is determined that the cultural relic is sensitive to the laser of the corresponding wavelength; Record all wavelengths that the cultural relics are sensitive to and form a sensitive wavelength spectrum.

7. The method for processing cultural relics data using a three-dimensional scanner according to claim 6, characterized in that: The step of dynamically adjusting scanning parameters according to the sensitivity identification result includes: Adjust the wavelength of the laser used for scanning and use a laser with a wavelength outside the sensitive wavelength spectrum; or, construct a laser wavelength adjustment function and merge the laser wavelength adjustment function with the environmental deformation model to adjust the scanning parameters.

8. The method for processing cultural relics data using a three-dimensional scanner according to claim 7, characterized in that: The laser wavelength adjustment function is constructed, and the laser wavelength adjustment function is integrated with the environmental deformation model to adjust the scanning parameters, including: Obtain the environmental deformation model ΔL=L0·α·ΔT+L0·β·ΔH; Introducing viscoelastic effects and constructing a time-dependent deformation model, Where J(t) represents the creep compliance function, σ env represents environmental stress, σ env =k T ΔT+k H ΔH, where k T , k H They represent the temperature stress coefficient and the humidity stress coefficient respectively, and η represents the material viscosity coefficient; Constructing a laser wavelength adjustment function specifically includes the following steps: Construct the material optical response function R(λ,T,H)=R0(λ)·[1+β T (λ)·ΔT+β H (λ)·ΔH]; where R0(λ) represents the reference reflectance spectrum, β T (λ) represents the temperature sensitivity coefficient, β H (λ) represents the humidity sensitivity coefficient; Construct the multi-objective optimization equation: Among them, P max (λ) represents the material damage threshold power, SNR(λ,P) represents the signal-to-noise ratio, SNR min represents the signal-to-noise ratio threshold; ΔL scan represents the deformation after scanning; Construct the equation of state, including: Define the state vector Where R(λ) represents the real-time reflectance spectrum; Then, the state equation is: Where W represents state noise; The laser ranging equation is: Among them, h(x xt ) represents the theoretical distance function, c represents the speed of light, N(λ,T,H) represents the refractive index, L0 represents the original size of the artifact, t TOF represents the time from the emission of the laser pulse to the reception, and v represents the measurement noise; The deformation of the cultural relic is estimated in real time based on the extended Kalman filter. According to the estimated deformation of the cultural relic, the target wavelength and power value that meet the multi-objective optimization equation are calculated. The three-dimensional scanner controls the wavelength and power of the output laser according to the target wavelength and power value.

9. The method for processing cultural relics data using a three-dimensional scanner according to claim 8, characterized in that: The three-dimensional scanner controls the wavelength and power of the output laser according to the target wavelength and power value, including: Building the driver matrix Among them, λ z represents the zth wavelength, f rfz The laser frequency corresponding to the zth wavelength, V z represents the driving voltage corresponding to the zth wavelength; According to the target wavelength, the corresponding laser frequency and driving voltage are obtained by querying the driving matrix; Fine-tune the wavelength through a linear interpolation algorithm; Construct the reflectivity-power adjustment matrix: Among them, R(λ z ) represents the reflectance spectrum corresponding to the zth wavelength; P zmax represents the maximum safe power corresponding to the zth wavelength, P zopt represents the target power corresponding to the zth wavelength; According to the target wavelength, the corresponding reflectivity spectrum, maximum safe power and target power are obtained from the driving matrix; the power is dynamically adjusted using a PID controller.

10. The method for processing cultural relics data using a three-dimensional scanner according to claim 9, characterized in that: The multimodal data fusion framework is constructed to fuse scanning data, photogrammetry data and multispectral data to realize the digitization of cultural relics, including: Construct a cross-modal data container, encapsulate the laser point cloud, RGB image and multispectral data into a hierarchical structure, and retain the original data, which includes device parameters and timestamps; specifically, the following steps are included: Through adaptive encoding, data in different formats are converted into intermediate representations in a shared latent space; Use lightweight compression algorithms such as Draco or LZ4 to reduce data size while maintaining multi-modal relevance; Through self-supervised learning, semantic features shared across modalities are extracted and feature mapping relationships are established. Specifically, the following steps are included: Use the trained dual-tower neural network to process point cloud data and RGB image data respectively; align the feature vectors of point cloud data and RGB image through comparative learning; An attention mechanism is introduced to dynamically assign weights of multimodal features, where the multimodal features include feature vectors of point cloud data, feature vectors of RGB images, and feature vectors of multispectral data.

Images