Automatic high-precision extraction method for cultural relic diseases based on three-dimensional fine data

Abstract

The application discloses a kind of based on three-dimensional fine data's cultural relic disease automatic high-precision extraction method, comprising: obtaining the orthographic image of cultural relic with projection parameter;According to the gray level co-occurrence matrix of image complexity calculation, the adaptive K value of orthographic image is carried out SLIC image segmentation clustering calculation, and the adaptive superpixel number K of orthographic image is calculated, and the superpixel segmentation is carried out to image according to the K value obtained by calculation;K-means clustering segmentation is carried out to superpixel segmentation result, and the disease pixel coordinates of cultural relic are extracted;The three-dimensional coordinate information of the disease pixel coordinates of cultural relic is converted, then three-dimensional reconstruction is carried out, and the three-dimensional geometric information of disease is obtained.The application combines image segmentation algorithm, realizes disease automatic extraction, simultaneously applies two three-dimensional data conversion, and automatically obtains three-dimensional information of disease, not only greatly saves the time of manual point selection, and simultaneously improves the accuracy of disease extraction.

Description

Technical Field

[0001] This invention belongs to the field of digital preservation of cultural heritage and relates to a method for automatic high-precision extraction of cultural relic defects based on three-dimensional fine data. Background Technology

[0002] Surface damage detection of cultural relics is a crucial aspect of cultural relic conservation. Traditional surveys of cultural relic damage have evolved from initial textual and image records using simple tools to manual and automated surveys based on measurable orthophotos of the relics. While image segmentation for damage extraction based on orthophotos is relatively mature, the results remain two-dimensional. For surfaces with significant curvature, the extraction accuracy is significantly reduced. Furthermore, for complex objects like painted sculptures, the standard six orthophoto projections (front, back, left, right, top, and bottom) cannot reveal all damage, impacting both the quantity and accuracy of the damage survey. With the increasing sophistication of digital photogrammetry, 3D laser scanning, and 3D modeling technologies, the refined 3D digitization of cultural relics provides the most comprehensive and accurate data for surface damage surveys, inevitably shifting the extraction of damage from two-dimensional to three-dimensional methods. Currently, point cloud data can be used to extract defects or cracks, but this only works for damage with significant geometric differences or deformations; damage manifested solely by color variations remains unextracted. Some scholars have also implemented manual selection of diseased areas, definition of disease types, and statistical analysis of disease geometry on 3D models. However, this method merely converts 2D manual vectorization into 3D manual vectorization, resulting in low efficiency and accuracy affected by human factors. Currently, there is no method for high-precision automatic extraction of cultural relic diseases directly from a detailed 3D color model. If the aforementioned 3D data could be used directly for automatic disease surveys, the process of creating 2D orthophotos could be eliminated, and the most realistic and comprehensive disease information could be automatically extracted from the 3D model, providing accurate and comprehensive data support for cultural relic restoration and conservation. Therefore, automatic 3D surveys of cultural relic diseases have significant practical significance and application value. Summary of the Invention

[0003] One object of the present invention is to solve at least the above-mentioned problems and / or defects, and to provide at least the advantages described below.

[0004] Another objective of this invention is to provide an automatic, high-precision method for extracting cultural relic defects based on three-dimensional fine data.

[0005] Therefore, the technical solution provided by this invention is as follows:

[0006] A method for automatic high-precision extraction of cultural relic defects based on three-dimensional fine data includes the following steps:

[0007] Step 1: Obtain an orthophoto of the cultural relic with attached projection parameters;

[0008] Step 2: Calculate the gray-level co-occurrence matrix based on image complexity and perform adaptive K-value SLIC image segmentation and clustering on the orthophoto, including:

[0009] 2.1) Calculate the number of adaptive superpixels K of the orthophoto according to equation (13), where K represents the number of pre-segments for superpixel segmentation performed by SLIC.

[0010]

[0011] Where W represents the width of the image, H represents the height of the image, and T represents the height of the image. k This indicates the complexity of the image itself. Indicates rounding up;

[0012] 2.2) SLIC segmentation based on the calculated K value: First, seed points are evenly distributed in the image according to the number of adaptive superpixels; then, cluster centers are reselected in the n-neighborhood of the cluster centers, where n is a positive integer. The gray-level gradient of all pixels in each neighborhood is calculated, and the gradient values ​​of pixels in each neighborhood are traversed. Finally, the pixel with the smallest value is taken as the adjusted cluster center; then, the adjusted cluster center is used as the search center, and a certain number of center distances are used as the neighborhood search range to construct a center identifier for each original pixel in the segmented image to determine the possible cluster center to which each pixel may belong. For each searched pixel, the distance between the pixel and the seed point to which it was searched is calculated, and the seed point corresponding to the minimum value is taken as the cluster center of the pixel. Superpixel segmentation is then performed on the image.

[0013] 2.3) Perform K-means clustering on the superpixel segmentation results to extract the coordinates of the damaged pixels of the cultural relics;

[0014] Step 3: Convert the pixel coordinates of the damage to three-dimensional planar coordinates to obtain the three-dimensional coordinate information of the damage. Then, perform three-dimensional reconstruction on the three-dimensional coordinates of the damage to obtain the three-dimensional geometric information of the damage.

[0015] Preferably, in the automatic high-precision extraction method for cultural relic defects based on three-dimensional fine data, step 2.1), the step of obtaining the calculation formula for the number of adaptive superpixels K, includes:

[0016] First, the number of adaptive superpixel segments K is calculated using the image complexity described by the gray-level co-occurrence matrix. Then, considering the characteristics of the image data itself, information entropy is introduced, which is the probability of a specific piece of information appearing. The higher the entropy value, the more complex the image, and vice versa. Information entropy (Ent, denoted by E) Ent The calculation formula for (represented) is as follows:

[0017]

[0018] In the formula, k represents the number of distinct flag values ​​within the generalized set, ni represents the number of pixels with gray value i, and N represents the total number of pixels in the entire image. The parameters of the gray-level co-occurrence matrix that have typical metric characteristics include: energy (E), contrast (Con, denoted by Ccon), and correlation (Cor, denoted by Ccor). The image complexity Tk is calculated using the following formula:

[0019] T k =E Ent +C Con -EC Cor (6)

[0020] in,

[0021]

[0022]

[0023] In the formula: Q 2 (i, j, d, θ) are obtained through gray-level co-occurrence matrix normalization; m represents the number of pixels in the image along the x-axis in the plane; n represents the number of pixels in the image along the y-axis in the plane; θ represents the generation direction of the gray-level co-occurrence matrix, which can be 0°, 45°, 90°, or 135°; where:

[0024]

[0025]

[0026] u1 and u2 are the mean, d1 and d2 represent the variance, and p(i,j,d,θ) represents the pixel information represented by the element in the i-th row and j-th column of the gray-level co-occurrence matrix.

[0027] The formula for calculating the number K of the adaptive superpixels was established:

[0028]

[0029] Preferably, in the automatic high-precision extraction method for cultural relic defects based on three-dimensional fine data, in step 2.2), the distance metric D' between the pixel and its surrounding seed points is as follows:

[0030]

[0031] Where, d c Represents color distance, d sLet m represent the spatial distance, m be the compactness, and the size of each superpixel be N / K, where N is the number of pixels in the image. Then the adjacent seed step size is approximately s = sqrt(N / K).

[0032] Preferably, in the aforementioned automatic high-precision extraction method for cultural relic defects based on three-dimensional fine data,

[0033]

[0034]

[0035]

[0036] Where, N s It is the maximum spatial distance within the class, defined as N. s =S, applicable to each cluster, calculated as follows:

[0037]

[0038] Preferably, in the aforementioned automatic high-precision extraction method for cultural relic defects based on three-dimensional fine data, step one, the method for obtaining the orthophoto image of the cultural relic with attached projection parameters, includes:

[0039] Model transformation (M) converts the three-dimensional color model of the cultural relic from object coordinates to world coordinates;

[0040] Viewport transformation (V) converts model information from world coordinates to camera 3D coordinates, as shown in the following formula:

[0041]

[0042] Projection transformation (P) converts three-dimensional coordinate information into two-dimensional coordinates. The formula is as follows:

[0043]

[0044] Window transformation (W) maps the result of viewport clipping onto the screen. It obtains the screen display area size through the viewport and transforms the data within the framebuffer area into pixels that can be displayed on the screen. The formula is as follows:

[0045]

[0046] Based on the above four steps, the formula can be summarized as follows:

[0047]

[0048] Obtain an orthophoto of the cultural relic with attached projection parameters.

[0049] Preferably, in the method for automatic high-precision extraction of cultural relic defects based on three-dimensional fine data, in step three, the defect pixel coordinates obtained in step 2.3) are returned to the projection plane according to the projection matrix parameters, and then the three-dimensional spatial coordinates of the defect are obtained by colliding the coordinate points with the three-dimensional model from the normal direction. This includes the following steps:

[0050] First, based on the window transformation parameters in the MVPW matrix, the screen pixel coordinates (u,v) are converted to projection plane coordinates (x,y). The conversion formula is as follows:

[0051]

[0052] In the formula, w and h are the width and height of the orthophoto, respectively, and r is the minimum bounding box radius of the model.

[0053] Then, the projection plane coordinates are back-projected back to the 3D model coordinates: the projection coordinates are first transformed into 3D, the above (x,y) is changed to (x,y,0), and the intersection point (X,Y,Z) with the 3D model is obtained with the z-axis as the normal.

[0054] Finally, the returned 3D coordinates of the disease are reversed through viewport transformation to obtain the true 3D coordinate information of the disease.

[0055] Preferably, in the method for automatic high-precision extraction of cultural relic defects based on three-dimensional fine data, in step 2.2), the size of each superpixel is N / K, where N is the number of image pixels, and the adjacent seed step size is approximately s = sqrt(N / K).

[0056] Preferably, in the method for automatic high-precision extraction of cultural relic defects based on three-dimensional fine data, step 2.3) of the K-means clustering segmentation includes the following steps: a) selecting the number of clusters k; b) the distance from each sample point to the cluster center; c) updating the cluster center according to the newly divided clusters; d) repeating the above steps b) and c) until the cluster center has not moved.

[0057] Preferably, in the method for automatic high-precision extraction of cultural relic defects based on three-dimensional fine data, step two further includes:

[0058] 2.4) Iterative optimization: Iterate continuously until the error converges;

[0059] 2.5) Perform connectivity processing on neighboring superpixel segments that belong to the same category.

[0060] The present invention has at least the following beneficial effects:

[0061] The research method of this invention activates a large amount of existing three-dimensional fine data in cultural heritage sites, transforming data mining and analysis from conventional two-dimensional data to three-dimensional data. This method is more in line with the scientific use of data in terms of accuracy and efficiency, and has certain scientific research value, guiding value and practical significance.

[0062] This invention achieves higher accuracy than existing methods for automatic lesion extraction from two-dimensional orthophotos. This is because two-dimensional orthophotos are inherently two-dimensional information, resulting in an incomplete lesion survey, and the extraction accuracy is significantly reduced for surfaces with high curvature. This invention utilizes two-dimensional to three-dimensional data conversion, allowing the two-dimensional extraction results to be fed back into a three-dimensional model, ultimately yielding three-dimensional information about the lesions.

[0063] This invention surpasses existing manual 3D vectorization extraction methods in both efficiency and accuracy. 3D vectorization requires manual selection of lesion edges, which is inefficient and its accuracy is affected by human factors. This invention combines image segmentation algorithms for automatic extraction and applies 2D / 3D data conversion to automatically obtain 3D lesion information. This significantly reduces the time spent on manual selection and improves the accuracy of lesion extraction.

[0064] Other advantages, objectives and features of the present invention will be apparent in part from the following description, and in part from the understanding of those skilled in the art through study and practice of the invention. Attached Figure Description

[0065] Figure 1 This is a flowchart illustrating the automatic high-precision extraction method for surface delamination defects of cultural relics based on three-dimensional fine data in an embodiment of the present invention.

[0066] Figure 2 This is a schematic diagram of a three-dimensional colored model of a painted clay artifact in an embodiment of the present invention.

[0067] Figure 3 This is a schematic diagram of the viewport transformation process in the MVPW transformation of this invention.

[0068] Figure 4 This is a schematic diagram of the projection transformation process in the MVPW transformation of this invention.

[0069] Figure 5 These are orthophoto images of three typical damaged areas of the painted clay artifacts in this embodiment of the invention.

[0070] Figure 6 This is a diagram showing the adaptive K-value SLIC segmentation result in an embodiment of the present invention.

[0071] Figure 7 This is a schematic diagram of the K-Means clustering and segmentation algorithm in an embodiment of the present invention.

[0072] Figure 8This is a schematic diagram of pixel coordinates to projection plane coordinates in the inverse projection transformation in an embodiment of the present invention.

[0073] Figure 9 This is a schematic diagram of the projection plane coordinates being back-projected back to the three-dimensional model coordinates during the inverse projection transformation in an embodiment of the present invention.

[0074] Figure 10 This is a schematic diagram of the inverse process of viewport transformation in the inverse projection transformation of this invention.

[0075] Figure 11 These are three-dimensional point clouds of three defects automatically extracted in this embodiment of the invention.

[0076] Figure 12 This is a comparison chart of the accuracy of different superpixel segmentation algorithms in the embodiments of the present invention.

[0077] Figure 13 This is a comparison chart of the accuracy of SLIC segmentation algorithms with different m values ​​in the embodiments of the present invention.

[0078] Figure 14 This is a comparison chart of the accuracy of SLIC segmentation algorithms with different K values ​​in embodiments of the present invention.

[0079] Figure 15 This is a comparison chart of the results of the method described in this embodiment of the invention and the three-dimensional manual extraction method.

[0080] Figure 16 This is a comparison chart of the results of the method and the two-dimensional automatic extraction method in the embodiments of the present invention. Detailed Implementation

[0081] The present invention will now be described in further detail with reference to the accompanying drawings, so that those skilled in the art can implement it based on the description.

[0082] It should be understood that terms such as “having,” “comprising,” and “including” as used herein do not imply the presence or addition of one or more other elements or combinations thereof.

[0083] This invention proposes a method for automatic three-dimensional extraction of surface defects of cultural relics based on a three-dimensional high-precision true-color model. The main technical problems include: (1) using a high-precision three-dimensional color model of the cultural relic, and obtaining a two-dimensional orthophoto of the three-dimensional defect area with projection parameters through two-dimensional and three-dimensional projection transformation; (2) automatically extracting the defect area in the orthophoto with high precision through improved SLIC segmentation and K-means clustering; (3) back-projecting the two-dimensional pixels of the defect area back into the three-dimensional model through back-projection transformation to obtain the three-dimensional defect geometric information of the surface of the three-dimensional model; (4) using OSG as a three-dimensional engine to establish a system with functions such as three-dimensional data reading, visualization, interoperability, defect area calculation, defect annotation, defect statistics and analysis.

[0084] Technical Solution: First, this paper designs a two-dimensional and three-dimensional integrated data conversion model based on the OSG three-dimensional engine, realizing the mutual conversion between three-dimensional color model texture and two-dimensional image; second, it proposes an adaptive K-value SLIC segmentation algorithm, which solves the problem of setting the superpixel K value and improves the image segmentation accuracy; finally, through the two-dimensional and three-dimensional integrated model, statistical analysis and annotation of diseases are performed on the three-dimensional model.

[0085] Specifically, the technical solution provided by this invention is as follows:

[0086] A method for automatic, high-precision extraction of surface flaking defects of cultural relics based on three-dimensional fine data includes the following steps:

[0087] Step 1: Perform MVPW projection transformation on the 3D color model of the cultural relic to achieve orthographic projection transformation of the model and obtain an orthographic image of the damaged area with attached projection parameters.

[0088] Preferably, the model is subjected to MVPW projection transformation to obtain an orthophoto of the projection parameters, specifically including:

[0089] (1). Model Transformation (M): Transform the artifact model from object coordinates to world coordinates;

[0090] (2). Viewport transformation (V) transforms the model from world coordinates to camera coordinates;

[0091] (3). Projection transformation (P) converts three-dimensional coordinate information into two-dimensional coordinates;

[0092] (4). Window transformation (W) maps the result after viewport clipping to the screen. The size of the screen display area is obtained through the viewport, and the data in the frame buffer area is transformed into pixels that can be displayed on the screen.

[0093] Step 2: Automatic extraction of lesion pixel coordinates is achieved by performing SLIC image segmentation and clustering with adaptive K values ​​on orthophotos.

[0094] Preferably, the pixel coordinates of the diseased area are automatically extracted through adaptive K-SLIC segmentation, specifically including:

[0095] (1). Calculate the number of superpixels K using the image gray-level co-occurrence matrix;

[0096] (2). Perform SLIC segmentation using the K calculated above;

[0097] (3) K-means clustering is performed on the segmentation results to obtain the pixel coordinates of the diseased area.

[0098] Step 3: Return the pixel coordinates of the lesion to the projection plane according to the projection matrix parameters, and then obtain the three-dimensional spatial coordinates of the lesion by colliding the coordinate points with the three-dimensional model from the normal direction. Finally, collect the geometric information of the lesion.

[0099] Preferably, the pixel coordinates of the disease are transformed into three-dimensional spatial coordinates through inverse projection transformation, ultimately obtaining the three-dimensional information of the disease. Specifically, this includes:

[0100] (1). Based on the orthographic projection matrix parameters, the pixel coordinates are transformed onto the projection plane to obtain the three-dimensional planar coordinates;

[0101] (2). The three-dimensional coordinates of the disease are obtained by colliding the normal direction of the planar coordinates with the three-dimensional model.

[0102] (3) Perform three-dimensional reconstruction of the three-dimensional coordinates of the disease to obtain the three-dimensional geometric information (perimeter and area) of the disease.

[0103] Specifically, the present invention provides an automatic high-precision extraction method for cultural relic defects based on three-dimensional fine data, comprising:

[0104] Step 1: Perform MVPW projection transformation on the 3D color model of the cultural relic to achieve orthographic projection transformation of the model and obtain an orthographic image of the damaged area with attached projection parameters.

[0105] First, images of the painted sculpture were acquired using a Canon EOS 5D Mark III camera at a resolution of 240 dpi. To obtain a detailed and high-quality 3D model, a sufficient number of overlapping images needed to be captured from various directions. After image acquisition, 3D reconstruction software was used to process the images and generate the 3D model. In this experiment, ContextCapture was used. The model size was then adjusted, and finally, an OBJ format 3D color model of the cultural relic was obtained. Figure 2 As shown.

[0106] Secondly, the model was read, displayed, rotated, translated, and scaled using OSG open-source functions via VS2019. To implement the MVPW matrix transformation, the camera function in OSG was called to perform RTT (render to textures), rendering the damaged surface from the normal direction to the texture. The MVPW matrix transformation includes: Model transformation, View transformation, Projection transformation, and Windows transformation. Since the 3D color model of cultural relics differs from a typical digital terrain model, it only requires the correct scale and its own relative coordinates. Therefore, the model, after initial 3D laser point cloud scale correction and angle adjustment, does not require model transformation; the viewport transformation uses the camera as a reference, with its position as the origin, to obtain the position of the model vertices in the camera coordinate system. Figure 3 This describes the viewport transformation process. The formula is as follows:

[0107]

[0108] In the formula, R and t are a 3×3 rotation matrix and a 3×1 translation vector, respectively.

[0109] The purpose of projection transformation is to convert three-dimensional coordinate information into two-dimensional coordinates. This invention uses orthographic projection to construct a clipped cuboid to clip the scene (taking the bounding box of the model, with a radius of r), and draws it onto the screen according to the original scale of the cultural relic model without distortion. Figure 4 The projection transformation process is illustrated by the following formula:

[0110]

[0111] In the formula, f is the focal length.

[0112] The purpose of window transformation is to map the viewport-cropped result onto the screen. It obtains the screen display area size through the viewport and transforms the data within the frame buffer into pixels that can be displayed on the screen. Ultimately, a high-resolution orthophoto of the model is automatically obtained. The formula is as follows:

[0113]

[0114] In the formula, dX and dY are the physical width and height corresponding to each pixel of the imaging element.

[0115] In summary, based on the four steps above, the formula is as follows:

[0116]

[0117] The final result is an orthophoto of the lesion with projection transformation parameters, such as Figure 5 As shown.

[0118] Step 2: Automatic extraction of lesion pixel coordinates is achieved by performing adaptive K-value SLIC image segmentation and clustering on orthophotos;

[0119] First, the number of adaptive superpixel segments K is calculated using the image complexity described by the gray-level co-occurrence matrix. Then, considering the characteristics of the image data itself, information entropy is introduced, which is the probability of a specific piece of information appearing. The higher the entropy value, the more complex the image, and vice versa. Information entropy (Ent, denoted by E) Ent The calculation formula for (represented) is as follows:

[0120]

[0121] In the formula, k represents the number of distinct flag values ​​within the generalized set, and n iThe number of pixels with a gray value of i represents the number of pixels in the entire image, and N represents the total number of pixels in the entire image.

[0122] The parameters that typically measure the gray-level co-occurrence matrix are: energy (E), contrast (Con, denoted by Ccon), and correlation (Cor, denoted by Ccor). Image complexity T k The calculation formula is as follows:

[0123] T k =E Ent +C Con -EC Cor (6)

[0124] in:

[0125]

[0126]

[0127]

[0128] In the formula: Q 2 (i, j, d, θ) are obtained through gray-level co-occurrence matrix normalization; m represents the number of pixels in the image along the x-axis in the plane; n represents the number of pixels in the image along the y-axis in the plane; θ represents the generation direction of the gray-level co-occurrence matrix, which can be 0°, 45°, 90°, or 135°; where:

[0129]

[0130]

[0131] u1 and u2 are the mean, d1 and d2 represent the variance, and p(i,j,d,θ) represents the pixel information represented by the element in the i-th row and j-th column of the gray-level co-occurrence matrix. Through extensive experiments, we established an empirical formula for calculating the adaptive K value as follows:

[0132]

[0133] In the formula: K represents the number of pre-segments for superpixel segmentation by SLIC, W represents the width of the image, H represents the height of the image, and T represents the height of the image. k This indicates the complexity of the image itself. This indicates rounding up.

[0134] Secondly, SLIC segmentation is performed based on the K value calculated above. The specific steps are as follows:

[0135] (1) Initialize cluster centers. Calculate the gray-level co-occurrence matrix based on image complexity, set the number of adaptive superpixels K, and distribute seed points evenly within the image. The size of each superpixel is N / K, where N is the number of pixels in the image. The adjacent seed step size is approximately S = sqrt(N / K).

[0136] (2) Select new cluster centers within the n-neighborhood of the cluster centers, typically n=3. Calculate the grayscale gradient of all pixels in the neighborhood, iterate through the gradient values ​​of pixels in each neighborhood, and finally select the pixel with the smallest gradient value as the adjusted cluster center.

[0137] (3) Using the adjusted cluster center as the search center and twice the center spacing as the neighborhood search range, construct a center identifier for each original pixel in the segmented image to determine the possible cluster center to which each pixel may belong.

[0138] (4) Calculate distance metrics. This includes color distance and spatial distance. For each searched pixel, calculate its distance to the seed point. The distance calculation method is as follows:

[0139]

[0140] In the formula, Lab consists of one luminance channel and two color channels, where l represents luminance, a represents the components from green to red, and b represents the components from blue to yellow. It can be derived from the RGB conversion of an image.

[0141]

[0142] In the formula, x and y represent the row and column where the pixel is located, respectively.

[0143]

[0144] Where, d c Represents color distance, d s N represents spatial distance. s The maximum spatial distance within a cluster is defined as Ns = s, applicable to each cluster. The maximum color distance Nc varies depending on both the image and the cluster, so we take the compactness m as a fixed constant (ranging from 10 to 40, typically 10). The final distance metric D' is as follows:

[0145]

[0146] Since each pixel is searched by multiple seed points, each pixel has a distance from its surrounding seed points. The seed point with the minimum distance is taken as the cluster center of that pixel.

[0147] (5) Iterative optimization. Iterate continuously until the error converges. Generally, 10 iterations yield the best results.

[0148] (6) Enhance the interconnectivity between superpixels after segmentation. It is common for superpixel segmentation sizes to be too small or for regions that originally belong to the same type to be cut into too many discontinuous superpixels. In this case, it is necessary to perform connectivity processing between them and their neighboring superpixels.

[0149] The purpose of superpixel segmentation is to use superpixels to replace pixels to represent image features, thereby reducing the complexity of image processing. Therefore, the improved SLIC segmentation described above is a preprocessing step for the next image segmentation step. The segmentation result is as follows: Figure 6 As shown.

[0150] Finally, K-means clustering was performed on the above results to extract the coordinates of the diseased pixels. The K-means algorithm mainly consists of the following four steps: (1) selection of the number of clusters k; (2) distance of each sample point to the "cluster center"; (3) update the "cluster center" according to the newly divided clusters; (4) repeat the above steps 2 and 3 until the "cluster center" has not moved. Figure 7 The diagram shown illustrates the K-means algorithm process.

[0151] Step 3: Return the pixel coordinates of the disease to the projection plane according to the projection matrix parameters, and then obtain the three-dimensional spatial coordinates of the disease by colliding the coordinate points with the 3D model from the normal direction. Finally, collect the geometric information of the disease.

[0152] First, based on the window transformation parameters in the MVPW matrix, the screen pixel coordinates (u,v) are converted into projection plane coordinates (x,y). Figure 8 This is a diagram illustrating the conversion from pixel coordinates to projected plane coordinates. The conversion formula is as follows:

[0153]

[0154] In the formula, w and h are the width and height of the orthophoto, respectively, and r is the minimum bounding box radius of the model. This parameter is determined when setting the clipping cuboid for orthogonal projection.

[0155] To backproject the projection plane coordinates back to the 3D model coordinates, first transform the projection coordinates into 3D, i.e., (x, y) becomes (x, y, 0). Then, find the intersection point (X, Y, Z) with the 3D model using the z-axis as the normal. The intersection process is as follows: Figure 9 As shown.

[0156] Finally, the returned 3D coordinates of the defect are reversed through viewport transformation, as follows: Figure 10 As shown, the true three-dimensional coordinate information of the model's defects is obtained. Figure 11The result is a 3D point cloud of the disease. Finally, the 3D point cloud of the disease is greedily triangulated to obtain geometric information such as the area and perimeter of the 3D disease region.

[0157] To demonstrate the superiority of this invention, various comparative experiments were conducted. First, to verify the superiority of SLIC segmentation, this invention used manually extracted defect edges from Photoshop as ground truth and selected commonly used superpixel segmentation algorithms for comparison, including the Seeds algorithm, the LSC algorithm, and the SLIC0 algorithm. All superpixel segmentation methods described below use an adaptive K value calculated from image complexity as the number of superpixels to ensure the comparability of the experiments. Figure 12 The image shows a comparison of the recall and precision of different superpixel segmentation algorithms. The results show that SLIC segmentation has the best extraction accuracy on complex surfaces.

[0158] There are two parameters that affect SLIC segmentation: compactness m and the number of superpixels K. To obtain the optimal compactness m, the experiment set the same number of superpixels, taking m=10, m=20, m=30, and m=40 for SLIC segmentation. Figure 14 The images show a comparison of SLIC segmentation under different m values. For defects with irregular edge shapes, a compactness of m=10 yields the highest accuracy. To verify the superiority of the adaptive k-value SLIC method, the experiment sets the same compactness m and performs SLIC segmentation with k=180, k=360, k=720, and k=1300. Figure 14 The comparison images show SLIC segmentation under different K values, and it is finally concluded that the adaptive K with m=10 achieves the best SLIC segmentation effect.

[0159] To demonstrate the advantages of this invention in disease extraction efficiency, an experiment was conducted comparing manual and automated three-dimensional disease extraction. First, the differences in extracted area and perimeter were compared. Second, the time required for each method was compared. Figure 15 A comparison of the results of automatic 3D extraction and manual 3D extraction methods shows that the extraction efficiency of the present invention is much higher than that of manual 3D extraction.

[0160] To verify the improvement in geometric accuracy achieved by automatic 3D extraction, the experiment compared the 2D result extracted by Photoshop with the 3D result obtained by the 2D-3D conversion method described in this paper. Figure 16 The comparison results of extraction methods in different dimensions show that the area and perimeter of the disease automatically extracted by the method of this invention are slightly larger than those of the two-dimensional automatic extraction method, indicating that the extraction of the present invention is more comprehensive and more accurate.

[0161] The above comparative experiments fully demonstrate the advantages of this invention in terms of extraction accuracy and extraction efficiency.

[0162] This invention proposes an automatic, high-precision extraction method for cultural relic defects (surface flaking) based on detailed 3D data. It includes: designing an integrated 2D / 3D data conversion model based on the OSG 3D engine, which involves acquiring orthophotos using the MVPW matrix and performing inverse projection transformation based on projection matrix parameters, achieving mutual conversion between 3D color model textures and 2D images; inventing an adaptive K-value SLIC segmentation algorithm, which includes calculating the adaptive superpixel count using the image gray-level co-occurrence matrix, SLIC segmentation, and K-Means clustering segmentation. This solves the problem of setting the superpixel K value and improves image segmentation accuracy; finally, through the integrated 2D / 3D model, statistical analysis and annotation of defects are performed on the 3D model. Experiments show that, for complex painted sculptures, the defect extraction method based on the detailed 3D model proposed in this invention improves geometric accuracy and provides a more comprehensive defect investigation compared to currently popular orthophoto extraction methods; compared to manual 3D extraction methods in commercial software, this invention significantly improves the efficiency of defect extraction while maintaining extraction accuracy. This invention activates a large amount of existing three-dimensional detailed data in cultural heritage sites, transforming data mining and analysis from conventional two-dimensional data to three-dimensional data. This is more in line with the scientific use of data in terms of accuracy and efficiency, and has certain scientific research value, guiding value and practical significance.

[0163] The number of modules and processing scale described herein are for the purpose of simplifying the description of the invention. Applications, modifications, and variations of the invention will be readily apparent to those skilled in the art.

[0164] Although embodiments of the present invention have been disclosed above, they are not limited to the applications listed in the specification and embodiments. They can be applied to various fields suitable for the present invention. For those skilled in the art, other modifications can be easily made. Therefore, without departing from the general concept defined by the claims and their equivalents, the present invention is not limited to the specific details and illustrations shown and described herein.

Claims

1. A method for automatic high-precision extraction of cultural relic defects based on three-dimensional fine data, characterized in that, Includes the following steps: Step 1: Obtain an orthophoto of the cultural relic with attached projection parameters; Step 2: Calculate the gray-level co-occurrence matrix based on image complexity and perform adaptive K-value SLIC image segmentation and clustering on the orthophoto, including: 2.1) Calculate the number of adaptive superpixels K of the orthophoto according to equation (14). (14) Where W represents the width of the image and H represents the height of the image. This indicates the complexity of the image itself. 2.2) Perform SLIC segmentation based on the calculated K value: First, seed points are evenly distributed in the image according to the number of adaptive superpixels; then, cluster centers are reselected in the n-neighborhood of the cluster center, where n is a positive integer. The gray-level gradient of all pixels in each neighborhood is calculated, and the gradient values ​​of pixels in each neighborhood are traversed. Finally, the pixel with the smallest value is taken as the adjusted cluster center; then, the adjusted cluster center is used as the search center, and a number of center distances are used as the neighborhood search range to construct a center identifier for each original pixel in the segmented image to determine the possible cluster center to which each pixel may belong. For each searched pixel, the distance between the pixel and the seed point to which it was searched is calculated, and the seed point corresponding to the minimum value is taken as the cluster center of the pixel. Superpixel segmentation is performed on the image; 2.3) K-means clustering is performed on the superpixel segmentation results to extract the pixel coordinates of the defects of cultural relics; Step 3: Convert the pixel coordinates of the damage to three-dimensional planar coordinates to obtain the three-dimensional coordinate information of the damage. Then, perform three-dimensional reconstruction on the three-dimensional coordinates of the damage to obtain the three-dimensional geometric information of the damage. In step three, the defect pixel coordinates obtained in step 2.3) are returned to the projection plane according to the projection matrix parameters. Then, the coordinate points collide with the 3D model along the normal direction to obtain the 3D spatial coordinates of the defect. This includes the following steps: First, according to the window transformation parameters in the MVPW matrix, the screen pixel coordinates (u,v) are converted into projection plane coordinates (x,y). The conversion formula is as follows: , (19,20) In the formula, w and h are the width and height of the orthophoto, respectively, and r is the minimum bounding box radius of the model; Then, the projection plane coordinates are back-projected back to the 3D model coordinates: the projection coordinates are first transformed into 3D, and the above (x,y) is changed to (x,y,0). Then, the intersection point (X,Y,Z) is obtained with the z-axis as the normal and the 3D model. Finally, the returned 3D coordinates of the disease are reversed through viewport transformation to obtain the true 3D coordinate information of the disease.

2. The automatic high-precision extraction method for cultural relic defects based on three-dimensional fine data as described in claim 1, characterized in that, Step 2.1), the step of obtaining the calculation formula for the number of adaptive superpixels K, includes: First, the number of adaptive superpixel segments K is calculated using the image complexity described by the gray-level co-occurrence matrix. Image complexity T k The calculation formula is as follows: (6) in, (5) ； ； ； In the formula: E Ent The information entropy of the image data is represented by k, which represents the number of distinct label values ​​within the generalized set, and n is the number of distinct label values ​​within the generalized set. i Represents the number of pixels with grayscale value i, N represents the total number of pixels in the entire image; E represents the energy of the grayscale co-occurrence matrix, and C... con Represents contrast, C cor Represents relevance; It is obtained through gray-level co-occurrence matrix normalization; m represents the number of pixels in the image along the x-axis in the plane; n represents the number of pixels in the image along the y-axis in the plane; θ represents the generation direction of the gray-level co-occurrence matrix; where: (10,11) (12,13) u1 and u2 are the mean, d1 and d2 represent the variance, and p(i,j,d,θ) represents the pixel information represented by the element in the i-th row and j-th column of the gray-level co-occurrence matrix. Therefore, the formula for calculating the number of adaptive superpixels K is established: (14)。 3. The automatic high-precision extraction method for cultural relic defects based on three-dimensional fine data as described in claim 1, characterized in that, In step 2.2), the distance metric D' between the pixel and its surrounding seed points is as follows: = (18) Where, d c Represents color distance, d s Let m represent spatial distance, compactness, and the size of each superpixel be N / K, where N is the number of pixels in the image. Then, the adjacent seed stride is approximately... .

4. The automatic high-precision extraction method for cultural relic defects based on three-dimensional fine data as described in claim 3, characterized in that, = (15) = (16) = (17) Where, N s It is the maximum spatial distance within the class, defined as N. s =s, applicable to each cluster, is calculated as follows: = (18)。 5. The automatic high-precision extraction method for cultural relic defects based on three-dimensional fine data as described in claim 1, characterized in that, In step one, the method for obtaining an orthophoto of the cultural relic with its accompanying projection parameters includes: Model transformation M transforms the three-dimensional color model of the cultural relic from object coordinates to world coordinates; The viewport transformation V transforms the model information from world coordinates to camera 3D coordinates, as shown in the following formula: (1) The projection transformation P converts three-dimensional coordinate information to two-dimensional coordinates, as shown in the following formula: (2) The window transformation W maps the result of the viewport clipping onto the screen. The size of the screen display area is obtained through the viewport, and the data within the framebuffer area is converted into pixels that can be displayed on the screen. The formula is as follows: (3) Based on the above four steps, the formula can be summarized as follows: (4) Obtain an orthophoto of the cultural relic with attached projection parameters.

6. The automatic high-precision extraction method for cultural relic defects based on three-dimensional fine data as described in claim 1, characterized in that, In step 2.2), the size of each superpixel is N / K, where N is the number of pixels in the image. Therefore, the adjacent seed step size is approximately... .

7. The automatic high-precision extraction method for cultural relic defects based on three-dimensional fine data as described in claim 1, characterized in that, In step 2.3), the K-means clustering segmentation includes the following steps: a) selecting the number of clusters k; b) the distance from each sample point to the cluster center; c) updating the cluster center according to the newly divided clusters; d) repeating the above steps b) and c) until the cluster center has not moved.

8. The automatic high-precision extraction method for cultural relic defects based on three-dimensional fine data as described in claim 1, characterized in that, Step 2 also includes: 2.4) Iterative optimization: iterate continuously until the error converges; 2.5) Connectivity processing is performed on neighboring superpixel segments that belong to the same class.

Images