Three-dimensional digital protection and micro-change detection method for cultural relic mural

Abstract

According to the cultural relic mural three-dimensional digital protection and micro-change detection method, non-contact and damage-free image data acquisition and calculation are performed on the mural collection wall painting by using a digital photogrammetry correlation technology, and an orthographic image DOM and a digital surface model DSM of the mural collection wall painting are obtained. Color management is implemented in the links from data acquisition, calculation to result display, so that the color of the obtained digital orthographic image of the mural is consistent with the color of the actual mural. And carrying out fresco disease change analysis detection on the obtained fresco orthoimage and the digital surface model in a combined manner, obtaining the position and the area of the disease change, identifying the type of the disease, and finally outputting a fresco micro-change detection result extraction image. Scientific detection means and evaluation methods are provided for mural protection, and the efficiency of mural protection work is greatly improved. Meanwhile, detection data are analyzed, the change rule of the mural is mastered, the micro change trend of the mural is predicted, and the mural is effectively protected.

Description

Technical Field

[0001] The present application relates to a mural digital change detection method, and in particular to a cultural relic mural three-dimensional digital protection and micro-change detection method, belonging to the mural digital protection technology field. Background Art

[0002] The murals in the museum collection come from tombs, temples and grottoes, but most of them are from tomb murals, where the environmental conditions at the original site are poor. The temperature, humidity, wind and sand, soil moisture content, carbon dioxide concentration and other complex factors at the original site seriously affect the preservation status of the murals. Groundwater, through capillary action, makes the ground layer and the support body become brittle and soft, causing the murals to fall off. The microenvironment of the cave, such as temperature and humidity, CO2 content and light, is complex and changeable. The repeatedly changing humidity changes the soluble salts in the ground layer, threatening the existence of the murals. The tomb chamber is damp and cold, with microorganisms and groundwater infiltration, which is already a severe challenge for the murals. In addition, the invasion of robbers is a devastating damage to the murals.

[0003] The temperature and humidity in the museum can be controlled and relatively stable, which is suitable for the preservation of murals, but this cannot prevent the continued occurrence of mural diseases. The murals themselves will undergo certain changes due to oxidation, temperature and humidity changes, light and other factors. According to the investigation and research on mural diseases by mural experts over the years, it is found that murals may have several typical diseases such as discoloration, fading, shedding, cracks and hollowing.

[0004] Murals have extremely high historical research value, but due to the long history of murals, they have been damaged by natural and human factors for a long time. The existing murals have different degrees of pests and diseases. If they are not protected, the degree of damage to the murals will become more and more serious. At present, there is an urgent need for a scientific means and evaluation and detection method to effectively protect murals.

[0005] The problems that need to be solved in the existing technology of museum mural protection and change detection and the key technical difficulties of this application include:

[0006] (1) The existing technology detects the pests, colors, salt content, etc. of murals, but only tests and analyzes the murals locally, extracts and repairs one of the diseases separately, and there are few cases of comprehensive detection of mural diseases. At the same time, there are also some other problems: the workload of post-photo calculation and comparison is too large; the images of long-term detection are too large to browse in real time and cannot be viewed intuitively; the post-detection comparison takes a long time; the workload of manual comparison of detection results is relatively large, and the degree of automation is not high. Based on the current status of mural detection and the shortcomings of the above detection methods, in order to meet the needs of information-based real-time detection, there is an urgent need for a mural detection system with a high degree of automation, accurate and reliable detection, simple and convenient operation, and output of quantifiable results.

[0007] (2) The existing technology is unable to achieve effective protection of museum murals, and lacks methods for digital protection of murals and detection of micro-changes in murals. There is a lack of research and development of image mural shooting and detection systems, a lack of equipment systems specifically for shooting murals, a lack of cameras with high resolution and light sources with adjustable color temperature and brightness. The camera and light source cannot move synchronously on the pan-tilt and rail, and cannot ensure that the shooting light environment is balanced, and cannot accurately control the moving distance and angle. There is a lack of photography methods and processes that are different from conventional ones specifically for the characteristics of murals. The existing technology lacks measurement of mural control points, and lacks the layout and measurement of control points. The traditional measurement method is to evenly layout control points, which cannot obtain high-precision digital results, and is not conducive to the three-dimensional digital protection and micro-change detection of cultural relics murals.

[0008] (3) Existing technologies are unable to implement color consistency management for murals. There is a lack of color management methods and processes for murals, and it is impossible to manage and implement color consistency in every link of the digitalization of murals in the collection, from data collection to results generation. There is a lack of special solution design in the selection and layout of displays, light sources and color standards, illumination uniformity control, camera white balance setting, etc.; there is a lack of research on display calibration, camera ICC file generation and application processes, and it is impossible to implement color consistency management of mural images from collection to generation and then to display. This results in a low degree of automation in the detection of digital changes in murals, low detection accuracy, and poor reliability.

[0009] (4) The existing technology is unable to detect micro-changes in murals. It lacks the ability to convert the murals from a coordinate system with the total station as the origin to a coordinate system with the murals as the reference to achieve coarse alignment, and lacks precise alignment based on pixel-level matching. It is unable to detect the four types of mural defects, namely fading, discoloration, cracks and hollowing, based on methods such as HSV color space transformation, gradient and Gaussian-Laplacian operators, and image difference algebraic operations, and combined with the color information of the orthophoto image and the elevation information of the digital surface model. It is unable to extract the location and area of ​​the defect changes, and is unable to identify the type of defect. The digital change detection operation of murals is complicated and cumbersome, and it is unable to output quantifiable results, and it is unable to achieve high-quality micro-change detection of murals. Summary of the invention

[0010] This application uses digital photogrammetry correlation technology to perform non-contact and non-destructive image data collection and calculation on the murals in the collection, and obtains the orthophoto image (DOM) and digital surface model (DSM) of the murals in the collection. Color management is implemented in the data collection, calculation and results display links to ensure that the color of the obtained digital orthophoto image of the mural is consistent with the actual mural color. The obtained mural orthophoto image and digital surface model are jointly analyzed and detected for mural disease changes, the location and area of ​​disease changes are obtained, and the types of diseases are identified, and finally the mural micro-change detection result extraction map is output. It is an important innovation in mural protection methods, combining innovative technologies with cultural relics protection work, providing scientific detection methods and evaluation methods for mural protection, and greatly improving the efficiency of mural protection work. At the same time, the detection data is analyzed to grasp the changing laws of murals and predict the trend of mural micro-changes. According to the prediction information, corresponding preventive measures are taken in advance to effectively protect the murals.

[0011] In order to achieve the above technical effects, the technical solutions adopted in this application are as follows:

[0012] Three-dimensional digital protection and micro-change detection methods for cultural relics murals, including system solutions for digitizing museum murals, color consistency management of murals, and micro-change detection methods for murals;

[0013] 1) Three-dimensional digitization of murals, including: mural shooting detection system, control point layout, and generation of digital results. Based on the special environment of the collection murals, an equipment system for the collection mural shooting is constructed, and a collection mural shooting plan is established, including camera and lens selection, illumination uniformity and image overlap control, photography distance and photography bracket. In terms of control point layout, a total station is used to measure the coordinates of the control points, and coded control points are added. The adjustment accuracy of the murals is controlled by identifying and solving the coded control points contained in the image; after obtaining the sequence images and control points of the collection murals, two digital results, digital orthophotos DOM and digital surface models DSM, are generated for the collection murals;

[0014] 2) Color consistency method for mural images, including: light source selection and layout, color mark selection and layout, camera white balance setting, illumination uniformity control, camera ICC file preparation, image chromatic aberration correction, using flat color management method, selecting a wide color gamut display with stable chromaticity and brightness, and strict on-site correction and color gamut comparison; selecting a suitable light source and deploying it according to on-site conditions; using a color temperature meter and illuminance meter to strictly control illumination uniformity; using a gray card to preset the camera's white balance; selecting an SG color mark that covers more mural colors and preparing the camera's ICC file; embedding the camera's ICC file in the image sequence to complete image correction; maintaining consistency in color space when transferring images between software; finally, comparing the color difference between the standard color block value and the correction value, completing the color consistency management of the mural image from acquisition to generation and then to display;

[0015] 3) Analysis and detection of mural micro-changes: The first is mural image registration, including plane fitting and coordinate transformation, pixel-level image registration; the second is mural disease analysis, including mural disease display, edge improvement global critical value calculation, and mural multi-data joint analysis. Based on the remote sensing image change detection method, the two phases of murals in the collection are subjected to change detection of four diseases: fading, discoloration, cracks and hollowing. The two phases of murals are converted from RGB color space to HSV color space, and Laplace and gradient transformations are performed on the differential operation results of the S and V components respectively. The critical value of the transformation result is calculated as a percentage, and this critical value is used as the limit. Combined with the elevation difference information of the two phases of digital surface models, the location and area of ​​the four diseases are comprehensively judged, and an extraction map of the disease change detection results is generated.

[0016] Preferably, the mural shooting and detection system: is equipped with a high-resolution camera and a light source with adjustable color temperature. The camera and the light source move synchronously on the pan / tilt and the guide rail to ensure that the light environment for shooting is balanced; the guide rail is engraved with line drawing scales and angle values, and can be moved left and right, up and down, and tilted, and the distance movement and angle tilt can be accurately controlled by software;

[0017] (1) Overlap design: The camera needs to take multiple shots and then stitch them together. The overlap within and between flight strips is set to ensure that the image overlap accuracy is the same within the entire mural frame and that every part of the entire mural has the same shooting accuracy. The overlap is 80% in the heading and lateral directions. In the images at the starting and ending shooting positions, the mural content in the horizontal and vertical directions does not exceed 20% of the image.

[0018] (2) Light uniformity control: Using directional geometry, place two light sources on the left and right sides of the camera at a 45-degree angle to the camera.

[0019] (3) Photography support design: The distance the guide rail moves is controlled by setting parameters in the software. The horizontal baseline length is the distance the guide rail moves horizontally, and the vertical baseline length is the distance the guide rail moves vertically. Every time the guide rail moves one baseline length, the camera automatically exposes and completes the shooting without other instructions. The length and width of the mural are determined in advance, and the program automatically calculates the number of photos required in the horizontal direction and the number of times to move in the vertical direction, and automatically controls the guide rail to move horizontally and vertically to complete the shooting of the entire mural.

[0020] (4) Design of photographic distance: Before taking an image, use a single-point laser rangefinder to measure the photographic distance when the camera is aimed at the four corners of the mural. Adjust the front, back, left, right, and tilt angle of the photographic bracket so that the photographic distance values ​​of the camera when photographing the four corners of the mural are consistent. The errors of the four photographic distance values ​​must be within the range required for photographic shooting. When measuring the photographic distance from the camera to the mural, find the photographic center of the camera in advance, that is, the perspective geometric center of the camera. The method is as follows:

[0021] Step 1: Hold the camera and point it at a calibration plate, and take 5-10 images at different positions.

[0022] Step 2: Place the camera on a fixed tripod and take an image facing the calibration plate.

[0023] Step 3: Use the camera calibration algorithm to solve the internal and external orientation elements of the camera, especially the orientation elements of the last photo, where the Z position Zs is the distance from the center of the photo to the calibration plate;

[0024] Step 4: Use a tape measure to measure the distance from Zs to the camera's photographic center.

[0025] Preferably, non-coded control point layout: control points directly measured by total station, specific positions and numbers of control points are manually specified, and the control points are evenly distributed over the entire mural, with one control point arranged for every four baseline lengths;

[0026] Rules for selecting control points: 1) There must be control points near the four corners of the mural, and the control points inside the mural must be evenly distributed; 2) The control points must be located in the pigment layer of the mural, and it is not allowed to select points on the ground layer or support layer of the mural; 3) It is not allowed to select control points at the location of hollowing or collapse; 3) The control points must be easily identifiable corner points, which can be clearly identified on the mural image and in the total station eyepiece;

[0027] A point record file is created for the selected control points. The point record file records the specific position of each control point on the mural sheet, providing a reference for surveyors when measuring the second phase of murals, so that there is a basis for explanation when using control point data for data calculation;

[0028] The point record file is made on the panoramic view of the mural. After the original image of the mural is taken, the panorama is quickly generated using photogrammetry software. The panorama marks the approximate position of the control point. The precise position of the control point needs to be marked on the single original image of the mural. The point record is made into a three-level display. The panoramic mark is the first level, the second level is to mark the position of a certain control point on the original image, and the third level is the mark when the position of the control point of the second level is enlarged to the pixel level. The three-level record of the point record file clearly records the orientation characteristics of the control point, which is convenient for identification;

[0029] After the point record file is prepared, the total station is used to measure the three-dimensional coordinates of the control points. The total station is set up in front of the mural, and all the control points recorded in the point record file are observed in turn to obtain the three-dimensional coordinates of the control points in the current survey station coordinate system. A new coordinate system is established based on the mural. The rules include: 1) All the control point coordinates are fitted into a plane, and the normal direction of the plane points from the mural surface to the total station setting direction as the positive direction of the Z axis of the new coordinate system; 2) The horizontal line in the plane of the mural is used as the positive direction of the X axis, and the right-hand direction of the Z axis and the X axis is used as the positive direction of the Y axis to establish a right-handed coordinate system; 3) The origin of the coordinate system is located in the plane of the mural fitted by all the control points, close to the lower left corner of the mural.

[0030] Preferably, the digital results are: orthophotos and digital surface models of the murals. The orthophotos represent the plane features of the murals, and the digital surface models reflect the characteristics of the ups and downs of the mural surface, providing data for the analysis and detection of changes in the murals.

[0031] Step 1: The color-calibrated mural image is subjected to free network joint adjustment to obtain a sparse three-dimensional point cloud, in which the point cloud is a relative coordinate.

[0032] Step 2: Convert the coordinate system of the control point measured by the total station to the coordinate system of the mural with the total station as the origin, the horizontal line in the mural plane as the X-axis, and the Z-axis and the right-hand direction of the X-axis as the positive direction of the Y-axis.

[0033] Step 3: Import the control points after coordinate system conversion into the photogrammetry software, and correspond each control point one by one to the corresponding position on the mural image, so that the sparse three-dimensional point cloud constructed by the free network is unified to the mural coordinate system; detect the coded identification points on the coded identification stick, and make the detected adjacent coded identification points into one coded identification stick, and use multiple coded identification sticks to optimize the free network adjustment;

[0034] Step 4: Densely match the sparse point cloud to obtain a dense 3D point cloud;

[0035] Step 5: Perform triangulation calculation on the obtained dense 3D point cloud, and perform manual editing and repair;

[0036] Step 6: Export the digital surface model in obj and tif formats in the photogrammetry software;

[0037] Step 7: Perform digital differential correction in the photogrammetry software and output the digital orthophoto in tif format.

[0038] Preferably, the mural image color consistency method:

[0039] (1) Light source selection and layout: The color rendering index of the light source cannot be lower than 90. The color temperature of 5500K is selected as the color temperature when shooting the mural. The light source is selected to adjust the color temperature within the range of 2000K to 6500K. The brightness is adjusted to adapt to the dark environment of the mural gallery. The light source's own illumination uniformity is not less than 90%, the color temperature difference is not greater than 200K, the illumination difference between light sources is less than 10%, and the color temperature difference is less than 200K.

[0040] (2) Color mark selection and placement: Use a professional digital photography SG color mark, 27.94 cm long and 21.59 cm wide, and place it close to an area with no strong reflections and a single color in the entire mural, so that the color mark is minimally affected by the reflected light from the surrounding mural surface content and the color mark is minimally deformed within the camera frame. At the same time, use the pure white and pure black blocks around the color mark to detect whether the lighting is uniform;

[0041] (3) Camera white balance setting: Use the preset white balance and select an 18% gray card for preset white balance. First, place the 18% gray card close to the front of the mural in the same way as the digital SG color standard. Aim the digital camera at the gray card for evaluative metering to set the aperture, shutter speed, and sensitivity. Use manual focus. Take a picture of the gray card with the camera. This image is used as the preset white balance image. The camera then uses the ambient color temperature value of the gray card image as the color temperature reference value under the current shooting conditions.

[0042] (4) Illumination uniformity control: Directional geometric conditions are selected, and two light sources with controllable color temperature and brightness are placed on the left and right sides of the camera at a 45-degree angle to the camera. The illumination uniformity of each mural image is quantitatively controlled by using a light meter to measure the four corners, edges, and middle parts within the actual camera frame. The measured illuminance values ​​of 13 positions, including 4 at the four corners, 4 at the four edges, 1 at the middle, and 4 at the interior, are obtained. If the maximum and minimum differences of these 13 illuminances are within 1000 illuminances and the differences between them are within 500 illuminances, the illumination is considered uniform. If the error range is exceeded, the direction and distance of the light are fine-tuned and the measurement is repeated until the measured illuminance value is within the error range. At this time, the positions of the two light sources are fixed and the entire mural is photographed with the light source positions arranged accordingly.

[0043] Preferably, plane fitting and coordinate conversion: firstly, the coordinate system with the total station as the origin is converted to the mural coordinate system with the mural near the lower left corner as the coordinate origin, the horizontal line in the mural plane as the X-axis, and the Z-axis and the right-hand direction of the X-axis as the positive direction of the Y-axis, to achieve rough alignment;

[0044] The orthophoto images of different periods are unified under a coordinate reference, and some characteristic points that are not easy to change and easy to identify are uniformly selected around and inside the mural to be photographed as the control points of the reference, and the point record files of these control points are prepared. Using these control points, a plane is fitted by the least square method. The direction of the normal vector of the plane from the mural to the viewing point is the positive direction of the Z axis in the mural coordinate system; the horizontal projection of the line connecting the control points at the lower left corner and the lower right corner of the mural on the fitting plane is the X axis of the mural coordinate system, and the direction from the lower left corner point to the lower right corner point is the positive direction of the X axis; the right-hand direction of the Z axis and the X axis in the mural coordinate system is the positive direction of the Y axis;

[0045] When collecting mural data in the later stage, the point record made in the first phase of mural data collection was followed. Each control point recorded in the point record file of each mural was measured one by one. However, when the points in the point record file changed slightly, these changed points were no longer measured, and the remaining control points that did not change were measured repeatedly. A plane was fitted using these repeatedly measured control points, and the coordinates of the measured control points were converted to the mural coordinate system. The orthophotos and digital surface models of murals from different periods were established in a coordinate system based on the same control points. The orthophotos and corresponding digital surface models were in the same coordinate system, and the orthophotos and digital surface models from different periods were also in the same coordinate system.

[0046] Preferably, edge improvement global critical value calculation: For the selection of critical value, based on the Otsu critical value, the grayscale difference less than the critical value Q is set to 0, that is, black, and the grayscale difference greater than the critical value Q is set to 255, that is, white, to distinguish the changed area and position from the background, as shown in the following formula:

[0047] Otsu performs calculations entirely on the histogram of an image. In an M×N digital image, the grayscale L of pixels of different levels is represented by [0, 1, 2, …, L-1], and the number of pixels of grayscale level i is n. i , the total number of pixels MN = n0 + n1 + n2 + ... + n L-1 , the normalized histogram has P i =n i / MN, we get:

[0048] Assume that a critical value T(k)=k is selected, 0<k<L-1, and the critical value of the input image is calculated into two categories C1 and C2, where C1 consists of all pixels in the image whose grayscale values ​​are in the range [0, k], and C2 is the sum of pixels in the image whose grayscale values ​​are in the range [k+1, L-1]. Based on this critical value, the probability P1(k) of a pixel being classified into C1 is:

[0049]

[0050] The probability of class C2 occurring is:

[0051] The average grayscale value of pixels assigned to class C1 is:

[0052]

[0053] The average grayscale value of pixels assigned to class C2 is:

[0054] The cumulative mean up to level k is:

[0055] The average grayscale of the entire image:

[0056] Use the normalized dimensionless matrix to evaluate the quality of the critical value at level k:

[0057]

[0058] is the global variance, i.e. the grayscale variance of all pixels in the image, is the between-class variance, and k is introduced, and finally we get:

[0059]

[0060] The optimal critical value is k * , which maximizes

[0061]

[0062] if The maximum value of corresponds to multiple k values. The largest k values ​​are averaged, and once k* is obtained, the input image is segmented:

[0063]

[0064] Gaussian noise and salt and pepper noise are added, and edge improvement is used to calculate the global critical value based on the Otsu method. The valleys between the histogram peaks are deepened by using pixels that meet the gradient and Laplacian operator metrics.

[0065] Preferably, the global critical value calculation is improved by using the gradient-based edge data: the first-order derivative of the image is calculated by the gradient amplitude, and for the function f(x, y), the gradient at the coordinate (x, y) is defined as a two-dimensional column vector, which indicates the direction of the maximum rate of change of f at the position (x, y);

[0066]

[0067] The magnitude of the vector ▽f is expressed as M(x,y), that is:

[0068]

[0069] The above formula can be approximated by the sum of absolute values, namely:

[0070] M(x,y)≈|g x |+|g y Formula 14

[0071] Construct edge detection gradient operator:

[0072] S(x,y)=|f(i-1,j-1)+2f(i-1,j)+(i-1,j+1)-[f(i+1,j-1)+2f(i+1,j)+f(i+1,j+1 )]|+|f(i-1,j-1)+2f(i,j-1)+f(+1,j-1)-[f(i-1,j+1)+2f(i,j+1)+f(i+1,j+1)]|

[0073] That is, the weighted difference operator is as follows:

[0074]

[0075] In the calculated gradient image, the edges of the areas where the grayscale levels of the original image change slowly will not be ignored.

[0076] Preferably, the global critical value calculation is improved based on the edge data of the Laplace operator: the second-order differential of the image is defined as follows:

[0077]

[0078] The Laplace operator of the two-dimensional image function f(x,y) is defined as:

[0079]

[0080] The Laplace transform is a linear operator. Using the definition of Equation 16, we get:

[0081]

[0082] In the y direction we get:

[0083]

[0084] The discrete Laplacian of two variables is:

[0085]

[0086] When extracting the edge, the Gaussian function is used for low-pass filtering, and then the Laplace operator is used for high-pass filtering. At the same time, the zero point is obtained to form a disease detection operator. The Gaussian function is:

[0087]

[0088] The low-pass filtering result is:

[0089] f(x, y)*g(x, y) Formula 22

[0090] Then the Laplace operator is used to calculate:

[0091]

[0092] get:

[0093]

[0094] After taking the second-order derivative, the maximum point is 0, and the area with a value of 0 is the boundary.

[0095] Preferably, the mural multi-data joint analysis: the fading, discoloration, cracks and hollowing diseases of the murals in the collection are analyzed. First, the orthophoto images of the murals containing simulated noise and diseases are converted from the RGB color space to the HSV color space to obtain the grayscale or intensity information of the three components H, S and V; secondly, the orthophoto images of the murals of the two periods are subjected to difference algebraic operations on the S and V components to obtain the difference data of the S and V components, and the digital surface models of the murals of the two periods are subjected to difference algebraic operations to obtain the elevation difference data; the difference image of the obtained V component is subjected to a global critical value calculation based on the gradient, and the difference image of the obtained S component is subjected to a global critical value calculation based on Gaussian-Laplace to obtain the preliminary data of the change detection of the two components S and V; finally, the elevation difference data of the digital surface model is combined to comprehensively judge the category of the changed disease;

[0096] Step 1: After obtaining the orthophotos and digital surface models of the two phases of the murals in the collection, the orthophotos of the second phase are subjected to the damage display of fading, discoloration, cracks and hollows, and a mixed noise of Gaussian noise and salt and pepper noise is added;

[0097] Step 2: The two phases of orthophoto images are converted from RGB color space to HSV color space to obtain grayscale data of the three components H, S, and V. The S saturation component and the V brightness component are used to extract the change information. The S saturation component is sensitive to discoloration and detects the manifestation of discoloration and crack diseases on the orthophoto image. The V component is a measure of the brightness of the color and detects the reduction of the color brightness caused by fading on the orthophoto image.

[0098] Step 3: Reduce the influence of single pixel errors. Use a 7*7 mean pixel window to perform difference operations on the two-phase orthophotos and the digital surface model to obtain the absolute value of the difference. Use the Gaussian-Laplacian operator on the S saturation component to ignore the types of fading diseases in the area where the grayscale changes slowly, leaving only the areas with drastic color changes caused by discoloration and cracks. Use the gradient operator on the V brightness component to detect the fading with slow grayscale changes and the areas with drastic grayscale changes. Combine the above two judgments to circle the fading area. For discoloration, cracks and hollows, the digital surface model's elevation difference data is combined for collaborative analysis and judgment.

[0099] The change areas extracted from the orthophoto image and the change areas extracted from the digital surface model are combined to identify the types of mural disease. First, for the area with changes on the digital surface model, if there is a corresponding grayscale change in the V component change image relative to this area, the crack disease change is screened out; if there is no corresponding grayscale change in the orthophoto image relative to this area, the hollow disease change is screened out, and then the condition of the area with no change on the digital surface model is analyzed. In this area, when the V component change image has a corresponding grayscale change in the corresponding regional subset, and at the same time in this area In an area, when the S component change image also has a corresponding grayscale change in the corresponding area subset, the discoloration disease change is screened out; if in this area, when the V component change image has a corresponding grayscale change in the corresponding area subset, but in this area, when the S component change image has no corresponding grayscale change in the corresponding area subset, the fading disease change is screened out, and the four types of mural disease changes, fading, discoloration, cracks and hollows, are detected, and the detection results are output to the first original mural image, so that the location and area of ​​the disease can be seen more clearly, and different disease categories are represented by different colors.

[0100] Compared with the prior art, the innovations and advantages of this application are:

[0101] (1) This application uses digital photogrammetry correlation technology to perform non-contact and non-destructive image data collection and calculation on museum murals to obtain orthophotos (DOM) and digital surface models (DSM) of museum murals. Color management is implemented in the data collection, calculation and display stages to ensure that the color of the digital orthophotos of the murals remains consistent with the actual color of the murals. The mural orthophotos and digital surface models obtained are jointly analyzed and tested for changes in mural diseases to obtain the location and area of ​​disease changes, and the types of diseases are identified, and finally an extraction map of mural micro-change detection results is output. This is an important innovation in mural protection methods, combining innovative technologies with cultural relics protection work, providing scientific detection methods and evaluation methods for mural protection, and greatly improving the efficiency of mural protection work. At the same time, the detection data is analyzed to understand the changing patterns of murals and predict the trend of micro-changes in murals. Based on the predicted information, corresponding preventive measures are taken in advance to effectively protect the murals.

[0102] (2) The present applicant has developed a set of equipment systems specifically for photographing murals. Unlike ordinary image capture equipment, this equipment system is equipped with a high-resolution camera and a light source with adjustable color temperature and brightness. The camera and light source move synchronously on the pan / tilt and guide rail to ensure that the light environment for shooting is balanced. The guide rail is engraved with line scales and angle values, and can be moved left and right, up and down, and tilted. The moving distance and angle can be accurately controlled by software. A photography method and process that is different from conventional methods has been developed specifically for the characteristics of murals. The measurement of mural control points has been achieved, and the layout and measurement of control points have been established. In order to improve the measurement accuracy of the total station, the control points are measured by combining traditional methods with coded control points, and high-precision digital results have been obtained, which is conducive to the three-dimensional digital protection and micro-change detection of cultural relics murals.

[0103] (3) This application implements the management of color consistency of murals. Based on the mature color management methods and processes of the plane, the color consistency of the digitalization results of the collection of murals is managed and implemented in each link from data collection to result generation. The scheme is designed in the selection and layout of displays, light sources and color standards, illumination uniformity control, camera white balance setting, etc.; the display calibration, camera ICC file generation and application processes are studied to achieve color consistency management of mural images from collection to generation and then to display. The digitization change detection of murals has a high degree of automation, high detection accuracy and good reliability.

[0104] 4) This application realizes the analytical detection of micro-changes in murals. First, the mural is converted from a coordinate system with the total station as the origin to a coordinate system with the mural as the reference to achieve coarse alignment, and then fine alignment is achieved based on pixel-level matching; then based on HSV color space transformation, gradient and Gaussian-Laplace operator, image difference algebraic operation and other methods, and combined with the color information of the orthophoto image and the elevation information of the digital surface model, the four mural diseases of fading, discoloration, cracks and hollowing are analyzed and detected, the location and area of ​​the disease changes are extracted, and the type of disease is identified. The mural detection with high degree of automation, accurate and reliable detection, simple and convenient operation, and output of quantifiable results is realized. BRIEF DESCRIPTION OF THE DRAWINGS

[0105] Figure 1 This is a schematic diagram of the flight strip array taken from the image of the museum's mural.

[0106] Figure 2 This is a schematic diagram of the requirements for photographing the starting and ending positions of the images of the murals in the museum collection.

[0107] Figure 3 It is a schematic diagram of the relationship between the positions of a digital camera and a light source.

[0108] Figure 4 It is a schematic diagram of the coordinate system for mural image registration.

[0109] Figure 5 This is the measurement selection chart of control points in the second phase.

[0110] Figure 6 This is a simulated schematic diagram of the discoloration disease area.

[0111] Figure 7 It is a simulation diagram of the gap disease area.

[0112] Figure 8 are the calculated gradient images and histograms.

[0113] Fig. 9 It is a flow chart for detecting micro-changes in three-dimensional digitization of murals.

[0114] Fig.10 This is a structural diagram of the multi-data joint analysis of murals.

[0115] Fig.11 This is the coordinate and comparison result diagram under the mural coordinate system.

[0116] Fig.12 This is the result extraction diagram of mural disease change detection. DETAILED DESCRIPTION

[0117] The following, in conjunction with the accompanying drawings, further describes the technical solution of the three-dimensional digital protection and micro-change detection method of cultural relics murals provided by the present application, so that technical personnel in the field can better understand the present application and implement it.

[0118] This application includes the design of a system solution for the digitization of museum murals, the color management of mural color consistency, and the research on mural micro-change detection methods. 1) Three-dimensional digitization of murals, including: mural shooting detection system, control point layout, and generation of digital results. Based on the special environment of museum murals, an equipment system for shooting museum murals is built, and a shooting plan for museum murals is established, including camera and lens selection, lighting uniformity and image overlap control, photography distance and photography support. In terms of control point layout, a total station is used to measure the coordinates of control points, and coded control points are added. The coded control points contained in the image are identified and solved to control the adjustment accuracy of the mural; after obtaining the sequence images and control points of the museum murals, a mural is generated. The two digital results are the digital orthophoto DOM and the digital surface model DSM of the murals in the collection; 2) The color consistency method of the mural images, including: light source selection and layout, color scale selection and layout, camera white balance setting, lighting uniformity control, camera ICC file production, and image chromatic aberration correction. The flat color management method is used to select a display with stable chromaticity and brightness and a wide color gamut, and strict on-site calibration and color gamut comparison are performed; the appropriate light source is selected and arranged according to the on-site conditions; the color temperature meter and illuminance meter are used to strictly control the lighting uniformity; the gray card is used for camera White balance preset; select SG color scale that covers most of the mural colors and make the camera's ICC file; embed the camera's ICC file in the image sequence to complete the image correction; maintain the consistency of the color space when transferring images between software; finally, compare the color difference between the standard color block value and the correction value to complete the color consistency management of the mural image from acquisition to generation and then to display; 3) Analysis and detection of mural micro-changes: one is mural image registration, including plane fitting and coordinate transformation, pixel-level image registration, and the other is mural disease analysis, including mural disease display and edge improvement Global critical value calculation and joint analysis of multiple mural data are carried out based on remote sensing image change detection methods. The two phases of murals are subjected to change detection of four diseases, namely fading, discoloration, cracks and hollowing. The two phases of murals are converted from RGB color space to HSV color space, and Laplace and gradient transformations are performed on the differential operation results of the S and V components respectively. The critical value is calculated by percentage for the transformation results. This critical value is used as the limit, and the elevation difference information of the two phases of digital surface models is combined to comprehensively judge the location and area of ​​the four diseases, and generate an extraction map of the disease change detection results.

[0119] The special environment of the museum's mural exhibition creates the following difficulties for photographing the murals: the glass on the front and sides of the display cabinets will reflect light to varying degrees; there will be varying degrees of color interference between adjacent and surrounding murals; and the height of the top of the mural from the roof is low, making it inconvenient for the photographic equipment to work.

[0120] 1. 3D digitalization of murals

[0121] (I) Mural photography and detection system

[0122] A set of equipment system specially designed for photographing and detecting murals in the collection has been established. Different from ordinary image capture equipment, the system is equipped with a high-resolution camera and a light source with adjustable color temperature. The camera and light source move synchronously on the pan / tilt and guide rail to ensure a balanced light environment for shooting. The guide rail is engraved with line scales and angle values ​​and can be moved and tilted left and right, up and down. The software can be used to precisely control the distance movement and angle tilt to suit the needs of mural placement. A photography method and process that is different from conventional methods and processes has been developed specifically for the characteristics of the murals in the collection.

[0123] (1) Overlap design: The murals in the collection are relatively large, and they need to be photographed multiple times by the camera and then stitched together to form a complete mural. The overlap within and between flight strips is set to ensure that the image overlap accuracy is the same within the entire mural. Figure 1 As shown. At the start and end of the shooting, the mural content in the image frame is quite special, and the image frame will contain a relatively small proportion of the mural content. During the whole shooting process, the middle and edge parts of the mural need to have the same overlap, and each part of the mural needs to have the same shooting accuracy. According to the 80% overlap in the heading and lateral directions, the mural content in the images at the start and end shooting positions should not occupy more than 20% in the horizontal and vertical directions of the image, as shown in the figure. Figure 2 (2) Light uniformity control: The color of the orthophoto image of the museum murals obtained by shooting should be consistent with the color of the original murals. It is necessary to ensure that each mural image is under the same lighting conditions when shooting, and the lighting should be uniform in the camera frame to avoid the phenomenon of bright in the middle and dark on both sides. It is also necessary to avoid strong reflective areas in the frame. Using directional geometric conditions, place two light sources on the left and right sides of the camera and place them at a 45-degree angle to the camera. The specific placement is as follows: Figure 3As shown, 45 degrees excludes specular reflection light, retains color information to the maximum extent, and makes the measurement results have a good correlation with visual evaluation. (3) Design of photography stand: The murals in the museum are not all vertically erected in the display cabinets. Some are slightly leaning against the wall of the display cabinet. Therefore, the photography stand is designed to be able to pitch forward and backward, and rotate left and right. According to the display posture of the mural, the inclination and rotation angle of the photography stand are adjusted. For example, when the angle between the mural and the vertical direction is about 5 degrees, the photography stand is adjusted to tilt forward 5 degrees to make it parallel to the horizontal plane of the mural, ensuring that it does not exceed the depth of field of the camera, and realizing the control and shooting of high-definition murals. In order to control the uniformity of lighting, it is necessary to reasonably arrange the relative position of the light source and the camera. Therefore, in the early stage, it is necessary to adjust the position of the two light sources and the camera. In order to facilitate the adjustment of the distance and angle of the light source, the gimbal that fixes the camera and the light source is designed to be able to move forward and backward, up and down, and rotate at any angle. Once the lighting is evenly arranged, the position relationship between the light source and the camera is fixed and kept relatively unchanged. The distance the rail moves is controlled by setting parameters in the software. The horizontal baseline length is the distance the rail moves horizontally, and the vertical baseline length is the distance the rail moves vertically. Each time the rail moves a baseline length, the camera automatically exposes and completes the shooting without other instructions. The length and width of the mural are determined in advance, and the program automatically calculates the number of shots required in the horizontal direction and the number of times the mural is moved in the vertical direction, and automatically controls the rail to move horizontally and vertically to complete the shooting of the entire mural. (4) Design of shooting distance: In the process of shooting murals, due to the requirements of high-resolution shooting accuracy, the depth of field of the camera is small. If the camera moves or shakes slightly, the shooting distance will exceed the depth of field of the camera, and the mural image will be blurred. Therefore, the distance of camera movement must be very accurate, and the distance between the camera center and the mural to be shot must also be accurately measured. Before taking an image, a single-point laser rangefinder is used to measure the photographic distance when the camera is aimed at the four corners of the mural. The front, back, left, right, and right positions of the photographic bracket and the size of the inclination angle are adjusted so that the photographic distance values ​​of the camera when shooting the four corners of the mural are consistent, and the errors of the four photographic distance values ​​must be within the range required for photographic shooting. When measuring the photographic distance from the camera to the mural, the photographic center of the camera, that is, the perspective geometric center of the camera, is found in advance. The method is as follows: Step 1: Hold the camera and aim it at a calibration plate, and shoot 5-10 images at different positions; Step 2: Put the camera on a fixed tripod and shoot an image facing the calibration plate almost completely; Step 3: Use the camera calibration algorithm to solve the internal and external orientation elements of the camera, especially for the solution of the orientation elements of the last photo, where the Z direction position Zs is the distance from the photographic center to the calibration plate; Step 4: Use a tape measure to measure the position of the camera's photographic center according to the distance Zs. Find the position of the camera's photographic center, mark the position of the camera's photographic center, and use a single-point laser rangefinder to control the distance from the camera's photographic center to the mural.

[0124] (II) Deployment of control points

[0125] In order to constrain the scaling ratio of the digitalized murals and ensure the measurement accuracy of the digitalized murals, it is necessary to select several control points on each mural to control the image accuracy. The layout design of the control points for mural photography includes: one is to select points on the mural, measure the selected points with the total station, and obtain the three-dimensional coordinates of the selected points; the other is to arrange coded control points around the mural, and further improve the measurement accuracy of the control points by photographing the coded control points while photographing the mural image. (1) Non-coded control point layout: The control points are directly measured by the total station, and the specific position and number of the control points are manually specified. The control points are evenly distributed on the entire mural, and one control point is arranged for every 4 baseline lengths; control point selection rules: 1) There must be control points near the four corners of the mural, and the control points inside the mural must be evenly distributed; 2) The control points must be located in the paint layer of the mural, and it is not allowed to select points on the ground layer or support layer of the mural; 3) It is not allowed to select control points at the location of hollowing or collapse diseases; 3) The control points must be easily identifiable The corner point positions of the mural can be clearly identified on the mural image and in the total station eyepiece; a point record file is made for the selected control points. The point record file records the specific position of each control point on the mural sheet, providing a reference for surveyors to measure the second phase of the mural, so that there is a basis for explanation when using the control point data for data calculation; the point record file is made on the mural panorama. After the original image of the mural is taken, the photogrammetry software is used to quickly generate a panorama. The panorama marks the approximate position of the control point, and the precise position of the control point needs to be marked on the single original image of the mural. The point record is made into a three-level display, with the panorama mark as the first level, the second level is to mark the position of a certain control point on the original image, and the third level is the mark when the position of the control point of the second level is enlarged to the pixel level. The three-level record of the point record file clearly records the orientation characteristics of the control point, which is convenient for identification. If this file is not created, after the total station measurement, the total station observer will have to rely entirely on his or her brainpower to remember the positions of the control points. Faced with a large number of control points, this will increase the observer's mental burden and will not be convenient for the use of control points in subsequent digital production.After the point record file is prepared, the total station is used to measure the three-dimensional coordinates of the control points. The total station is set up in front of the mural, and all the control points recorded in the point record file are observed in turn to obtain the three-dimensional coordinates of the control points in the current station coordinate system. The coordinates of the control points obtained by the total station are the coordinates in the coordinate system with the total station as the origin, which does not meet the requirements of the three-dimensional digitization of the mural. Therefore, it is necessary to establish a new coordinate system based on the mural. The rules include: 1) All the control point coordinates fit a plane, and the normal direction of the plane points from the mural surface to the total station setting direction as the positive direction of the Z axis of the new coordinate system; 2) The horizontal line in the mural plane is used as the positive direction of the X axis, and the right-hand direction of the Z axis and the X axis is used as the positive direction of the Y axis to establish a right-handed coordinate system; 3) The origin of the coordinate system is located in the mural plane fitted by all the control points, close to the lower left corner of the mural. In order to reduce human observation errors, stations are set up at two observation positions and observations are carried out twice. After the observation, the coordinates of the control points are converted, and the coordinate values ​​of each control point in the new coordinate system are compared. If the difference is within 1mm, it is considered qualified, otherwise it is considered unqualified. (2) Coding control point layout: Since the accuracy of the total station measurement point is usually 1mm, in order to further improve the measurement accuracy of the control points and the overall measurement accuracy of the mural control points, coding markers are placed around the mural. Each coding marker has 3 marking points with different numbers, surrounded by garbled patterns; the marking point numbers on each coding marker are different to ensure the uniqueness of the marking points; the garbled patterns on each coding marker are also different to ensure that the match is 0. The coding markers are placed around the mural to ensure that the camera shoots the mural together with the coding markers.

[0126] (III) Generate digital results: orthophotos and digital surface models of the murals. The orthophotos represent the planar features of the murals, and the digital surface models reflect the ups and downs of the mural surface, providing data for analyzing and detecting changes in the murals.

[0127] Step 1: The color-calibrated mural image is subjected to joint adjustment and solution by FreeNet to obtain a sparse three-dimensional point cloud, in which the point cloud is a relative coordinate. Step 2: The coordinate system of the control point coordinates measured by the total station is converted from the coordinate system with the total station as the origin to the mural coordinate system with the origin near the lower left corner of the mural, the horizontal line in the mural plane as the X-axis, and the Z-axis and the right-hand direction of the X-axis as the positive direction of the Y-axis. Step 3: The control points after the coordinate system conversion are imported into the photogrammetry software, and each control point is matched one by one at the corresponding position on the mural image, so that the sparse three-dimensional point cloud constructed by FreeNet is unified. to the mural coordinate system; detect the coded identification points on the coded identification stick, and make the detected adjacent coded identification points into one coded identification stick, and use multiple coded identification sticks to optimize the free network adjustment; Step 4: densely match the sparse point cloud to obtain a dense three-dimensional point cloud; Step 5: perform triangulation calculation on the obtained dense three-dimensional point cloud, and perform manual editing and repair; Step 6: output the digital surface model in obj format and tif format in the photogrammetry software; Step 7: perform digital differential correction in the photogrammetry software, and output the digital orthophoto image in tif format.

[0128] 2. Method for color consistency of mural images

[0129] Murals cannot be easily moved or touched many times, and have high requirements for shooting equipment and technology. They are sensitive to light and cannot be exposed to the light environment for a long time. The large size of the murals requires taking multiple images and each image needs to maintain consistent light environment and camera equipment conditions. The process from shooting to displaying the murals in the collection includes light source selection and layout, color code selection and layout, camera white balance setting, display calibration, and lighting uniformity control.

[0130] (1) Selection and layout of light sources: Considering the color rendering characteristics of the light source, the color rendering index of the selected light source cannot be lower than 90. Such a light source can reproduce the color of the object relatively well with less color loss. At the same time, considering the adjustable color temperature and brightness range of the light source, select 5500K as the color temperature when shooting the mural, select the light source in the range of 2000K to 6500K to adjust the color temperature, and adjust the brightness to adapt to the darker environment of the mural gallery. The light source's own illumination uniformity is not less than 90%, the color temperature difference is not greater than 200K, the illumination difference between light sources is less than 10%, and the color temperature difference is less than 200K; when shooting, all lights in the gallery are turned off, and the selected light source is the only light source for shooting. The position of the light source is based on Figure 3Layout: Select at least two light sources, refer to the measured value of the light meter, and fine-tune the position of the light source to make the camera frame evenly illuminated. (2) Color scale selection and layout: The non-human and non-scene areas on the frame are mostly brown earth color, and they account for a large proportion. This earth color is close to human skin color. There are also certain people in the frame, so the color blocks of the color scale should contain such colors. In the commonly used standard 24 color scales, there is only one skin color block. The skin colors contained in the 48-color or SG color scale are arranged in steps. The error will be relatively reduced in the conversion of the color model and the mapping of the color gamut, so it is better to use the 48-color or SG color scale. In addition, the SG color scale contains 140 color blocks. The surrounding white, gray and black color blocks allow the characterization software to offset the uneven lighting and improve the accuracy of the color relationship. Therefore, a digital photography professional SG color standard with a length of 27.94cm and a width of 21.59cm is used. The digital SG color standard is placed close to the area of ​​the mural without strong reflection and single color, so that the color standard is least affected by the reflected light from the surrounding mural surface content. When placing the color standard, try to place it upright without excessive front and back tilt or left and right rotation, so that the color standard can achieve the minimum deformation within the camera frame. At the same time, the pure white and pure black blocks around the color standard are used to detect whether the lighting is uniform. (3) Camera white balance setting: Use the preset white balance and select an 18% gray card for preset white balance. First, place the 18% gray card close to the front of the mural in the same way as the digital SG color standard. The digital camera is aimed at the gray card for evaluation metering to set the aperture, shutter and sensitivity. Manual focus is used for focus. The camera takes a picture of the gray card and uses the image taken as the preset white balance picture. The camera then uses the ambient color temperature value carried by the gray card image currently taken as the color temperature reference value under the current shooting conditions. (4) Light uniformity control: Select directional geometry and place two light sources with controllable color temperature and brightness on the left and right sides of the camera at a 45-degree angle to the camera. The specific placement is as follows: Figure 3As shown. To quantitatively control the illumination uniformity of each mural image, a light meter is used to measure the four corners, edges and middle parts within the actual camera frame, and the measured illumination values ​​of 13 positions, including 4 at the four corners, 4 at the four edges, 1 at the middle position and 4 at the inside, are obtained. If the maximum and minimum differences of these 13 illuminations are within 1000 illuminations and the mutual differences are within 500 illuminations, the illumination is considered uniform; if it exceeds the error range, the direction and distance of the light are fine-tuned and then measured again until the measured illumination value is within the error range. At this time, the positions of the two light sources are fixed, and the entire mural is photographed with the light source positions arranged. (5) Camera ICC file creation: The camera characteristic file establishes the correspondence between the color value of the camera device and the CIE chromaticity value of the corresponding color. Take a RAW format image with an SG color mark or other selected color mark, convert it to a 16-bit tif format, load this tif image containing the color mark into the ProfileMaker software, compare and calculate it with the standard color mark measurement file, and obtain the camera's ICC characteristic file. If the illumination of the captured color mark is uneven, repeatedly adjust the exposure, saturation and contrast in ProfileMaker to make the color difference value within 6. (6) Color difference correction of captured images: Perform color correction on each mural image taken under the same lighting conditions, and embed each image into the generated camera ICC characteristic file. If the image needs to be edited, the edited color space should be the AdobeRGB color space. After editing, the saved image still needs to embed the camera's ICC file.

[0131] 3. Analysis and detection of micro-changes in murals

[0132] The earlier the occurrence of the disease is detected, the earlier the mural disease can be contained in its infancy, which is an effective means of mural protection. The digital results of the two phases of the collection of murals are first registered, and then the disease change analysis and detection are carried out.

[0133] (i) Mural image registration: The orthophoto images of the murals obtained in the two phases and the digital surface model are matched and superimposed. First, the murals are converted from a coordinate system with the total station as the origin to a coordinate system with the murals as the reference to achieve coarse registration, and then fine registration is achieved based on pixel-level matching.

[0134] 1. Plane fitting and coordinate transformation: Change detection is to compare and analyze the orthophotos of murals from different periods. Therefore, the coordinate systems of the orthophotos from different periods need to be unified into a reference coordinate system before analytical detection can be performed. First, the coordinate system with the total station as the origin is converted to the mural coordinate system with the mural near the lower left corner as the coordinate origin, the horizontal line in the plane of the mural as the X-axis, and the Z-axis and the right-hand direction of the X-axis as the positive direction of the Y-axis to achieve rough alignment. Unify the orthophotos from different periods under a coordinate reference, evenly select some feature points that are not easy to change and easy to identify around and inside the mural to be photographed as control points of the reference, and keep a record file of these control points. Use these control points to fit into a plane through the least squares method. The direction of the plane's normal vector from the mural to the viewing direction is the positive direction of the Z axis in the mural coordinate system; the horizontal projection of the line connecting the control points at the lower left corner and the lower right corner of the mural on the fitting plane is the X axis of the mural coordinate system, and the direction from the lower left corner point to the lower right corner point is the positive direction of the X axis; the right-hand direction of the Z axis and the X axis in the mural coordinate system is the positive direction of the Y axis. Specifically, Figure 4 When the mural data is collected later, the point record of the first phase of mural data collection is followed. Each control point recorded in the point record file of each mural is measured one by one. However, when the points in the point record file change slightly, these changed points are no longer measured, such as Figure 5 The remaining unchanged control points are measured repeatedly, and a plane is fitted using these repeatedly measured control points, and the coordinates of the measured control points are converted to the mural coordinate system. The orthophotos and digital surface models of the murals of different periods are established in the coordinate system based on the same control points. The orthophotos and the corresponding digital surface models are in the same coordinate system, and the orthophotos and digital surface models of different periods are also in the same coordinate system.

[0135] 2. Pixel-level image registration: Perform feature search within a specific small range to improve search efficiency, obtain the homography matrix through feature point pairs, perform local registration for murals with larger areas, use downsampling for sparse matching, and correspond the obtained sparse matching points to the original image to calculate the homography matrix; perform overall registration for murals with smaller areas, and achieve one-to-one correspondence between the spatial positions of pixels in images of different periods through local or overall image registration.

[0136] (II) Analysis of mural damage

[0137] In just one or two months, it is not enough for the human eye to detect the disease of the mural. In fact, the mural is changing or brewing every day, but it takes several years or even more than ten years for the naked eye to detect the changes. Once the mural is damaged, the speed of the disease will increase faster and faster. Manual detection of mural damage changes alone is not only a waste of manpower and material resources, but also difficult to detect the changes that are not visible to the human eye, and the detection efficiency is not high. The detection method includes analysis of the discoloration, fading, cracks, and hollowing of the mural.

[0138] 1. Display of mural disease: For discoloration disease, select a location on the mural where discoloration may occur, change the original color to another color, and complete the simulation of the discoloration disease area, such as Figure 6 As shown. For fading damage, select a location on the mural where fading may occur, lighten the color of the original location, and complete the simulation of the fading disease area. For crack changes, find the location where there is a crack on the mural, select one place, darken the color of the selected crack on the orthophoto image, and make the crack thicker and longer around the crack. Thickening means expanding the edge of the original crack outward, and lengthening means lengthening the length of the original crack. When the crack becomes wider and longer, the depth of the ground layer increases, and the elevation value of the changed location decreases compared to the original. Changes are made simultaneously on the orthophoto image and the digital surface model to complete the simulation of the crack disease area, as shown. Figure 7 As shown in the figure, for hollowing damage, an area with a gap is selected on the orthophoto image, and the elevation within the approximate ellipse is increased to varying degrees at the position corresponding to the digital surface model. The elevation change in the middle of the ellipse is the largest, and the elevation change gradually decreases from the center to the outside, completing the simulation of the hollowing damage area.

[0139] 2. Edge improvement global critical value calculation: For the selection of the critical value, based on the Otsu critical value, the grayscale difference less than the critical value Q is set to 0, that is, black, and the grayscale difference greater than the critical value Q is set to 255, that is, white, to distinguish the changed area and position from the background, as shown in the following formula:

[0140] Otsu performs calculations entirely on the histogram of an image. In an M×N digital image, the grayscale L of pixels of different levels is represented by [0, 1, 2, …, L-1], and the number of pixels of grayscale level i is n. i , the total number of pixels MN = n0 + n1 + n2 + ... + n L-1 , the normalized histogram has P i =n i / MN, we get:

[0141] Assume that a critical value T(k)=k is selected, 0<k<L-1, and the critical value of the input image is calculated into two categories C1 and C2, where C1 consists of all pixels in the image whose grayscale values ​​are in the range [0, k], and C2 is the sum of pixels in the image whose grayscale values ​​are in the range [k+1, L-1]. Based on this critical value, the probability P1(k) of a pixel being classified into C1 is:

[0142]

[0143] The probability of class C2 occurring is:

[0144] The average grayscale value of pixels assigned to class C1 is:

[0145]

[0146] The average grayscale value of pixels assigned to class C2 is:

[0147]

[0148] The cumulative mean up to level k is:

[0149] The average grayscale of the entire image:

[0150] Use the normalized dimensionless matrix to evaluate the quality of the critical value at level k:

[0151]

[0152] is the global variance, i.e. the grayscale variance of all pixels in the image, is the between-class variance, and k is introduced, and finally we get:

[0153]

[0154] The optimal critical value is k*, which maximizes

[0155] if The maximum value of corresponds to multiple k values. The largest k values ​​are averaged, and once k* is obtained, the input image is segmented:

[0156] Gaussian noise and salt and pepper noise are added, and edge improvement is used to calculate the global critical value based on the Otsu method. The valleys between the histogram peaks are deepened by using pixels that meet the gradient and Laplacian operator metrics.

[0157] (1) Improving global critical value calculation using gradient-based edge data

[0158] The first-order derivative of the image is calculated by the gradient magnitude. For the function f(x, y), the gradient at the coordinate (x, y) is defined as a two-dimensional column vector, which indicates the direction of the maximum rate of change of f at the position (x, y);

[0159]

[0160] The magnitude of the vector ▽f is expressed as M(x,y), that is:

[0161]

[0162] The above formula can be expressed approximately by the sum of absolute values, namely: M(x, y)≈|gx|+|gy| Formula 14

[0163] Construct edge detection gradient operator:

[0164] S(x,y)=|f(i-1,j-1)+2f(i-1,j)+f(i-1,j+1)-[f(i+1,j-1)+2f(i+1,j)+f(i+1,j+ 1)]|+|f(i-1,j-1)+2f(i,j-1)+f(i+1,j-1)-[f(i-1,j+1)+2f(ij+1)+f(+1,j+1)]|

[0165] That is, the weighted difference operator is as follows:

[0166] Figure 8 It is a calculated gradient image. The edges of the areas where the grayscale levels on the original image change slowly will not be ignored, and it responds strongly to the edges of noise points.

[0167] (2) Improved global critical value calculation based on edge data of Laplace operator

[0168] The second-order differential of an image is defined as follows:

[0169] The Laplace operator of the two-dimensional image function f(x,y) is defined as:

[0170] The Laplace transform is a linear operator. Using the definition of Equation 16, we get:

[0171]

[0172] In the y direction we get:

[0173] The discrete Laplacian of two variables is:

[0174]

[0175] When extracting the edge, the Gaussian function is used for low-pass filtering, and then the Laplace operator is used for high-pass filtering. At the same time, the zero point is obtained to form a disease detection operator. The Gaussian function is:

[0176]

[0177] The result of low-pass filtering is: f(x, y)*g(x, y) Formula 22

[0178] Then the Laplace operator is used to calculate:

[0179] get:

[0180]

[0181] After taking the second-order derivative, the maximum point is 0, and the area with a value of 0 is the boundary;

[0182] Obtain the gradient and Gaussian-Laplace images, set the critical values ​​for the gradient and Gaussian-Laplace images using percentage statistics, and transform a set of n data x1, x2, …, x n Arrange them in order from small to large, count the accumulated percentiles, perform Gaussian-Laplace calculation on the image of the S component to obtain the absolute value of the Gaussian-Laplace image, perform percentage critical value based on this image to obtain the critical value Ts, perform gradient transformation on the image of the V component to obtain the gradient amplitude image, perform percentage critical value based on this image to obtain the critical value Tv, and the percentage is set to 99.7%.

[0183] 3. Joint analysis of multiple mural data

[0184] The orthophoto image and digital surface model data of the mural are combined to detect the micro-changes of the mural. The orthophoto image of the mural only reflects the color attributes and geographic coordinate information of the mural on the horizontal plane. Cracks and hollows are diseases that occur in the ground layer. For example, although cracks can also show color changes in the orthophoto image, they are very similar to the changes in color changes and are difficult to distinguish. For another example, hollow diseases, it is difficult to change the color information in the orthophoto image, and the orthophoto image data alone is not enough to detect hollow diseases. In view of the above reasons, the orthophoto image and digital surface model of the mural, that is, the plane data and elevation data, are used to analyze the changes in mural diseases in a coordinated manner. The fading, discoloration, cracks and hollowing diseases of the murals in the collection are analyzed. First, the orthophoto images of the murals containing simulated noise and diseases are converted from RGB color space to HSV color space to obtain the grayscale or intensity information of the three components H, S, and V. Secondly, the difference algebraic operation is performed on the S and V components of the orthophoto images of the two phases of the murals to obtain the difference data of the S and V components, and the difference algebraic operation is performed on the digital surface models of the two phases of the murals to obtain the elevation difference data. The gradient-based global critical value calculation is performed on the difference image of the V component, and the Gaussian-Laplace-based global critical value calculation is performed on the difference image of the S component to obtain the preliminary data for change detection of the S and V components. Finally, the elevation difference data of the digital surface model is combined to comprehensively judge the category of the changed disease; the result map of the disease change detection is output. Fig. 9 The flowchart of change detection.

[0185] Step 1: After obtaining the orthophotos and digital surface models of the two phases of the murals in the collection, the orthophotos of the second phase are used to show the diseases of fading, discoloration, cracks and hollowing, and a mixed noise of Gaussian noise and salt and pepper noise is added; Step 2: The two phases of orthophotos are converted from RGB color space to HSV color space to obtain the grayscale data of the three components H, S, and V. The S saturation component and the V brightness component are used to extract the change information. The S saturation component is sensitive to discoloration, and the performance of discoloration and cracks in the orthophoto is detected; the V component is a measure of the strength of color brightness, and the color brightness reduction caused by fading in the orthophoto is detected; Step Step 3: Reduce the impact of single pixel errors. Use a 7*7 mean pixel window to perform difference operations on the two-phase orthophotos and the digital surface model to obtain the absolute value of the difference. Apply the Gaussian-Laplacian operator to the S saturation component to ignore the types of fading diseases in the area with slowly changing grayscale levels, leaving only the areas with drastic color changes caused by discoloration and cracks. Apply the gradient operator to the V brightness component to detect the fading with slowly changing grayscale levels and the areas with drastic grayscale changes. Combining the above two judgments, the fading area is circled. For discoloration, cracks and hollows, the elevation difference data of the digital surface model is combined for collaborative analysis and judgment.

[0186] The critical value of the orthophoto difference image is set using percentage statistics. The S component image is subjected to Gaussian-Laplace transformation to obtain the absolute value of the Gaussian-Laplace image. The critical value Ts is calculated for the histogram of this image. If it is greater than Ts, its grayscale value is set to 255, otherwise the grayscale value is 0, and the change area on the S component is extracted; the V component image is subjected to gradient transformation to obtain a gradient amplitude image. The critical value Tv is calculated for the histogram of this image to obtain the critical value Tv. If it is greater than Tv, its grayscale value is set to 255, otherwise the grayscale value is 0, and the change area on the V component is extracted;

[0187] For the digital surface models of the two phases of murals, images in tif format are used. The image amplitude records the elevation value of each geographical location, represented by 32-bit type data. The image uses black and white grayscale levels to represent the degree of ups and downs on the image. The whiter the area, the smaller the elevation, and the darker the area, the larger the elevation. The entire area is in the coordinate system where the murals are located. The empirical value of 0.001m is used as the critical value for the difference operation of the two phases of digital surface model images. Based on this critical value, the changed area on the digital surface model is extracted.

[0188] The change areas extracted from the orthophoto image and the change areas extracted from the digital surface model are combined to identify the types of mural disease. First, for the area that has changed on the digital surface model, if there is a corresponding grayscale change in the V component change image relative to this area, then the crack disease change is screened out; if there is no corresponding grayscale change in the orthophoto image relative to this area in this area, then the hollowing disease change is screened out. Then, for the condition that there is no change in the area on the digital surface model, the analysis is performed. In this area, when the V component change image has a corresponding grayscale change in the corresponding area subset, and at the same time, when the S component change image has a corresponding grayscale change in the corresponding area subset in this area, then the discoloration disease change is screened out; if in this area, the V component change image has a corresponding grayscale change in the corresponding area subset, but in this area, the S component change image has no corresponding grayscale change in the corresponding area subset, then the fading disease change is screened out, and the four mural disease changes of fading, discoloration, cracks and hollowing are detected. The specific analysis ideas are as follows Fig.10 Output the detection results to the first original mural image, and you can see the location and area of ​​the disease more clearly, and use different colors to represent different disease categories.

[0189] IV. Example Presentation and Analysis

[0190] Taking the environment of the Mural Gallery of Shaanxi History Museum as the research condition and the murals in the gallery as the experimental objects, this paper conducts a case study from mural data collection, digital calculation, color consistency implementation, digital results generation to change analysis and detection.

[0191] 1. Digitalization of murals

[0192] (1) Photography of murals: The murals in Shaanxi History Museum are all displayed in display cases. The display cases are arranged along the wall, with transparent glass on the front. Usually, only the top of the display case and a small number of roof lights are turned on for appreciation. The murals in the display cases are inclined at an angle of about 5 degrees to the vertical wall. The preliminary shooting experiment of the murals was carried out. The shooting equipment used was a rail mobile bracket, which can move horizontally and vertically. There are scales on the rails, and the movement accuracy is 1mm. The bracket can carry a camera and two light sources, which are located on both sides of the camera. The murals are also placed at an angle, with the bottom of the mural tilted forward, so that the murals are about 5 degrees from the vertical direction. A Nikon D800 camera with a 50mm standard lens is used. The camera has a maximum resolution of 7360*4912 and a shooting distance of 55cm. The spatial resolution of each pixel reaches 0.04mm (600DPI); the light source is a photography lamp with a built-in bulb. The entire shooting process involves manually moving the guide rails with the overlap set to 80%. It takes about 20 minutes to shoot the entire mural, but the preparation time is a full 2 ​​hours, including the deployment of the equipment, the preset of the instrument, the measurement of the shooting distance, the calculation of related parameters, etc.

[0193] The above is a preliminary design example. In order to improve the efficiency and accuracy of mural shooting, the entire mural shooting and detection system will be improved later. A Canon camera (Canon 5DS) was selected, and three lenses with different focal lengths of 50mm, 70mm and 85mm were configured to meet the image shooting requirements of different mural placement situations on site. The reserved space between the top of the mural and the roof is relatively narrow, not exceeding 50cm, so two LED lights with adjustable color temperature and brightness of the LEP-1496C model were selected; the shelf of the mobile guide rail was automatically improved, and the guide rail was automatically controlled to move in the horizontal and vertical directions according to the input parameter value, and the overall rotation was automatically controlled according to the input angle parameter to keep the photography stand level with the mural plane; the relative relationship between the camera and the light source can be changed, and once determined, the relative position relationship is fixed. Through the above improvements, the demand for automated shooting is realized to improve the efficiency and accuracy of shooting. The distance of shooting and the selection of elevation accuracy are contradictory. The selection principle of this mural shooting is to obtain a higher elevation accuracy while ensuring the safety of mural operations.

[0194] (2) Control point layout and accuracy assessment: In the initial stage, the control point measurement experiment was carried out using a Sokkia total station. The angle measurement accuracy was nominally 0.5 seconds, and the nominal accuracy of the non-prism distance measurement was (1+1ppmxD)mm. Since the distance between the mural and the total station was about 2m, the initial measurement accuracy of the control points was 1.002mm. In order to improve the measurement accuracy of the control points, coded control points were used at the same time, each of which was 600mm long, 60mm wide, and 5mm thick. The coded control points were arranged around the mural, and according to the requirements for point record production, point record files for 10 control points were produced on the panoramic map. The Sokkia total station was used to measure all the selected 10 control points twice. In these two measurements, the total station origin was placed at different positions. Measure 10 control points. The data of each measurement was converted from the coordinate system with the total station as the origin to the mural coordinate system, and then the data results of the two measurements in the mural coordinate system were compared. If the error of the two measurement results is within 1mm, the measurement is qualified, otherwise it is re-measured. like Fig.11 The comparison results in the mural coordinate system are illustrated, and the errors are all within 1 mm.

[0195] (II) Display of mural digitization results: 19 copied murals were photographed and calculated. The following are digital orthophotos and digital surface model examples of two of the murals.

[0196] (III) Display of the results of the detection of micro-changes in murals: Fig.12This is an extraction diagram of the results of mural disease change detection. The extraction diagram shows the specific location and area of ​​the disease on the mural. Different colors are used on the original image to indicate the specific location and area of ​​the disease, and also indicate the type of disease. The embodiment uses the environment in the mural gallery of Shaanxi History Museum as the condition and the murals in the gallery as the experimental objects. An example study was conducted on the murals in the collection from data collection, digital calculation, color management, generation of digital results, to change analysis and detection, and the digitization results and change detection results of the murals in the collection were displayed. In the process of data collection for the murals, initial experiments were conducted based on the design of the shooting equipment system. Insufficient results were found during the experiments and further improvements were made later. After comparing the two methods of measuring control points with coded control points using a traditional total station, it was found that the adjustment of coded control points was more accurate and the process from data acquisition to data calculation was more efficient than the traditional method. In the process of color consistency experiments, the entire process from on-site color calibration of the display to uniformity measurement of LED light sources, to uniformity control of light within the frame, camera white balance setting, color mark shooting, and color space control between later image calculation software was experimented with. The color difference between the shooting value and the standard value was statistically analyzed, and the color effect after color calibration was better than that before color calibration. In the process of change detection, the conversion from RGB to HSV color space was used to perform gradient and Gaussian-Laplace transforms on the difference image. The critical value of the transformed histogram image was calculated using percentages. The information of the S and V components and the elevation difference of the digital surface model was comprehensively analyzed. The obtained critical value was used to obtain the location and area of ​​the disease change, and the disease category was identified to generate a disease change detection extraction map.

Claims

1. A method for three-dimensional digital protection and micro-change detection of cultural relic murals, characterized in that: Including system solutions for digitizing museum murals, color management for mural color consistency, and methods for detecting micro-changes in murals; 1) Three-dimensional digitization of murals, including: mural shooting detection system, control point layout, and generation of digital results. Based on the special environment of the collection murals, an equipment system for the collection mural shooting is constructed, and a collection mural shooting plan is established, including camera and lens selection, illumination uniformity and image overlap control, photography distance and photography bracket. In terms of control point layout, a total station is used to measure the coordinates of the control points, and coded control points are added. The adjustment accuracy of the murals is controlled by identifying and solving the coded control points contained in the image; after obtaining the sequence images and control points of the collection murals, two digital results, digital orthophotos DOM and digital surface models DSM, are generated for the collection murals; 2) Color consistency method for mural images, including: light source selection and layout, color mark selection and layout, camera white balance setting, illumination uniformity control, camera ICC file preparation, image chromatic aberration correction, using flat color management method, selecting a wide color gamut display with stable chromaticity and brightness, and strict on-site correction and color gamut comparison; selecting a suitable light source and deploying it according to on-site conditions; using a color temperature meter and illuminance meter to strictly control illumination uniformity; using a gray card to preset the camera's white balance; selecting an SG color mark that covers more mural colors and preparing the camera's ICC file; embedding the camera's ICC file in the image sequence to complete image correction; maintaining consistency in color space when transferring images between software; finally, comparing the color difference between the standard color block value and the correction value, completing the color consistency management of the mural image from acquisition to generation and then to display; 3) Analysis and detection of mural micro-changes: The first is mural image registration, including plane fitting and coordinate transformation, pixel-level image registration; the second is mural disease analysis, including mural disease display, edge improvement global critical value calculation, and mural multi-data joint analysis. Based on the remote sensing image change detection method, the two phases of murals in the collection are subjected to change detection of four diseases: fading, discoloration, cracks and hollowing. The two phases of murals are converted from RGB color space to HSV color space, and Laplace and gradient transformations are performed on the differential operation results of the S and V components respectively. The critical value of the transformation result is calculated as a percentage, and this critical value is used as the limit. Combined with the elevation difference information of the two phases of digital surface models, the location and area of ​​the four diseases are comprehensively judged, and an extraction map of the disease change detection results is generated.

2. The method for three-dimensional digital protection and micro-change detection of cultural relic murals according to claim 1 is characterized in that: Mural shooting and detection system: equipped with a high-resolution camera and a light source with adjustable color temperature. The camera and light source move synchronously on the cradle and rails to ensure a balanced light environment for shooting. The rails are engraved with line scales and angle values, and can be moved left and right, up and down, and tilted. The software can be used to accurately control the distance movement and angle tilt. (1) Overlap design: The camera needs to take multiple shots and then stitch them together. The overlap within and between flight strips is set to ensure that the image overlap accuracy is the same within the entire mural frame and that every part of the entire mural has the same shooting accuracy. The overlap is 80% in the heading and lateral directions. In the images at the starting and ending shooting positions, the mural content in the horizontal and vertical directions does not exceed 20% of the image. (2) Light uniformity control: Using directional geometry, place two light sources on the left and right sides of the camera at a 45-degree angle to the camera. (3) Photography support design: The distance the guide rail moves is controlled by setting parameters in the software. The horizontal baseline length is the distance the guide rail moves horizontally, and the vertical baseline length is the distance the guide rail moves vertically. Every time the guide rail moves one baseline length, the camera automatically exposes and completes the shooting without other instructions. The length and width of the mural are determined in advance, and the program automatically calculates the number of photos required in the horizontal direction and the number of times to move in the vertical direction, and automatically controls the guide rail to move horizontally and vertically to complete the shooting of the entire mural. (4) Design of photographic distance: Before taking an image, use a single-point laser rangefinder to measure the photographic distance when the camera is aimed at the four corners of the mural. Adjust the front, back, left, right, and tilt angle of the photographic bracket so that the photographic distance values ​​of the camera when photographing the four corners of the mural are consistent. The errors of the four photographic distance values ​​must be within the range required for photographic shooting. When measuring the photographic distance from the camera to the mural, find the photographic center of the camera in advance, that is, the perspective geometric center of the camera. The method is as follows: Step 1: Hold the camera and point it at a calibration plate, and take 5-10 images at different positions. Step 2: Place the camera on a fixed tripod and take an image facing the calibration plate. Step 3: Use the camera calibration algorithm to solve the internal and external orientation elements of the camera, especially the orientation elements of the last photo, where the Z position Zs is the distance from the center of the photo to the calibration plate; Step 4: Use a tape measure to measure the distance from Zs to the camera's photographic center.

3. The method for three-dimensional digital protection and micro-change detection of cultural relic murals according to claim 1 is characterized in that: Non-coded control point layout: The control points are directly measured by the total station, and the specific positions and numbers of the control points are manually specified. The control points are evenly distributed over the entire mural, with one control point arranged for every four baseline lengths. Rules for selecting control points: 1) There must be control points near the four corners of the mural, and the control points inside the mural must be evenly distributed; 2) The control points must be located in the pigment layer of the mural, and it is not allowed to select points on the ground layer or support layer of the mural; 3) It is not allowed to select control points at the location of hollowing or collapse; 3) The control points must be easily identifiable corner points, which can be clearly identified on the mural image and in the total station eyepiece; A point record file is created for the selected control points. The point record file records the specific position of each control point on the mural sheet, providing a reference for surveyors when measuring the second phase of murals, so that there is a basis for explanation when using control point data for data calculation; The point record file is made on the panoramic view of the mural. After the original image of the mural is taken, the panorama is quickly generated using photogrammetry software. The panorama marks the approximate position of the control point. The precise position of the control point needs to be marked on the single original image of the mural. The point record is made into a three-level display. The panoramic mark is the first level, the second level is to mark the position of a certain control point on the original image, and the third level is the mark when the position of the control point of the second level is enlarged to the pixel level. The three-level record of the point record file clearly records the orientation characteristics of the control point, which is convenient for identification; After the point record file is prepared, the total station is used to measure the three-dimensional coordinates of the control points. The total station is set up in front of the mural, and all the control points recorded in the point record file are observed in turn to obtain the three-dimensional coordinates of the control points in the current station coordinate system. A new coordinate system is established based on the mural. The rules include: 1) All the control point coordinates are fitted into a plane, and the normal direction of the plane points from the mural surface to the total station setting direction as the positive direction of the Z axis of the new coordinate system; 2) Establish a right-handed coordinate system with the horizontal line in the mural plane as the positive direction of the X axis and the right-hand directions of the Z axis and the X axis as the positive direction of the Y axis; 3) The origin of the coordinate system is located in the mural plane fitted by all control points, close to the lower left corner of the mural.

4. The method for three-dimensional digital protection and micro-change detection of cultural relic murals according to claim 1 is characterized in that: Generate digital results: orthophotos and digital surface models of the murals. The orthophotos represent the plane features of the murals, and the digital surface models reflect the ups and downs of the mural surface, providing data for the analysis and detection of mural changes. Step 1: The color-calibrated mural image is subjected to free network joint adjustment to obtain a sparse three-dimensional point cloud, in which the point cloud is a relative coordinate. Step 2: Convert the coordinate system of the control point measured by the total station to the coordinate system of the mural with the total station as the origin, the horizontal line in the mural plane as the X-axis, and the Z-axis and the right-hand direction of the X-axis as the positive direction of the Y-axis. Step 3: Import the control points after coordinate system conversion into the photogrammetry software, and correspond each control point one by one to the corresponding position on the mural image, so that the sparse three-dimensional point cloud constructed by the free network is unified to the mural coordinate system; detect the coded identification points on the coded identification stick, and make the detected adjacent coded identification points into one coded identification stick, and use multiple coded identification sticks to optimize the free network adjustment; Step 4: Densely match the sparse point cloud to obtain a dense 3D point cloud; Step 5: Perform triangulation calculation on the obtained dense 3D point cloud, and perform manual editing and repair; Step 6: Export the digital surface model in obj and tif formats in the photogrammetry software; Step 7: Perform digital differential correction in the photogrammetry software and output the digital orthophoto in tif format.

5. The method for three-dimensional digital protection and micro-change detection of cultural relic murals according to claim 1 is characterized in that: Mural image color consistency method: (1) Light source selection and layout: The color rendering index of the light source cannot be lower than 90. The color temperature of 5500K is selected as the color temperature when shooting the mural. The light source is selected to adjust the color temperature within the range of 2000K to 6500K. The brightness is adjusted to adapt to the dark environment of the mural gallery. The light source's own illumination uniformity is not less than 90%, the color temperature difference is not greater than 200K, the illumination difference between light sources is less than 10%, and the color temperature difference is less than 200K. (2) Color mark selection and placement: Use a professional digital photography SG color mark, 27.94 cm long and 21.59 cm wide, and place it close to an area with no strong reflections and a single color in the entire mural, so that the color mark is minimally affected by the reflected light from the surrounding mural surface content and the color mark is minimally deformed within the camera frame. At the same time, use the pure white and pure black blocks around the color mark to detect whether the lighting is uniform; (3) Camera white balance setting: Use the preset white balance and select an 18% gray card for preset white balance. First, place the 18% gray card close to the front of the mural in the same way as the digital SG color standard. Aim the digital camera at the gray card for evaluative metering to set the aperture, shutter speed, and sensitivity. Use manual focus. Take a picture of the gray card with the camera. This image is used as the preset white balance image. The camera then uses the ambient color temperature value of the gray card image as the color temperature reference value under the current shooting conditions. (4) Illumination uniformity control: Directional geometric conditions are selected, and two light sources with controllable color temperature and brightness are placed on the left and right sides of the camera at a 45-degree angle to the camera. The illumination uniformity of each mural image is quantitatively controlled by using a light meter to measure the four corners, edges, and middle parts within the actual camera frame. The measured illuminance values ​​of 13 positions, including 4 at the four corners, 4 at the four edges, 1 at the middle, and 4 at the interior, are obtained. If the maximum and minimum differences of these 13 illuminances are within 1000 illuminances and the differences between them are within 500 illuminances, the illumination is considered uniform. If the error range is exceeded, the direction and distance of the light are fine-tuned and the measurement is repeated until the measured illuminance value is within the error range. At this time, the positions of the two light sources are fixed and the entire mural is photographed with the light source positions arranged accordingly.

6. The method for three-dimensional digital protection and micro-change detection of cultural relic murals according to claim 1 is characterized in that: Plane fitting and coordinate conversion: First, the coordinate system with the total station as the origin is converted to the mural coordinate system with the lower left corner of the mural as the origin, the horizontal line in the mural plane as the X-axis, and the Z-axis and the right-hand direction of the X-axis as the positive direction of the Y-axis, to achieve rough alignment; The orthophoto images of different periods are unified under a coordinate reference, and some characteristic points that are not easy to change and easy to identify are uniformly selected around and inside the mural to be photographed as the control points of the reference, and the point record files of these control points are prepared. Using these control points, a plane is fitted by the least square method. The direction of the normal vector of the plane from the mural to the viewing point is the positive direction of the Z axis in the mural coordinate system; the horizontal projection of the line connecting the control points at the lower left corner and the lower right corner of the mural on the fitting plane is the X axis of the mural coordinate system, and the direction from the lower left corner point to the lower right corner point is the positive direction of the X axis; the right-hand direction of the Z axis and the X axis in the mural coordinate system is the positive direction of the Y axis; When collecting mural data in the later stage, the point record made in the first phase of mural data collection was followed. Each control point recorded in the point record file of each mural was measured one by one. However, when the points in the point record file changed slightly, these changed points were no longer measured, and the remaining control points that did not change were measured repeatedly. A plane was fitted using these repeatedly measured control points, and the coordinates of the measured control points were converted to the mural coordinate system. The orthophotos and digital surface models of murals from different periods were established in a coordinate system based on the same control points. The orthophotos and corresponding digital surface models were in the same coordinate system, and the orthophotos and digital surface models from different periods were also in the same coordinate system.

7. The method for three-dimensional digital protection and micro-change detection of cultural relic murals according to claim 1 is characterized in that: Edge improvement global critical value calculation: For the selection of critical value, based on the Otsu critical value, the grayscale difference less than the critical value Q is set to 0, that is, black, and the grayscale difference greater than the critical value Q is set to 255, that is, white, to distinguish the changed area and position from the background, as shown in the following formula: Otsu performs calculations entirely on the histogram of an image. In an M×N digital image, the grayscale L of pixels of different levels is represented by [0, 1, 2, …, L-1], and the number of pixels of grayscale level i is n. i , the total number of pixels MN = n0 + n1 + n2 + ... + n L-1 , the normalized histogram has P i =n i / MN, we get: Assume that a critical value T(k)=k is selected, 0<k<L-1, and the critical value of the input image is calculated into two categories C1 and C2, where C1 consists of all pixels in the image whose grayscale values ​​are in the range [0, k], and C2 is the sum of pixels in the image whose grayscale values ​​are in the range [k+1, L-1]. Based on this critical value, the probability P1(k) of a pixel being classified into C1 is: The probability of class C2 occurring is: The average grayscale value of pixels assigned to class C1 is: The average grayscale value of pixels assigned to class C2 is: The cumulative mean up to level k is: The average grayscale of the entire image: Use the normalized dimensionless matrix to evaluate the quality of the critical value at level k: is the global variance, i.e. the grayscale variance of all pixels in the image, is the between-class variance, and k is introduced, and finally we get: The optimal critical value is k * , which maximizes if The maximum value of corresponds to multiple k values. The largest k values ​​are averaged. Once k is obtained * , input image for segmentation: Gaussian noise and salt and pepper noise are added, and edge improvement is used to calculate the global critical value based on the Otsu method. The valleys between the histogram peaks are deepened by using pixels that meet the gradient and Laplacian operator metrics.

8. The method for three-dimensional digital protection and micro-change detection of cultural relic murals according to claim 7 is characterized in that: The global critical value calculation is improved by using gradient-based edge data: the first-order derivative of the image is calculated by the gradient amplitude. For the function f(x, y), the gradient at the coordinate (x, y) is defined as a two-dimensional column vector, which indicates the direction of the maximum rate of change of f at the position (x, y); The magnitude of the vector ▽f is expressed as M(x,y), that is: The above formula can be approximated by the sum of absolute values, namely: M(x,y)≈|g x |+|g y Formula 14 Construct edge detection gradient operator: S(x,y)=|f(i-1,j-1)+2f(i-1,j)+f(i-1,j+1)-[f(i+1,j-1)+2f(i+1,j)+f(i+1,j+1 )]|+|f(i-1,j-1)+2f(i,j-1)+f(i+1,j-1)-[f(i-1,j+1)+2f(i,j+1)+f(i+1,j+1)]| That is, the weighted difference operator is as follows: In the calculated gradient image, the edges of the areas where the grayscale levels of the original image change slowly will not be ignored.

9. The method for three-dimensional digital protection and micro-change detection of cultural relic murals according to claim 7 is characterized in that: Improved global critical value calculation based on edge data of Laplace operator: The second-order differential of the image is defined as follows: The Laplace operator of the two-dimensional image function f(x,y) is defined as: The Laplace transform is a linear operator. Using the definition of Equation 16, we get: In the y direction we get: The discrete Laplacian of two variables is: When extracting the edge, the Gaussian function is used for low-pass filtering, and then the Laplace operator is used for high-pass filtering. At the same time, the zero point is obtained to form a disease detection operator. The Gaussian function is: The low-pass filtering result is: f(x, y)*g(x, y) Formula 22 Then the Laplace operator is used to calculate: get: After taking the second-order derivative, the maximum point is 0, and the area with a value of 0 is the boundary.

10. The method for three-dimensional digital protection and micro-change detection of cultural relic murals according to claim 1 is characterized in that: Combined analysis of multiple mural data: Analyze the fading, discoloration, cracks and hollowing diseases of the murals in the collection. First, convert the orthophoto images of the murals containing simulated noise and diseases from RGB color space to HSV color space to obtain the grayscale or intensity information of the three components H, S and V. Secondly, perform difference algebraic operations on the S and V components of the orthophoto images of the two phases of the murals to obtain the difference data of the S and V components, and perform difference algebraic operations on the digital surface models of the two phases of the murals to obtain the elevation difference data. Perform a global critical value calculation based on the gradient on the difference image of the V component, and perform a global critical value calculation based on Gaussian-Laplace on the difference image of the S component to obtain preliminary data for change detection of the two components S and V. Finally, combine the elevation difference data of the digital surface model to comprehensively judge the category of the changed disease. Step 1: After obtaining the orthophotos and digital surface models of the two phases of the murals in the collection, the orthophotos of the second phase are subjected to the damage display of fading, discoloration, cracks and hollows, and a mixed noise of Gaussian noise and salt and pepper noise is added; Step 2: The two phases of orthophoto images are converted from RGB color space to HSV color space to obtain grayscale data of the three components H, S, and V. The S saturation component and the V brightness component are used to extract the change information. The S saturation component is sensitive to discoloration, and the discoloration and crack disease on the orthophoto image are detected; The V component is a measure of the intensity of color brightness, and detects the reduction in color brightness caused by fading in the orthophoto image; Step 3: Reduce the influence of single pixel errors. Use a 7*7 mean pixel window to perform difference operations on the two-phase orthophotos and the digital surface model to obtain the absolute value of the difference. Use the Gaussian-Laplacian operator on the S saturation component to ignore the types of fading diseases in the area where the grayscale changes slowly, leaving only the areas with drastic color changes caused by discoloration and cracks. Use the gradient operator on the V brightness component to detect the fading with slow grayscale changes and the areas with drastic grayscale changes. Combine the above two judgments to circle the fading area. For discoloration, cracks and hollows, the digital surface model's elevation difference data is combined for collaborative analysis and judgment. Combine the changed areas extracted from the orthophoto image and the changed areas extracted from the digital surface model to identify the types of mural disease. First, for the changed areas on the digital surface model, if there is a corresponding grayscale change in the V component change image relative to this area, the crack disease change is screened out; if there is no corresponding grayscale change in the orthophoto image relative to this area, the hollowing disease change is screened out. Then, analyze the condition that there is no change in the digital surface model. In this area, when the V component change image has a corresponding grayscale change in the corresponding area subset, and at the same time, when the S component change image has a corresponding grayscale change in the corresponding area subset, the color change disease change is screened out. If in this area, when the V component change image has a corresponding grayscale change in the corresponding area subset, but in this area, when the S component change image has no corresponding grayscale change in the corresponding area subset, then the fading disease changes are screened out, and the four types of mural disease changes, fading, discoloration, cracks and hollows, are detected. The detection results are output to the first original mural image, so that the location and area of ​​the disease can be seen more clearly, and different disease categories are represented by different colors.

Images