A method for identifying diseases of stone cultural relics based on hyperspectral images

Abstract

The application discloses a stone cultural relic disease identification method based on hyperspectral images, and aims to solve the problems of relying on manual experience and consuming time and effort in stone cultural relic disease investigation. The steps are as follows: collecting hyperspectral images of stone cultural relics to be investigated and performing pretreatment; extracting spectral features; selecting central bands of three bands of blue light, green light and red light to construct true color images; extracting texture features and color features of the true color images; fusing the spectral features, the texture features and the color features to obtain combined features; establishing a typical disease feature database; training a disease classification model by using a random forest, a support vector machine, a gradient boosting machine and a convolutional neural network; identifying typical diseases by using an intelligent identification model and drawing a disease cloud chart; and calculating a typical disease area ratio of a to-be-measured region to perform disease evaluation. The application realizes in-situ, non-destructive and rapid identification of stone cultural relic diseases, saves manpower and resources, and provides an intelligent identification method for cultural relic protection investigation.

Description

TECHNICAL FIELD

[0001] The embodiment of the present application relates to the field of rock and soil quality cultural relic protection, and particularly relates to a stone quality cultural relic disease identification and classification method based on hyperspectrum. BACKGROUND

[0002] Similar grottoes and other immovable stone quality heritage will suffer serious weathering due to long-term exposure to the natural environment, often accompanied by peeling, salt crystallization, crust, and the appearance of biological colonization, thereby causing the destruction of the structural integrity and value of stone quality cultural relics. Therefore, the research on stone quality cultural relic disease identification is the first important step for targeted protection of stone quality cultural relics.

[0003] However, the current rock quality cultural relic disease identification method is that the staff members shoot optical photos of the to-be-detected area, and then the affected area is delimited by using statistical software, which completely depends on the professional level of the staff members, has a high learning cost, and is not accurate enough in identification. Therefore, a fast, accurate, and low-cost stone quality cultural relic disaster identification method is urgently needed. SUMMARY

[0004] In view of the above problems, the embodiment of the present application provides a stone quality cultural relic disease identification and classification method based on hyperspectrum, which is used to solve the problems of low efficiency, inaccuracy, and high cost of traditional methods in stone quality cultural relic disaster identification.

[0005] According to an aspect of the embodiment of the present application, a stone quality cultural relic disease identification method based on hyperspectrum is provided, as shown in the accompanying drawings. Figure 1 The method comprises the following steps:

[0006] Step 1: collecting hyperspectrum images of the to-be-detected stone quality cultural relics and performing pretreatment;

[0007] Step 2: extracting spectral features of the pretreated hyperspectrum images;

[0008] Step 3: constructing a true color image by selecting the central wavebands of the blue, green, and red wavebands of the pretreated hyperspectrum images;

[0009] Step 4: extracting texture features and color features of the true color image;

[0010] Step 5: fusing the spectral features, texture features, and color features of the pretreated hyperspectrum images by using a weighted combination method to obtain combined features;

[0011] Step 6: labeling the features of four different disease types of peeling, salt crystallization, biological colonization, and crust with sample labels, and establishing a typical disease feature database;

[0012] Step 7: Train the disease classification model using random forest, support vector machine, gradient boosting machine and convolutional neural network with the combined feature vector as input;

[0013] Step 8: Identify typical diseases using the intelligent recognition model and draw disease cloud maps.

[0014] Step 9: Perform disease assessment by calculating the area ratio of typical diseases in the test area.

[0015] In an optional manner, the high spectral data of the stone cultural relics to be detected in step 1 is collected by a hyperspectral imager. The hyperspectral imager has the advantages of strong spectral information continuity, chemical component recognition ability, high spatial resolution, non-contact imaging, etc.

[0016] In an optional manner, the preprocessing work in step 1 includes processing the hyperspectral image with a Savitzky-Golay smoothing filter and normalization processing. The Savitzky-Golay smoothing filter has the advantages of effectively suppressing noise, retaining image features, strong adaptability to image processing requirements, and simple and efficient algorithm.

[0017] In an optional manner, in step 2, the positions and numbers of absorption peaks on the spectral curve in the wavelength range of 400 nm-2500 nm are analyzed as the spectral characteristics of the disease, and the disease types are distinguished based thereon.

[0018] In an optional manner, the spectral curve absorption characteristics are that there are multiple obvious absorption peaks near 700-1000 nm and 1400 nm for peeling and salt crystallization; the spectral curve of biological colonization and crust has an absorption peak near 550 nm (nitrogen absorption band) and a significant rising stage at 700 nm-750 nm (light scattering inside leaf cells); the spectral reflectance of salt crystallization is higher than that of other disaster types.

[0019] In an optional manner, in step 3 or step 4, the hyperspectral image of the stone cultural relics to be detected is selected to construct true color images at three wavebands of blue, green and red centers, i.e. 475 nm, 520 nm and 650 nm, for extracting texture features and color features.

[0020] In an optional manner, in step 3 or step 4, the light at wavelengths of 475 nm, 520 nm and 650 nm has high resolution and contrast in the spectral image, so that the color details in the image can be more accurately reflected when extracting features.

[0021] In an optional manner, in step 4, the color features are obtained by converting the RGB color model to the HSV color model, and the principle of color space conversion is:

[0022]

[0023]

[0024]

[0025]

[0026]

[0027]

[0028] The formula for calculating hue H is:

[0029]

[0030] The formula for calculating saturation S is:

[0031]

[0032] The formula for calculating luminance V is:

[0033]

[0034] In an alternative approach, the texture extraction based on the gray co-occurrence matrix (GLCM) in step 4 is obtained by calculating the joint conditional probability density between different pixel intensities in the image. It provides four texture features: energy (Ene), contrast (Con), uniformity (Hom), and entropy (Ent).

[0035] Energy: Reflects the grayscale concentration and texture thickness of an image. The thicker the texture, the greater the energy.

[0036]

[0037] in These are the element values ​​in the constructed gray-level co-occurrence matrix, where i and j represent the gray levels used as indices.

[0038] Contrast: Used to reflect the sharpness of an image, and also the depth of texture grooves.

[0039]

[0040] in This represents the difference between the grayscale values ​​of two pixels.

[0041] Uniformity: Reflects the texture similarity of an image. The higher the value, the more similar the local areas are, and the smaller the grayscale difference.

[0042]

[0043] Entropy: Represents the randomness of image content, but also reflects the amount of information and complexity of the image. The more complex the texture, the higher the entropy value.

[0044]

[0045] In an alternative approach, step 5 employs a weighted combination method for fusion. It is assumed that the spectral features of the hyperspectral image are... The color characteristic is Texture features are Then the combined feature a can be represented as follows:

[0046]

[0047] in, .

[0048] In one alternative approach, the disease database established in step 6 has the following specific structure, content, and technical advantages:

[0049] Database Structure: A hierarchical structure is adopted, consisting of a main directory layer for damage types, a secondary directory layer categorized by individual cultural relics, and a subdirectory layer detailing the specific damage to different parts of each cultural relic. This hierarchical structure facilitates the classification, management, and rapid retrieval of damage data for different types of stone cultural relics and their different parts.

[0050] The database contains spectral, color, and texture data of various typical defects in stone cultural relics, along with detailed descriptions of the corresponding defects (including causes and stages of development). It also records statistical information such as the frequency of occurrence of each defect in different cultural relic cases. This rich data provides comprehensive support for subsequent accurate defect classification and analysis.

[0051] Technical advantages: By integrating multi-dimensional disease characteristic data, compared with traditional single-feature databases, it can more accurately reflect the actual situation of disease, improve the accuracy and reliability of subsequent operations such as disease identification, classification and disease development trend prediction based on the database, and thus better serve the protection of stone cultural relics.

[0052] In an optional approach, step 7 involves training a disease classification model using multiple algorithms. The specific algorithm application steps and combinations are as follows:

[0053] The combined feature vectors are used as input, and the classifiers are trained using random forest, support vector machine, gradient boosting machine and convolutional neural network algorithms respectively.

[0054] Select samples that were not involved in the modeling process for classification testing, and then input the test samples into the trained classifier for classification prediction.

[0055] Model performance evaluation: The performance of the classification model is evaluated using global accuracy (GA) and kappa coefficient. GA and kappa coefficients are calculated using the confusion matrix. The calculation formulas are as follows:

[0056] Global precision (GA):

[0057]

[0058] Where n is the total number of columns in the confusion matrix (total number of categories); The number of samples in the i-th row and i-th column of the confusion matrix (the number of correctly classified samples); N is the total number of samples used for accuracy evaluation.

[0059] kappa coefficient:

[0060]

[0061] Through the above steps, disease classification models are constructed using four different algorithms, and the performance of these models is evaluated to select the best degradation identification model.

[0062] In one alternative approach, step 8 requires disease identification based on a trained model of the hyperspectral image, determining the regional range of different degradation types (stripping, salt crystallization, biological colonization, and crusting) in the image, that is, classifying and assigning values ​​to each pixel according to the original image size, and assigning different colors to different types of diseases to obtain a disease cloud map.

[0063] In an alternative approach, step 9 calculates the proportion of different colored pixels in the cloud image, multiplies it by the total area S of the region to obtain the typical damage area of ​​the stone artifact, and the calculation formula is as follows:

[0064]

[0065] Compared with existing technologies, the advantages and beneficial effects of this invention are as follows: by acquiring hyperspectral images of the area to be tested, extracting spectral features, texture features and color features, and fusing the spectral features, texture features and color features of the hyperspectral images into a combined feature, thereby establishing a disease feature database, training a disease classification model and conducting performance evaluation, and then classifying or identifying diseases, a lot of manpower and material resources used for disease exploration are saved, and it no longer relies on the experience of staff, and can achieve more accurate disease identification and statistics in micro-areas at a smaller scale.

[0066] The above description is merely an overview of the technical solutions of the embodiments of the present invention. In order to better understand the technical means of the embodiments of the present invention and to implement them in accordance with the contents of the specification, and to make the above and other objects, features and advantages of the embodiments of the present invention more apparent and understandable, specific embodiments of the present invention are described below. Attached Figure Description

[0067] The accompanying drawings are for illustrative purposes only and are not intended to limit the invention. Furthermore, the same reference numerals denote the same parts throughout the drawings. In the drawings:

[0068] Figure 1 A flowchart illustrating an embodiment of the present invention is shown.

[0069] Figure 2 The flowchart illustrating the specific process of establishing a disease identification model for stone cultural relics in an embodiment of the present invention is shown.

[0070] Figure 3 The diagram shows the identification results of the disease identification model in an embodiment of the present invention.

[0071] Table 1 shows the statistics of the area of ​​defects identified in stone carvings by the algorithm recognition model in the embodiments of the present invention. Detailed Implementation

[0072] Exemplary embodiments of the invention will now be described in more detail with reference to the accompanying drawings. While exemplary embodiments of the invention are shown in the drawings, it should be understood that the invention can be implemented in various forms and should not be limited to the embodiments set forth herein.

[0073] A method for identifying defects in stone cultural relics based on hyperspectral images, as shown in the attached figure. Figure 1 and attached Figure 2 As shown, it includes the following steps:

[0074] Step 1: Acquire hyperspectral images of the stone artifacts to be investigated and perform preprocessing;

[0075] In this step, a hyperspectral imager with a wavelength range of 400 nm to 2500 nm was used to acquire hyperspectral data of the area to be tested on the stone artifact. The spectral resolution was 5 nm, and a total of 420 bands were obtained through scanning in the hyperspectral image.

[0076] The preprocessing steps in this step include processing the hyperspectral image with a Savitzky-Golay smoothing filter and normalization.

[0077] Step 2: Extract the spectral features of the preprocessed hyperspectral image;

[0078] In this step, the position and number of absorption peaks on the spectral curves with wavelengths in the range of 400 nm to 2500 nm are analyzed as spectral characteristics of the disease, and the disease types are distinguished accordingly.

[0079] The absorption characteristics of the spectral curves are as follows: peeling and salt crystallization have multiple obvious absorption peaks near 700 nm-1000 nm and 1400 nm; the spectral curves of biological colonization and crusting have an absorption peak (nitrogen absorption band) near 550 nm, and a clear rising phase (light scattering inside leaf cells) at 700 nm-750 nm; the spectral reflectance of salt crystallization is higher than that of other disaster types.

[0080] Step 3: Select the center bands of the blue, green, and red light bands from the preprocessed hyperspectral image to construct a true-color image;

[0081] In this step, a true-color image is constructed by selecting three bands (475 nm, 520 nm, and 650 nm) at the center of blue, green, and red light in the hyperspectral image of the stone artifact in the area to be tested, and then extracting texture and color features.

[0082] Step 4: Extract texture and color features from the true-color image;

[0083] In this step, color features are obtained by converting the RGB color model to the HSV color model. The principle of color space conversion is:

[0084]

[0085]

[0086]

[0087]

[0088]

[0089]

[0090] The formula for calculating hue H is:

[0091]

[0092] The formula for calculating saturation S is:

[0093]

[0094] The formula for calculating luminance V is:

[0095]

[0096] In this step, texture extraction based on the Gray Co-occurrence Matrix (GLCM) is obtained by calculating the joint conditional probability density between different pixel intensities in the image. It provides four texture features: energy (Ene), contrast (Con), uniformity (Hom), and entropy (Ent).

[0097] Energy: Reflects the grayscale concentration and texture thickness of an image. The thicker the texture, the greater the energy.

[0098]

[0099] in These are the element values ​​in the constructed gray-level co-occurrence matrix, where i and j represent the gray levels used as indices.

[0100] Contrast: Used to reflect the sharpness of an image, and also the depth of texture grooves.

[0101]

[0102] in This represents the difference between the grayscale values ​​of two pixels.

[0103] Uniformity: Reflects the texture similarity of an image. The higher the value, the more similar the local areas are, and the smaller the grayscale difference.

[0104]

[0105] Entropy: Represents the randomness of image content, but also reflects the amount of information and complexity of the image. The more complex the texture, the higher the entropy value.

[0106]

[0107] Step 5: The spectral features, texture features, and color features of the preprocessed hyperspectral image are fused using a weighted combination method to obtain the combined features;

[0108] This step uses a weighted combination method for fusion. It is assumed that the spectral characteristics of the hyperspectral image are... The color characteristic is Texture features are Then the combined feature a can be represented as follows:

[0109]

[0110] in, .

[0111] Step 6: Label the characteristics of four different disease types—exfoliation, salt crystallization, biological colonization, and crusting—and establish a database of typical disease characteristics.

[0112] Step 7: Use the combined feature vectors as input to train the disease classification model using random forest, support vector machine, gradient boosting machine, and convolutional neural network;

[0113] In this step, multiple algorithms are used to train the disease classification model. The specific algorithm application steps and combination methods are as follows:

[0114] The combined feature vectors are used as input, and the classifiers are trained using random forest, support vector machine, gradient boosting machine and convolutional neural network algorithms respectively.

[0115] Select samples that were not involved in the modeling process for classification testing, and then input the test samples into the trained classifier for classification prediction.

[0116] Model performance evaluation: The performance of the classification model is evaluated using global accuracy (GA) and kappa coefficient. GA and kappa coefficients are calculated using the confusion matrix. The calculation formulas are as follows:

[0117] Global precision (GA):

[0118]

[0119] Where n is the total number of columns in the confusion matrix (total number of categories); The number of samples in the i-th row and i-th column of the confusion matrix (the number of correctly classified samples); N is the total number of samples used for accuracy evaluation.

[0120] kappa coefficient:

[0121]

[0122] in and are the total number of samples in the i-th row and the i-th column, respectively.

[0123] Through the above steps, disease classification models are constructed using four different algorithms, and the performance of these models is evaluated to select the best degradation identification model.

[0124] Step 8: Use the intelligent recognition model to identify typical diseases and draw disease cloud maps;

[0125] This step requires identifying the degradation of the hyperspectral image based on the trained model, determining the area range of different degradation types (stripping, salt crystallization, biological colonization, and crusting) in the image, that is, classifying and assigning values ​​to each pixel according to the original image size, and assigning different colors to different types of diseases to obtain a disease cloud map.

[0126] Step 9: Assess the disease by calculating the percentage of typical disease area in the area to be tested.

[0127] In this step, the proportion of different colored pixels in the cloud map is calculated, and multiplied by the total area S of the region to obtain the typical area of ​​damage to the stone artifact. The calculation formula is as follows:

[0128]

[0129] The following is an example of disease identification:

[0130] The Shakyamuni Buddha stone carvings at Dazu Rock Carvings are susceptible to various forms of damage due to long-term exposure to the natural environment, necessitating timely damage identification. To minimize errors and noise during spectral data acquisition, 10 spectral images were collected for each region while maintaining consistent settings. The final spectral data was obtained through arithmetic averaging.

[0131] A portable hyperspectral imager with a wavelength range of 400 nm to 2500 nm was used to acquire hyperspectral data of the target area. The spectral resolution was 5 nm, and a total of 420 bands of hyperspectral images were obtained through scanning. The hyperspectral images were processed using a Savitzky-Golay smoothing filter and normalization.

[0132] Then, the spectral characteristics of the hyperspectral image were analyzed. The hyperspectral image was then imported into Analysis software, and 475 nm, 520 nm, and 650 nm were selected as the centers of the red, green, and blue light bands, respectively, to export RGB images. These RGB images were then processed through a series of calculations to obtain HSV images for extracting color features from the hyperspectral image. Texture was described by calculating the joint conditional probability density between different pixel intensities in the image, thus reflecting the correlation between the intensity values ​​of any two points and the statistical properties of the texture features. When extracting texture features, the size and orientation of the moving window had little impact on the estimation. Therefore, a 5 × 5 window was selected in each period, and four texture features at a 45° angle were extracted for subsequent correlation analysis.

[0133] A weighted combination method is used to fuse the spectral features, texture features, and color features of the hyperspectral image into a single combined feature.

[0134] Based on the observed disease types, disease areas (ROIs) are delineated. This can be done using ENVI 5.6 software to establish a disease database.

[0135] Using combined features as input vectors, a disease classification model is trained using random forest, support vector machine, gradient boosting machine, and convolutional neural network. The samples are divided into training and test sets in a 7:3 ratio.

[0136] The performance of the four disease identification models varied depending on the type of disease. By evaluating the global accuracy (GA) and kappa coefficient, all four models demonstrated high accuracy (above 0.90) in identifying crusting and biocolonization, thus enabling them to identify typical diseases. Among them, the gradient booster model exhibited the best overall identification performance (GA: 0.97; Kappa: 0.97).

[0137] Disease classification and identification were performed on the test samples. The identification results are attached. Figure 3 The proportion of each type of disease in the tested area was calculated (area percentage), and the results are shown in Appendix 1. This allows experts to propose specific repair and maintenance plans in a timely manner, thereby ensuring the integrity and value of the stone cultural relics.

[0138] Numerous specific details are set forth in the specification provided herein. However, it will be understood that embodiments of the invention may be practiced without these specific details. Similarly, for the sake of brevity and to aid in understanding one or more aspects of the invention, in the description of exemplary embodiments of the invention above, various features of the embodiments are sometimes grouped together in a single embodiment, figure, or description thereof. The claims, which follow the detailed description, are hereby expressly incorporated into that detailed description, wherein each claim itself is a separate embodiment of the invention.

[0139] Those skilled in the art will understand that the modules in the device of the embodiment can be adaptively changed and placed in one or more devices different from that embodiment. Modules, units, or components in the embodiment can be combined into a single module, unit, or component, and further, they can be divided into multiple sub-modules, sub-units, or sub-components, except that at least some of such features and / or processes or units are mutually exclusive.

[0140] It should be noted that the above embodiments are illustrative of the invention and not restrictive, and that those skilled in the art can devise alternative embodiments without departing from the scope of the appended claims. In the claims, any reference signs placed between parentheses should not be construed as limiting the claims. The word "comprising" does not exclude the presence of elements or steps not listed in the claims. The word "a" or "an" preceding an element does not exclude the presence of a plurality of such elements. The invention can be implemented by means of hardware comprising several different elements and by means of a suitably programmed computer. In the unit claims enumerating several means, several of these means may be embodied by the same item of hardware. The use of the words first, second, and third, etc., does not indicate any order. These words can be interpreted as names. The steps in the above embodiments, unless otherwise specified, should not be construed as limiting the order of execution.

Claims

1. A method for identifying diseases of stone cultural relics based on hyperspectral images, characterized in that, The method comprises: Step 1: collecting a hyperspectral image of a stone cultural relic to be explored and performing pretreatment; Step 2: extracting spectral features of the pretreated hyperspectral image; Step 3: selecting the centers of three wavebands of blue light, green light and red light, i.e. 475 nm, 520 nm and 650 nm wavebands, of the pretreated hyperspectral image to construct a true color image; Step 4: converting the image from an RGB color space to an HSV color space, extracting color features under the HSV color model, specifically, counting proportions of pixel numbers in specific hue intervals in an H channel, proportions of pixel numbers in specific saturation intervals in an S channel and proportions of pixel numbers in specific brightness intervals in a V channel, and simultaneously obtaining texture features through a gray level co-occurrence matrix GLCM, calculating contrast, correlation, energy and entropy of the GLCM; Step 5: The spectral features, texture features and color features of the pretreated hyperspectral image are fused by a weighted combination method to obtain a combined feature; assuming that the spectral features of the hyperspectral image are , the color features are , and the texture features are , then the combined feature a can be expressed as follows: wherein, ; Step 6: labeling features of four different disease types of spalling, salt crystallization, biological colonization and encrustation with sample labels, and establishing a typical disease feature database; the disease database established in this way adopts a hierarchical structure, is divided into a disease type main directory layer, a two-level directory layer classified according to cultural relic individuals and a specific disease detail subdirectory layer for different parts of each cultural relic, covers spectral feature data, color feature data, texture feature data of various typical diseases of stone cultural relics and detailed description information of corresponding diseases, and further records frequencies of each disease in different cultural relic cases; Step 7: using random forest, support vector machine, gradient boosting machine and convolutional neural network to respectively train disease classification models with the combined feature vector as input, and evaluating performances of the models to select the best disease classification model; Step 8: performing disease recognition on the hyperspectral image based on the trained model, determining a range of the four typical diseases, i.e. spalling, salt crystallization, biological colonization and encrustation, in the image, i.e. performing classification and assignment according to each pixel point in the original image size, and obtaining a disease cloud image by assigning different colors to different types of diseases; Step 9: calculating proportions of pixel points of different colors in the cloud image, multiplying the proportions by a test area S to obtain an area of typical diseases of the stone cultural relic, and the calculation formula is: wherein, is the area ratio of flaking, salt crystallization, biological colonization or encrustation disease, S is the area of the test region, is the calculated area of flaking, salt crystallization, biological colonization or encrustation disease.

2. The method of claim 1, wherein, In the step 1, a hyperspectral imager is used to collect hyperspectral data of a region to be detected of the stone cultural relic.

3. The method of claim 1, wherein, In the step 3 or step 4, light of the wavelengths of 475 nm, 520 nm and 650 nm has high resolution and contrast in the spectral image, so that color details in the image can be more accurately reflected when features are extracted.

4. The method of claim 1, wherein, In the step 7, a plurality of algorithms are used to train disease classification models, and then samples not participating in modeling are selected for classification test, the test samples are input into the trained classifier for classification prediction, and specific algorithm application steps and combination modes are as follows: Model performance evaluation: global accuracy GA and kappa coefficient are used to evaluate performances of the classification models, GA and the kappa coefficient are calculated through a confusion matrix, and the calculation formulas are as follows: Global accuracy GA: where n is the total number of columns of the confusion matrix; is the number of samples in the i-th row and i-th column of the confusion matrix; N is the total number of samples used for accuracy evaluation; Kappa coefficient: wherein and N and N are the total number of samples in the i-th row and i-th column, respectively.

Images