Method and system for automatically identifying and classifying ancient silk damage types

Abstract

The invention discloses an ancient silk damage type automatic identification and classification method and system. The method comprises the following steps: S100, collecting hyperspectral data; s200, positioning a damaged area: processing the reflectivity data cube to generate a binary mask of the damaged area; s300, extracting multi-dimensional features: for each damaged area, extracting features from a plurality of dimensions including geometric morphology, spatial texture, spectral characteristics and physical parameters to form an initial feature vector; s400, performing feature optimization and classification: performing feature selection and fusion on the initial feature vector, performing classification by adopting a hybrid machine learning model, and outputting a damage type and confidence; s500, result visualization and decision support are carried out; according to the method, the four-dimensional feature system fusing the macroscopic morphology, the microscopic texture, the material composition and the internal physical state is systematically constructed in the cultural relic detection field for the first time, and the defect that the single feature representation capability is weak is overcome.

Description

Technical Field

[0001] This invention relates to the technical field of non-destructive testing of cultural relics, and in particular to an automatic identification and classification method and system for the types of damage to ancient silk. Background Technology

[0002] Silk artifacts, especially ancient silk, suffer from various types of damage during the aging process, such as insect infestation, mold, tearing, abrasion, and contamination. Currently, the assessment of such damage mainly relies on the visual observation and manual recording of cultural relic conservation experts. This method has problems such as low efficiency, strong subjectivity, inability to quantify, and easy to miss early or microscopic damage.

[0003] Hyperspectral imaging technology offers a new approach for non-destructive testing of cultural relics. However, existing technologies, such as those described in patent publication CN110110596B, which discloses hyperspectral image feature extraction, classification model construction, and classification methods, while improving classification performance, do not fully consider the material characteristics and physical mechanisms of damage to silk artifacts, resulting in insufficient feature representation capabilities. Patent publication CN117115147B discloses a machine vision-based textile inspection method and system. This method primarily uses multispectral image acquisition technology for defect identification based on feature extraction from convolutional neural networks and recurrent neural networks. However, it underutilizes spatial information such as morphology and texture, and its model generalization ability and result interpretability are limited. Therefore, existing technologies cannot meet the high requirements of multi-dimensional feature fusion, strong model generalization, and result interpretability for ancient silk damage detection. Summary of the Invention

[0004] The purpose of this invention is to solve the problems in the prior art by proposing an automatic identification and classification method and system for the types of damage to ancient silk, which can solve the above problems.

[0005] To achieve the above objectives, this invention proposes an automatic identification and classification method for the types of damage to ancient silk, comprising the following steps: S100. Acquire hyperspectral data: Use a hyperspectral imaging system to acquire reflectance data cubes of ancient silk samples; S200, Locate the damaged area: Process the reflectivity data cube to generate a binary mask of the damaged area; S300. Extracting multi-dimensional features: For each damaged area, features are extracted from multiple dimensions, including geometric shape, spatial texture, spectral characteristics and physical parameters, to form an initial feature vector; S400, Feature Optimization and Classification: After feature selection and fusion of the initial feature vector, a hybrid machine learning model is used for classification, and the damage type and confidence level are output. S500, Results Visualization and Decision Support: Based on classification results and physical parameters, generate visualization results, calculate damage severity, and output diagnostic reports.

[0006] Preferably, in step S100, the hyperspectral imaging system first performs a pre-scan at a lower resolution to generate an illumination distribution map of the sample surface and calculate a compensation coefficient map, and then controls the projector to project compensation light onto the dark area; after compensation, a hyperspectral reflectance data cube with high uniformity is formally acquired.

[0007] The S100 employs a hyperspectral imaging system that integrates a multimodal reference plate and active illumination compensation. Through a three-step method of pre-scanning, compensation, and formal acquisition, it obtains a cube of hyperspectral reflectance data of ancient silk with high uniformity and high signal-to-noise ratio.

[0008] Preferably, S200 specifically includes: A dual-branch collaborative partitioning strategy combining physical model-driven and data-driven approaches is adopted; based on a linear hybrid model. The inverted physical parameter map is used in conjunction with the damage probability map extracted by deep learning for segmentation to achieve precise positioning.

[0009] Preferably, in S300, the physical parameter dimension features are obtained by physical model inversion of the reflectance data cube, and are parameters characterizing the physical state of the silk surface or volume; the spectral characteristic dimension features include customized spectral indices based on the reflectance ratio of a specific wavelength.

[0010] Preferably, in step S300, a four-layer feature description system is constructed for each located damaged area, including: Geometric morphology layer: Extracts macroscopic shape features from the binary mask of the damaged area, including: 1) Circularity C, which characterizes the degree to which the shape of the region approximates a circle, calculated using the following formula: 1) Where A is the area of ​​the region and P is the perimeter of the region; 2) The contour fractal dimension D represents the irregularity and complexity of the region boundary. Box counting is used, and linear regression is applied to fit the data. The slope D is obtained from the relationship, where The required side length to cover the contour is The minimum number of boxes.

[0011] Spatial texture layer: Extracts microscopic texture pattern features from grayscale images at selected representative wavelengths (e.g., 750 nm), including: 1) Gray-level co-occurrence matrix statistics, used to describe the spatial dependencies of pixel pairs, such as calculating contrast. Multiple characteristics, including correlation, energy, and homogeneity, among which It is a gray-level co-occurrence matrix; 2) Multi-resolution local binary mode features, by setting different neighborhood radii and number of pixels (e.g. and ), extract feature vectors that can characterize local texture structures at different scales.

[0012] Spectral Characteristic Layer: Based on the spectral data of the damaged area, extract relevant features of material composition, including: 1) Regional average spectrum, which calculates the average reflectance of all pixels in the region across all bands to form a representative spectral curve; 2) Customized spectral indices, designed for typical silk defects, such as the silk aging index. It is used to detect the degree of protein aging and the suspected mold index. , used to indicate the presence of microbial metabolites.

[0013] Physical parameter layer: Intrinsic parameters obtained by inversion from physical models of light-matter interaction (such as Kubelka-Munk theory or linear mixture models), and their statistical distribution characteristics within the damaged area are calculated, including: 1) Surface roughness mean and standard deviation 1) Characterizes changes in surface scattering properties; 2) Volume absorption scattering ratio The mean value in the characteristic band (e.g., 1650 nm) and standard deviation It reflects changes in the body's composition and internal structure.

[0014] As a preferred approach, S400 proposes a three-level progressive feature optimization and weighted fusion strategy: Feature optimization: Initial screening is performed using a filtering method based on variance and mutual information, followed by fine screening using recursive feature elimination based on stability.

[0015] The feature stability score is calculated by the frequency with which it is retained in the core verification.

[0016] Weighted fusion: Quantify feature importance using SHapley Additive exPlanations (SHAP) values.

[0017] feature Fusion weights Determined by the absolute mean of its SHAP value and normalized accordingly: .

[0018] The weighted eigenvectors are .

[0019] Hybrid Classification: Constructing a Two-Stage Hybrid Expert Model: First Stage - Output Probability Distribution of Multiple Base Classifiers The second-stage meta-classifier uses the concatenated meta-features. Use the input to make the final decision.

[0020] Preferably, the S500 generates a multi-level intelligent diagnostic report system, the core of which is: Severity quantification assessment: The system automatically calculates the comprehensive severity index S for each region based on physical parameters.

[0021] The calculation formula is as follows: ,in, and These are the surface roughness and absorption-scattering ratio values ​​after Min-Max normalization, respectively. , Weighting coefficients (e.g., based on expert prior knowledge) , ).

[0022] This index translates optical characteristics into quantifiable protection priorities.

[0023] Interactive visualization and report generation: The system integrates spatial distribution, classification results, severity index, and knowledge base to automatically generate risk warnings (such as confidence levels < 0.05). Or severity index > Structured reports on protection recommendations.

[0024] This invention also proposes an automatic identification and classification system for ancient silk damage types, used to implement the above method, comprising: A hyperspectral imaging module is used to acquire hyperspectral data; it includes a multimodal reference plate and an active illumination compensation unit; the multimodal reference plate includes a high-reflectivity white area, a low-reflectivity black area, several grayscale levels with known reflectivity, and characteristic material points for wavelength calibration; the active illumination compensation unit includes a main light source assembly and a compensation light source assembly; the main light source assembly includes an LED lamp group, and the compensation light source assembly includes a projector; Damage localization module: includes a dual-branch collaborative segmentation network; The dual-branch collaborative segmentation network architecture includes: Spectral Feature Branch: The input is the spectral curve after SG filtering and SNV normalization. The spectral features are extracted through a three-layer MLP, and the output is a preliminary damage probability map. Physical parameter branch: Based on the linear mixture model, the surface roughness parameters of each pixel are inverted using the LM algorithm to generate a physical parameter map; The feature engineering module is used to perform feature extraction, selection, and fusion steps; The intelligent classification module is used to perform the classification and recognition steps; The decision support module is used to generate intelligent diagnostic reports.

[0025] The beneficial effects of this invention are: 1. This invention is the first to systematically construct a four-dimensional feature system in the field of cultural relic detection that integrates macroscopic morphology, microscopic texture, material composition and internal physical state, overcoming the shortcomings of weak characterization ability of single features; 2. The two-stage hybrid expert model proposed in this invention effectively utilizes the advantages of different algorithms by dynamically fusing the decisions of heterogeneous base classifiers through a meta-classifier, significantly improving the classification accuracy and stability in scenarios with limited sample size and imbalanced classes. 3. This invention deeply integrates the physical model inversion parameters into the entire process of feature extraction and severity assessment, giving the decision-making process of artificial intelligence a clear physical meaning and meeting the needs of cultural relic protection for causal analysis.

[0026] The features and advantages of the present invention will be described in detail through embodiments and in conjunction with the accompanying drawings. Attached Figure Description

[0027] Figure 1 This is the overall flowchart of the present invention; Figure 2 This is a schematic diagram of the multi-scale feature extraction of the present invention; Figure 3 This is a schematic diagram of the hybrid expert model of the present invention. Detailed Implementation

[0028] See Figure 1 , Figure 2 , Figure 3 An automatic identification and classification method for types of damage to ancient silk, specifically including the following steps: S100, Data Acquisition and Preprocessing: In a temperature and humidity controlled darkroom environment, the silk sample was placed flat on the sample stage; data acquisition was performed using the Headwall Nano-Hyperspec hyperspectral imaging system (spectral range 400-1000nm, spectral resolution 5nm); the illumination system consisted of two strip LED light sources with CRI>97, symmetrically illuminating at a 45° angle.

[0029] Before data acquisition, the system is pre-calibrated: dark current (lens cap closed) and reference board image are acquired.

[0030] The multimodal reference board includes a Spectralon® high-reflectivity white board (reflectivity >99%), an Avian-D® black paint (reflectivity <2%), a Kodak Q-13 grayscale scale (reflectivity known), and neodymium and erbium oxide feature points for wavelength calibration.

[0031] Using active illumination compensation technology, a pre-scan is first performed at a lower resolution (Binning 4×4) to generate an illumination distribution map of the sample surface. ;Calculate the compensation coefficient diagram ,in To average the illumination intensity, a TI DLP LightCrafter 4500 projector was used to project compensating light onto the dark area. After compensation, a high-uniformity hyperspectral reflectance data cube was officially acquired. The dimensions are 1600px × 1280px × 120bands, and it is stored in ENVI format.

[0032] S200, Damage Area Location: Call the pre-trained dual-branch collaborative segmentation network (DCS-Net).

[0033] The network architecture is as follows: Branch 1 (Spectral Feature Branch): The input is the spectral curve after SG filtering (window 11, order 2) and SNV normalization. Spectral features are extracted through a three-layer MLP (64-32-16 neurons), and the output is a preliminary damage probability map. .

[0034] Branch 2 (Physical Parameter Branch): Based on Linear Hybrid Model ,in The surface roughness parameters of each pixel are calculated using KM theory and the LM algorithm. Generate physical parameter maps .

[0035] Collaborative segmentation module: will and The input is processed by a collaborative segmentation module, which fuses features through feature concatenation and 3×3 convolution, and then passes through a graph cut algorithm (with energy function smoothing term weights). The optimization process ultimately outputs a binary mask with pixel-level precision.

[0036] In this embodiment, the system successfully located 12 independent damaged areas.

[0037] S300, Multi-dimensional Feature Extraction (taking region ID03 as an example): This step constructs a hierarchical feature descriptor to characterize the ID03 region from four dimensions: Geometric dimensional features: Calculate the area from the binary mask. pixels, perimeter pixels.

[0038] Circularity This indicates that the shape is close to a circle.

[0039] elongation .

[0040] Contour fractal dimension D: Using box counting; the contour image is divided into grids of different sizes s, and the minimum number of boxes required to cover the contour is calculated. ;right Performing linear regression, the slope is... This reflects that the boundary has moderate complexity.

[0041] Spatial texture dimension features: Select 750nm band grayscale image Perform the calculation.

[0042] GLCM features: Calculating distance For each pixel, the gray-level co-occurrence matrix in four directions (0°, 45°, 90°, and 135°) is used to extract the contrast, correlation, energy, and homogeneity of each direction. Finally, the average value and range value (maximum value - minimum value) of the four directions are calculated, resulting in a total of 8-dimensional features. For example, the average contrast is 15.2 and the contrast range is 3.1.

[0043] Multi-resolution LBP features: respectively using ( , )and( , The parameters were used to calculate the uniform LBP mode, resulting in 59-dimensional and 243-dimensional histograms. To reduce the dimensionality, the 243-dimensional histogram was reduced to 59-dimensional by segmented aggregation approximation (PAA), and finally spliced ​​together to form a 118-dimensional feature vector.

[0044] Spectral characteristics: Regional average spectrum: Calculate the average reflectance of all pixels within ID03 in each band to obtain a 120-dimensional spectral curve. .

[0045] Spectral heterogeneity description: Calculate the standard deviation of reflectance of all pixels within ID03 in each spectral band, forming a 120-dimensional "spectral standard deviation curve". A higher average value indicates uneven internal material composition.

[0046] Customized spectral index: Silk Aging Index (Amide II / III band change).

[0047] Mold Suspected Index (Absorption of microbial metabolites).

[0048] Physical parameter dimensional characteristics: Obtained from step S200 Figures and Extract the statistics from the ID03 region in the image: Surface roughness mean Standard deviation .

[0049] Absorption scattering ratio Mean at 1650nm Standard deviation .

[0050] Finally, the initial feature vector of region ID03 totals: dimension.

[0051] S400, Feature Optimization and Classification: This step combines three-level progressive feature selection with SHAP weighted fusion.

[0052] Feature standardization: Robust standardization is applied to the initial 377-dimensional features, i.e. This is to suppress the impact of outliers.

[0053] Feature selection: First-level filtering: Remove features with variance less than 0.01 (such as constant features), leaving 365 dimensions; calculate the mutual information (MI) between each feature and the damage type label, and retain the top 70% of features with the highest MI scores, approximately 256 dimensions.

[0054] Secondary fine screening (stability RFE): based on LightGBM model ( , ), and perform recursive feature elimination.

[0055] In each round of eliminating the least important 10% of features, a 5-fold cross-validation was performed; the frequency of each feature being retained in the 5-fold cross-validation was recorded (stability score); finally, the mean feature importance and the stability score (weighted in a 7:3 ratio) were combined to select the 55 most discriminative and stable features.

[0056] Adaptive weighted fusion: Train a LightGBM classifier on the above 55 features using the entire training set; then, compute the weighted fusion for each feature. SHAP values ​​on all training samples; features global weight .

[0057] Normalize all weights so that .

[0058] Finally, the feature vector of ID03 is represented as .

[0059] Hybrid expert model classification: Base classifier group: LightGBM: , .

[0060] SVM (RBF kernel): Employs a One-vs-Rest strategy, hyperparameters... Determined through grid search.

[0061] MLP: Structure is The activation function is ReLU, and the output layer is Softmax.

[0062] Meta-classifier: Concatenates the 4-class probability vectors (12 dimensions in total) output by the three base classifiers into a meta-feature. Train a logistic regression model as a meta-classifier.

[0063] Decision: The meta-features of ID03 are input into the meta-classifier, and the final output probability distribution is as follows. Therefore, it was determined to be "insect-eaten", with a confidence level of 0.91.

[0064] S500, Results Visualization and Decision Support: This step generates an interactive, drillable, content-based assessment decision support report.

[0065] Interactive visual interface: The main view displays an RGB composite image of the silk, with 12 damaged areas superimposed in different colors (insect damage - red, mildew - green, tear - blue, abrasion - yellow) based on the classification results.

[0066] After the user clicks on the ID03 area, the details panel on the right updates dynamically: This displays a magnified view of the area.

[0067] List: Type: Insect-infested | Confidence: 91% | Area: 0.95 Severity: 0.45.

[0068] Plot a comparison graph of its average spectral curve and the spectral curve of the healthy region to show its physical parameter values.

[0069] Severity Quantification Assessment: The system automatically calculates the comprehensive severity index S for each region based on physical parameters; the formula is as follows: ,in and It is the value after Min-Max normalization; ID03 It falls under the category of "moderate" severity.

[0070] Structured diagnostic report: The system automatically generates a PDF report, which includes: Statistical summary: The table lists the quantity, total area, and average confidence level of each type of damage.

[0071] Risk warning: Areas with confidence level <0.85 or severity index >0.7 will be automatically highlighted (e.g., ID07, mold, confidence level 78%, manual review recommended).

[0072] Protection decision recommendations: Match recommendations from the knowledge base, such as: "Three insect infestations were found in this inspection, with moderate severity. It is recommended to focus on monitoring and improve the storage microenvironment; two low-confidence mold spots were found. It is recommended to take samples for laboratory analysis."

[0073] An automatic identification and classification system for ancient silk damage types, used to implement the above method, includes: A hyperspectral imaging module is used to acquire hyperspectral data; it includes a multimodal reference plate and an active illumination compensation unit; the multimodal reference plate includes a high-reflectivity white area, a low-reflectivity black area, several grayscale levels with known reflectivity, and characteristic material points for wavelength calibration; the active illumination compensation unit includes a main light source assembly and a compensation light source assembly; the main light source assembly includes an LED lamp group, and the compensation light source assembly includes a projector; Damage localization module: includes a dual-branch collaborative segmentation network; The dual-branch collaborative segmentation network architecture includes: Spectral Feature Branch: The input is the spectral curve after SG filtering and SNV normalization. The spectral features are extracted through a three-layer MLP, and the output is a preliminary damage probability map. Physical parameter branch: Based on the linear mixture model, the surface roughness parameters of each pixel are inverted using the LM algorithm to generate a physical parameter map; The feature engineering module is used to perform feature extraction, selection, and fusion steps; The intelligent classification module is used to perform the classification and recognition steps; The decision support module is used to generate intelligent diagnostic reports.

[0074] The above embodiments are illustrative of the present invention and are not intended to limit the present invention. Any simple modifications to the present invention are within the scope of protection of the present invention.

Claims

1. A method for automatically identifying and classifying types of damage to ancient silk, characterized in that, The method comprises the following steps: S100, collecting hyperspectral data: using a hyperspectral imaging system to obtain a reflectance data cube of the ancient silk sample; S200, locating the damaged area: processing the reflectance data cube to generate a binary mask of the damaged area; S300, extracting multi-dimensional features: for each damaged area, features are extracted from multiple dimensions including geometric morphology, spatial texture, spectral characteristics and physical parameters to form an initial feature vector; S400, feature optimization and classification: after feature selection and fusion of the initial feature vector, a hybrid machine learning model is used for classification to output the damage type and confidence; S500, result visualization and decision support: based on the classification results and physical parameters, a visualization result is generated and the damage severity is calculated to output a diagnosis report.

2. The method of claim 1, wherein the method comprises the steps of: a) acquiring the image of the ancient silk fabric; b) determining the type of the ancient silk fabric by using the image of the ancient silk fabric; and c) classifying the type of the ancient silk fabric. In S100, the hyperspectral imaging system first performs pre-scanning at a lower resolution to generate an illumination distribution map of the sample surface and calculate a compensation coefficient map, and then controls the projector to project compensation light to the dark area; After compensation, a high uniformity hyperspectral reflectance data cube is obtained.

3. The automatic identification and classification method for ancient silk damage types as described in claim 1, characterized in that: In S200, a double-branch collaborative segmentation algorithm based on physical model inversion and deep learning feature extraction is used to locate the damaged area; the physical model is a linear mixing model representing the surface scattering and volume scattering of silk.

4. The ancient silk damage type automatic identification and classification method according to claim 1, characterized in that: In S300, the physical parameter dimension feature is obtained by physical model inversion on the reflectance data cube, representing the parameters of the physical state of the silk surface or volume; the spectral characteristic dimension feature includes customized spectral indices based on specific wavelength reflectance ratios.

5. The ancient silk damage type automatic identification and classification method according to claim 1, characterized in that, In S300, for each located damaged area, a four-layer feature description system is constructed, including: Geometric morphology layer: macro shape features are extracted from the binary mask of the damaged area, including: Circularity: representing the degree to which the region shape approximates a circle; Contour fractal dimension: representing the irregularity and complexity of the region boundary; Spatial texture layer: micro texture pattern features are extracted on the selected representative waveband grayscale image, including: Gray level co-occurrence matrix statistics: used to describe the spatial dependence relationship of pixel pairs; Multi-resolution local binary pattern feature: by setting different neighborhood radius and pixel number, feature vectors are extracted to represent local texture structure at different scales; Spectral characteristics layer: material composition related features are extracted based on the spectral data of the damaged area, including: Regional average spectrum: calculating the average value of all pixel reflectance at each waveband within the region to form a representative spectral curve; Customized spectral indices: including silk aging index and mildew suspected index; Physical parameter layer: intrinsic parameters are obtained based on the physical model of light and matter interaction, and their statistical distribution characteristics in the damaged area are calculated, including: Mean and standard deviation of surface roughness, representing the change of surface scattering characteristics; 6. The ancient silk damage type automatic identification and classification method according to claim 1, characterized in that, Mean and standard deviation of volume absorption scattering ratio at characteristic waveband. In S400, the specific steps are as follows: Feature optimization: initial screening is performed using a filtering method based on variance and mutual information, and fine screening is performed using recursive feature elimination based on stability; Among them, the feature stability score is calculated by the frequency of being retained in the centring verification; Weighted fusion: SHapley Additive exPlanations value is used to quantify feature importance; Hybrid classification: a two-stage hybrid expert model is constructed: the first stage contains multiple heterogeneous base classifiers, whose output probabilities are spliced into meta-features; the second stage is a meta-classifier that makes a final decision on the meta-features.

7. The ancient silk damage type automatic identification and classification method according to claim 1, characterized in that: The severity index S in the S500 step is calculated from the parameters obtained by inversion from the physical model.

8. An ancient silk damage type automatic identification and classification system for implementing the method of any one of claims 1-7, characterized in that, Comprise: A hyperspectral imaging module for collecting hyperspectral data; including a multi-modal reference plate and an active illumination compensation unit; the multi-modal reference plate includes a high reflectivity white area, a low reflectivity black area, a plurality of gray scales with known reflectivity, and feature substance points for wavelength calibration; The active illumination compensation unit includes a main light source assembly and a compensation light source assembly; the main light source assembly includes an LED lamp group, and the compensation light source assembly includes a projector; The damage positioning module includes a double-branch collaborative segmentation network; The double-branch collaborative segmentation network architecture comprises: The spectral feature branch: the input is the spectral curve after SG filtering and SNV standardization, the spectral feature is extracted through a three-layer MLP, and the preliminary damage probability map is output; The physical parameter branch: based on the linear mixing model, the L-M algorithm is used to invert the surface roughness parameter of each pixel to generate a physical parameter map; A feature engineering module for performing feature extraction, selection and fusion steps; An intelligent classification module for performing classification and identification steps; A decision support module for generating an intelligent diagnosis report.

Images