Method and system for automatically positioning and selecting damaged silk area

Abstract

The invention discloses an automatic silk damage area positioning and selecting method and system. The method comprises the following steps: S1, acquiring hyperspectral data; s2, generating a damage probability graph; s3, generating a physical parameter diagram; s4, collaborative segmentation: taking the damage probability graph and the physical parameter graph as input, performing fusion processing by using a spectrum-space collaborative segmentation network, and outputting a silk damage area segmentation mask after global optimization; according to the invention, a hyperspectral imaging system integrated with a multi-modal reference plate and active illumination compensation is designed according to unique properties of silk, a compensation graph is generated through pre-scanning, and local brightness compensation is carried out by using a DMD projector, so that high uniformity and high fidelity of spectral data are ensured from the source; and a stable and reliable precondition is created for subsequent accurate detection.

Description

Technical Field

[0001] This invention relates to the technical field of hyperspectral image classification, and in particular to a method and system for automatically locating and selecting damaged areas of silk. Background Technology

[0002] Silk products are prone to damage during production and storage; traditional manual inspection methods are inefficient, subjective, and prone to missed detections; the unique properties of silk products present unique challenges to damage detection, such as surface optical characteristics, minor damage, and the coexistence of chemical and physical changes; hyperspectral imaging technology can simultaneously acquire spatial and rich spectral information of the target, which makes it possible to identify silk damage non-destructively and accurately; however, directly applying hyperspectral technology to silk damage detection faces the following unresolved problems: Silk fibers have a smooth surface and a triangular cross-section, making them prone to directional specular reflection. During imaging, pixels in the highlight areas capture strongly reflected ambient light, rather than the diffuse reflection light from the silk itself, resulting in "overexposure" of the spectral signal in that area. Characteristic absorption peaks are masked (such as the amide bond absorption peak of silk fibroin and dye characteristic peaks), and there may even be an overall rise in the spectral baseline, with spectral differences from normal areas far exceeding the differences in the material itself. Silk is made of interwoven silk fibers, with a large number of micron-sized gaps between the fibers, and the fibers themselves have natural textures. When detecting slight damage to silk, the spectral differences in the fiber texture may be misjudged as "damage signals," or the damaged area may be masked by texture shadows, leading to missed detection. Traditional methods often rely on single-type features, such as image texture or raw spectrum, and fail to effectively integrate spectral details, spatial context and physical mechanism features. They have low accuracy in identifying weak and complex damage and are difficult to obtain high-fidelity spectral data under uniform illumination. Purely data-driven models lack physical interpretability and have poor adaptability to different types and aging levels of silk. Simple spectral classification or image segmentation methods are susceptible to noise interference and have inaccurate boundary positioning, making it difficult to meet the needs of precision detection.

[0003] In the prior art, patent document CN114821167A discloses a hyperspectral image classification method based on functional representation under multiple constraints. This method mainly focuses on the spectral classification algorithm itself and does not solve the special characteristics of silk imaging and the problem of deep fusion of spectral and spatial features. Patent document CN116524250A discloses a belt damage identification method with improved ResNeXt50 and multispectral imaging. Although this method uses multispectral imaging, it aims to acquire multi-channel images for classification and does not solve the problem of spectral distortion caused by highlights and shadows on the silk surface. Therefore, in order to address the practical difficulties of silk damage detection, there is an urgent need for a collaborative segmentation method that integrates high-quality imaging, multi-feature fusion, and physical model guidance. Summary of the Invention

[0004] The purpose of this invention is to solve the problems in the prior art by proposing an automatic location and selection method and system for damaged areas of silk, which can solve the above problems.

[0005] To achieve the above objectives, this invention proposes an automatic location and selection method for damaged areas in silk, comprising the following steps: S1. Acquiring hyperspectral data: Hyperspectral data of the silk sample to be tested is acquired using a hyperspectral imaging system and calibrated using a multimodal reference plate. With the help of a coaxial active illumination compensation system, a hyperspectral reflectance data cube with high uniformity is obtained. S2. Generate a damage probability map: Perform spectral preprocessing on the hyperspectral reflectance data cube, extract features of multiple dimensions for each pixel and perform adaptive weighted fusion, and obtain a damage probability map through a classification model; S3. Generate physical parameter map: Based on the physical model of silk spectral reflectance, the hyperspectral reflectance data cube is inverted to obtain at least one physical parameter map reflecting the physical state of the silk surface or volume. S4. Collaborative Segmentation: Using the damage probability map and the physical parameter map as input, a spectral-spatial collaborative segmentation network is used for fusion processing, and after global optimization, a silk damage area segmentation mask is output.

[0006] Preferably, in S1, the multimodal reference plate includes a high-reflectivity diffuse white area, a low-reflectivity black area, several neutral gray levels with known reflectivities, and feature point materials for real-time calibration of spectral wavelengths.

[0007] Preferably, in step S1, the working steps of the coaxial active illumination compensation system are as follows: pre-scanning the sample to generate an illumination distribution map; calculating a compensation coefficient map based on the illumination distribution map; controlling a programmable projection light source to project compensation light onto the dark area of ​​the sample according to the compensation coefficient map, and then performing formal hyperspectral data acquisition.

[0008] Preferably, in S2, the multi-dimensional features include fine spectral features, deep spatial-spectral features, and physical optical features; the weights of the adaptive weighted fusion are determined based on the inverse variance of each type of feature on the training sample set.

[0009] Preferably, the fine spectral features include wavelet coefficients and spectral derivatives extracted by continuous wavelet transform; the deep spatial-spectral features are extracted by a 3D convolutional neural network; and the physical optical features include the absorption-scattering ratio K / S and surface roughness factor α obtained based on the Kubelka-Munk theory.

[0010] Preferably, in S3, The physical model for the spectral reflectance of silk is a linear mixture model, and its total reflectance... satisfy: ; in For surface scattering weights, The specular reflection coefficient, The volume reflectance is calculated based on the Kubelka-Munk theory.

[0011] Preferably, the physical parameter diagram obtained in S3 includes a surface roughness α diagram and / or a volume absorption scattering ratio. picture.

[0012] These parametric maps not only serve as feature input classifiers, but also as guiding information inputs to the segmentation network with clear physical meanings, achieving deep synergy between physical mechanisms and data-driven algorithms, and endowing the model with physical interpretability.

[0013] Preferably, in step S4, the spectral-spatial cooperative segmentation network is a dual-branch network, comprising: Spatial branching is used to process the physical parameter map and output a spatial probability map. The spectral branch is used to process the spectral features corresponding to the damage probability map and output the spectral probability value. The collaborative decision-making module is used to dynamically fuse the spatial probability map and spectral probability values ​​to generate the final probability map.

[0014] Preferably, the collaborative decision-making module uses a lightweight neural network to dynamically analyze contextual features from the spatial branch and high-dimensional spectral features from the spectral branch. This module concatenates the mid-level features output from the spatial branch encoder with the feature vectors from the spectral branch, and generates a pair of dynamically fused weights for each pixel or feature region through fully connected layers and nonlinear activation. and The final fusion probability is obtained as follows: ; In regions with complex textures and mixed spectra (such as the intersections of warp and weft threads in a fabric), the network learns to rely more on... (Right now Larger areas (where fine spectral differences are more accurate than spatial texture in determining damage) are better identified by their size; in homogeneous regions with blurred spectral features but exceptionally clear physical morphology (such as large areas of uniform wear), the network learns to rely more heavily on these features. (Right now (larger), using the spatial continuity provided by the physical parameter map to correct the segmentation results.

[0015] Preferably, the global optimization in step S4 employs a graph cut algorithm, whose energy function E(L) satisfies: ; in For data items, This is a smoothing term.

[0016] This invention also proposes an automatic positioning and selection system for damaged areas of silk, used to implement the above positioning method, comprising: The hyperspectral imaging module is used to acquire hyperspectral data; The data processing module is used to generate damage probability maps and physical parameter maps; The collaborative segmentation module is used for collaborative segmentation of the damage probability map and the physical parameter map; The results output module is used to visualize the segmentation results.

[0017] Preferably, the hyperspectral imaging module is equipped with a coaxial active illumination compensation system, which includes an imaging platform, a main light source, a supplementary light source, and a beam splitter. The silk sample to be tested is laid on the imaging platform, and a hyperspectral camera is installed above the imaging platform. The main light source includes two strip LED light sources symmetrically arranged on both sides above the imaging platform. A beam splitter is installed between the hyperspectral camera and the imaging platform, and a supplementary light source is installed on the side of the beam splitter. The supplementary light source is a programmable DMD projector.

[0018] The beneficial effects of this invention are as follows: Based on the unique properties of silk, this invention designs a hyperspectral imaging system integrating a multimodal reference plate and active illumination compensation. By generating a compensation map through pre-scanning and utilizing a DMD projector for local brightness compensation, it ensures high uniformity and high fidelity of the spectral data from the source, creating stable and reliable prerequisites for subsequent accurate detection. The adaptive fusion of multi-dimensional features fully utilizes the complementarity of information, improving the model's generalization ability to various types of damage, resulting in high recognition accuracy and strong robustness. This invention introduces a physical model, giving the segmentation results clear physical meaning, which helps in analyzing the causes of damage and provides physical interpretability. The combination of collaborative segmentation networks and graph cut optimization achieves pixel-level precise positioning, making the segmentation boundaries more accurate. This invention forms a complete end-to-end solution with strong practicality and the ability to automate the entire process.

[0019] The features and advantages of the present invention will be described in detail through embodiments and in conjunction with the accompanying drawings. Attached Figure Description

[0020] Figure 1 This is a method roadmap of the present invention; Figure 2 This is an exploded view of S1 of the present invention; Figure 3 This is a schematic diagram of the coaxial active illumination compensation system of the present invention; Figure 4 This is an exploded view of the present invention, S2. Figure 5 This is an exploded view of the present invention, S3; Figure 6 This is an exploded view of the present invention, S4.

[0021] In the figure: 1. Imaging platform; 2. Main light source; 3. Supplementary light source; 4. Beam splitter; 5. Hyperspectral camera; 6. Silk sample to be tested. Detailed Implementation

[0022] This invention employs an imaging system that uses two cameras working in tandem: visible-near infrared (VNIR, 400-1000nm) and short-wave infrared (SWIR, 1000-2500nm). The working distance between the Headwall Nano-Hyperspec (VNIR) and SWIR-Hyperspec cameras is 80cm. The reference board includes a Spectralon white board, Avian-D black paint, Kodak Q-13 grayscale, and neodymium and erbium oxide calibration points.

[0023] See Figures 1 to 6 A method for automatically locating and selecting damaged areas in silk, specifically including the following steps: S1. Hyperspectral data acquisition: Hyperspectral data of the silk sample to be tested is acquired using a hyperspectral imaging system and calibrated using a multimodal reference plate. With the help of a coaxial active illumination compensation system, a hyperspectral reflectance data cube with high uniformity is obtained.

[0024] In S1, the coaxial active illumination compensation system uses two strip LED light sources with a CRI>97 and a TI DLPLightCrafter 4500 projector as light sources. During operation, the coaxial active illumination compensation system first pre-scans the sample to generate an illumination distribution map; then calculates a compensation coefficient map based on the illumination distribution map; and controls the programmable projection light source to project compensation light onto the dark area of ​​the sample according to the compensation coefficient map before performing formal hyperspectral data acquisition.

[0025] The specific steps of S1 are as follows: First, perform system pre-calibration (acquire dark current and reference plate images); then perform pre-scanning (speed 10cm / s, Binning 4x4) to generate an illumination distribution map and calculate the compensation coefficient map, which is then compensated by the DMD projector; finally, acquire a high-resolution hyperspectral data cube under uniform illumination and correct it to reflectance data in real time, storing it in ENVI format.

[0026] S2. Feature Extraction and Probabilistic Map Generation: The data obtained from S1 is preprocessed using Savitzky-Golay filtering and Standard Normal Variable Transform (SNV), and then three types of features are extracted in parallel: Fine spectral features were extracted using the continuous wavelet transform coefficients (scales 10, 15, 20) and first derivative spectra of the Morlet wavelet (ω0=6).

[0027] Extract deep spatial-spectral features using a lightweight 3D-CNN (input 5x5x20, 2-layer convolution) to extract spatial-spectral features of the pixel neighborhood.

[0028] Physical optical features were extracted, and the absorption scattering ratio (K / S) and surface roughness factor (α) were obtained by inversion based on Kubelka-Munk theory.

[0029] Subsequently, an adaptive weighted fusion strategy based on the inverse of variance was adopted to fuse the three types of features. ; The data is then input into the LightGBM classifier (hyperparameters: num_leaves=31, learning_rate=0.05) for classification, generating a pixel-level damage probability map.

[0030] S3. Steps for generating physical parameter diagrams: Constructing a microscopic spectral reflectance hybrid model for the properties of silk to invert physical parameters: The model is expressed as follows: ,in Calculated using Kubelka-Munk theory, and Using quadratic polynomials Fitting.

[0031] The Levenberg-Marquardt algorithm is used to invert parameters {α, a, b, c} to generate a surface roughness α map and / or bulk absorption scattering ratio. These parametric graphs not only serve as feature inputs to the classifier, but also as guiding information inputs to the segmentation network with clear physical meanings, achieving deep synergy between physical mechanisms and data-driven algorithms, and endowing the model with physical interpretability.

[0032] S4. Collaborative segmentation steps: The failure probability map of S2 and the physical parameter map of S3 are input into the spectral-spatial dual-branch collaborative segmentation network (DCS-Net).

[0033] This network architecture includes: The spatial branch uses a lightweight U-Net (encoder downsampled twice, 16 channels, 32 channels) to process the physical parameter map and output the spatial probability map. The spectral branch uses a 3-layer MLP (64-16-1 neurons) to process pixel spectral features and output spectral probability values. The collaborative decision-making module is used to dynamically fuse the spatial probability map and spectral probability values ​​to generate the final probability map.

[0034] The collaborative decision-making module is a lightweight three-layer fully connected neural network. It first performs parallel global average pooling and global max pooling on the input spatial branch feature map (dimension C×H×W) to obtain two C×1×1 global context descriptors. These two descriptors are then concatenated with the D-dimensional feature vector of the spectral branch to obtain a (2C+D)-dimensional fused feature vector. Subsequently, this vector passes through a dimensionality reduction fully connected layer (output dimension (2C+D) / 4, activation function ReLU), a feature enhancement fully connected layer (output dimension (2C+D) / 16, activation function ReLU), and a weight generation fully connected layer (output dimension 2, activation function Softmax), finally outputting normalized dynamic fused weights. .

[0035] Finally, the fusion probability is obtained. .

[0036] The collaborative decision-making module is the core of achieving organic integration. Instead of performing a simple weighted average, this module uses a lightweight neural network to dynamically analyze contextual features from the spatial branch and high-dimensional spectral features from the spectral branch.

[0037] In regions with complex textures and mixed spectra (such as the intersections of warp and weft threads in a fabric), the network learns to rely more on... (Right now Larger areas (where fine spectral differences are more accurate than spatial texture in determining damage) are better identified by their size; in homogeneous regions with blurred spectral features but exceptionally clear physical morphology (such as large areas of uniform wear), the network learns to rely more heavily on these features. (Right now (Larger), using the spatial continuity provided by the physical parameter graph to correct the segmentation results; this "dynamic weighted fusion" mechanism essentially solves the problem of contribution allocation of heterogeneous information sources in the segmentation process, and achieves accuracy and robustness that surpasses fixed weight or simple concatenated networks.

[0038] Finally, the graph cut algorithm (smoothing term weight) is used. The image grayscale standard deviation is optimized to obtain the final segmentation mask, which is then overlaid on the RGB image.

[0039] An automatic location and selection system for damaged areas of silk, used to implement the above method, includes: The hyperspectral imaging module is used to acquire hyperspectral data; The data processing module is used to generate damage probability maps and physical parameter maps; The collaborative segmentation module is used for collaborative segmentation of the damage probability map and the physical parameter map; The results output module is used to visualize the segmentation results.

[0040] The hyperspectral imaging module is equipped with a coaxial active illumination compensation system, which includes an imaging platform 1, a main light source 2, a supplementary light source 3, and a beam splitter 4. The silk sample 6 to be tested is laid on the imaging platform 1, and a hyperspectral camera 5 is installed above the imaging platform 1. The main light source 2 includes two strip LED light sources symmetrically arranged on both sides above the imaging platform 1, with the angle of the strip LED light sources being 45°. A beam splitter 4 is installed between the hyperspectral camera 5 and the imaging platform 1, and a supplementary light source 3 is installed on the side of the beam splitter 4. The supplementary light source 3 is a programmable DMD projector.

[0041] This invention employs a multimodal reference board, including white areas, black areas, grayscale, and wavelength calibration points, for real-time system calibration. It also utilizes active illumination compensation technology, including pre-scanning, generating a compensation map, and projection compensation, to effectively suppress shadows and highlights on silk fabrics, thereby obtaining a highly uniform hyperspectral reflectance data cube.

[0042] The method of this invention can accurately identify minute wear and chemical degradation areas that are difficult to detect with the naked eye, and the segmentation boundary matches the actual situation highly, which is significantly better than traditional single methods.

[0043] The above embodiments are illustrative of the present invention and are not intended to limit the present invention. Any simple modifications to the present invention are within the scope of protection of the present invention.

Claims

1. A method for automatically locating and selecting damaged areas of silk, characterized in that, Includes the following steps: S1. Acquiring hyperspectral data: Hyperspectral data of the silk sample to be tested is acquired using a hyperspectral imaging system and calibrated using a multimodal reference plate. With the help of a coaxial active illumination compensation system, a hyperspectral reflectance data cube with high uniformity is obtained. S2. Generate a damage probability map: Perform spectral preprocessing on the hyperspectral reflectance data cube, extract features of multiple dimensions for each pixel and perform adaptive weighted fusion, and obtain a damage probability map through a classification model; S3. Generate physical parameter map: Based on the physical model of silk spectral reflectance, the hyperspectral reflectance data cube is inverted to obtain at least one physical parameter map reflecting the physical state of the silk surface or volume. S4. Collaborative Segmentation: Using the damage probability map and the physical parameter map as input, a spectral-spatial collaborative segmentation network is used for fusion processing, and after global optimization, a silk damage area segmentation mask is output.

2. The method for automatically locating and selecting damaged areas of silk as described in claim 1, characterized in that: In S1, the multimodal reference plate includes a high-reflectivity diffuse white area, a low-reflectivity black area, several neutral gray levels with known reflectivities, and feature point materials for real-time calibration of spectral wavelengths.

3. The method for automatically locating and selecting damaged areas of silk as described in claim 1, characterized in that: In step S1, the working steps of the coaxial active illumination compensation system are as follows: pre-scanning the sample to generate an illumination distribution map; calculating a compensation coefficient map based on the illumination distribution map; controlling a programmable projection light source to project compensation light onto the dark area of ​​the sample according to the compensation coefficient map, and then performing formal hyperspectral data acquisition.

4. The method for automatically locating and selecting damaged areas of silk as described in claim 1, characterized in that: In S2, the multi-dimensional features include fine spectral features, deep spatial-spectral features, and physical optical features; the weights of the adaptive weighted fusion are determined based on the inverse variance of each feature on the training sample set; the fine spectral features include wavelet coefficients and spectral derivatives extracted through continuous wavelet transform; the deep spatial-spectral features are extracted by a 3D convolutional neural network; and the physical optical features include the absorption-scattering ratio K / S and surface roughness factor α obtained based on Kubelka-Munk theory inversion.

5. The method for automatically locating and selecting damaged areas of silk as described in claim 1, characterized in that: In S3, the physical model of silk spectral reflectance is a linear mixture model; the physical parameter diagrams obtained in S3 include the surface roughness α diagram and / or the volume absorption scattering ratio K / S(λ) diagram.

6. The method for automatically locating and selecting damaged areas of silk as described in claim 1, characterized in that: In step S4, the spectral-spatial cooperative segmentation network is a dual-branch network, comprising: Spatial branching is used to process the physical parameter map and output a spatial probability map. The spectral branch is used to process the spectral features corresponding to the damage probability map and output the spectral probability value. The collaborative decision-making module is used to dynamically fuse the spatial probability map and spectral probability values ​​to generate the final probability map.

7. The automatic location and selection method for damaged areas of silk as described in claim 6, characterized in that: The collaborative decision-making module uses a lightweight neural network to dynamically analyze the contextual features from the spatial branch and the high-dimensional spectral features from the spectral branch. This module concatenates the mid-level features output by the spatial branch encoder with the feature vectors of the spectral branch, and generates a pair of dynamic fusion weights for each pixel or feature region through fully connected layers and nonlinear activation. The values ​​of the two fusion weights are automatically adjusted according to different regions to obtain the final fusion probability.

8. The method for automatically locating and selecting damaged areas of silk as described in claim 1, characterized in that: In S4, the global optimization uses a graph cut algorithm.

9. An automatic positioning and selection system for damaged areas of silk, used to implement the method described in any one of claims 1-8, characterized in that, include: The hyperspectral imaging module is used to acquire hyperspectral data; The data processing module is used to generate damage probability maps and physical parameter maps; The collaborative segmentation module is used for collaborative segmentation of the damage probability map and the physical parameter map; The results output module is used to visualize the segmentation results.

10. The automatic positioning and selection system for damaged areas of silk as described in claim 9, characterized in that: The hyperspectral imaging module is equipped with a coaxial active illumination compensation system. The coaxial active illumination compensation system includes an imaging platform (1), a main light source (2), a supplementary light source (3), and a beam splitter (4). The silk sample (6) to be tested is laid on the imaging platform (1). A hyperspectral camera (5) is provided above the imaging platform (1). The main light source (2) includes two strip LED light sources symmetrically arranged on both sides above the imaging platform (1). A beam splitter (4) is provided between the hyperspectral camera (5) and the imaging platform (1). A supplementary light source (3) is provided on the side of the beam splitter (4). The supplementary light source (3) is a programmable DMD projector.

Images