A workflow-based method for automated analysis and model optimization of hyperspectral data

Abstract

The present invention discloses a workflow-based method for automated analysis and model optimization of hyperspectral data, and the steps are as follows: S1, processing the acquired hyperspectral image of the target sample, extracting and saving the spectral data of the characteristic area, and constructing a one-dimensional hyperspectral data set of all target samples; S2, constructing a DeepSpNet model based on a one-dimensional convolutional neural network; S3, setting the training parameters of the DeepSpNet model, and establishing a hyperparameter combination; S4, dividing the one-dimensional hyperspectral data set into a training set and a test set in proportion, training the DeepSpNet model with the training set, evaluating the accuracy, precision, and recall of the DeepSpNet model through the test set, and automatically screening the best hyperparameter combination; S5, saving the DeepSpNet model with the best hyperparameter combination for target prediction. The present invention can accurately capture the defect characteristics inside the sample and effectively improve the depth detection capability.

Description

Technical Field

[0001] The present invention relates to the technical field of hyperspectral analysis, and in particular to a workflow-based hyperspectral data automated analysis and processing and classification model optimization method. Background Art

[0002] Hyperspectral detection technology has been widely used in non-destructive testing of the internal quality of fruits and other crops. Hyperspectral imaging can quickly, efficiently and contactlessly detect the surface features of objects and some internal information. However, existing hyperspectral technology has certain limitations in some applications, especially in the detection of deep defects in objects. Hyperspectral imaging can usually only detect the spectral information of the sample surface, and has limited detection effect on internal features.

[0003] The problems existing in the prior art mainly include the following points:

[0004] 1) Traditional hyperspectral image acquisition methods mainly focus on sample surface information and lack the ability to detect internal features of objects;

[0005] 2) Manual processing of hyperspectral data is cumbersome and it is difficult to automate the entire data analysis process;

[0006] 3) Existing software cannot effectively support phase synthesis processing and cannot automatically extract and synthesize multi-angle phase spectral data;

[0007] 4) Model optimization mainly relies on manual tuning of hyperparameters, which is inefficient and difficult to ensure the optimal performance of the model;

[0008] 5) It is difficult to monitor and record the model training process in real time, and large amounts of data processing require too much computing power and takes too long. Summary of the invention

[0009] Purpose of the invention: The purpose of the present invention is to provide a workflow-based automated analysis and processing method for hyperspectral data and a classification model building method for automatically optimizing hyperparameters, which can accurately capture the defect characteristics inside the sample, effectively improve the depth detection capability, and provide a feasible and efficient workflow for detection technology based on hyperspectral data.

[0010] Technical solution: A workflow-based method for automated analysis and model optimization of hyperspectral data, including the following steps:

[0011] S1, using imaging equipment to obtain a hyperspectral image of the target sample, and processing the hyperspectral image, extracting and saving the spectral data of the characteristic area, and constructing a one-dimensional hyperspectral data set of all target samples;

[0012] S2, builds DeepSpNet model based on one-dimensional convolutional neural network;

[0013] S3, set the training parameters of the DeepSpNet model and establish a hyperparameter combination;

[0014] S4, divide the one-dimensional hyperspectral dataset into a training set and a test set in proportion, use the training set to train the DeepSpNet model, evaluate the accuracy, precision, and recall of the DeepSpNet model through the test set, and automatically select the best hyperparameter combination;

[0015] S5, saves the DeepSpNet model with the best hyperparameter combination for target prediction.

[0016] Furthermore, the implementation steps for constructing a one-dimensional hyperspectral dataset of all samples are as follows:

[0017] S11, using a hyperspectral imaging device to respectively acquire images of the target sample at three phases of 0°, 120° and 240°, and storing the image data;

[0018] S12, obtaining a structured light high-dimensional image and a uniform illumination image of the target sample by performing de-modeling and synthesis processing on the collected image data;

[0019] S13, after determining a feature area that needs attention on the structured light high-dimensional image and the uniform illumination image, extracting image data of the feature area;

[0020] S14, saving the image data of the feature area in an array form, wherein each one-dimensional sample data in the array is the hyperspectral band data of a pixel in the feature area;

[0021] S15, repeating steps S12 to S14 for all target samples, extracting and storing the structured light high-dimensional image and uniform illumination image of each target sample, and performing label classification on the data to obtain a one-dimensional hyperspectral data set for all target samples.

[0022] Furthermore, in step S11, the number of target samples of the collected hyperspectral images is evenly distributed according to the number of categories.

[0023] Furthermore, in step S12, spectral data of a specific band is extracted by performing dimensionality reduction processing on the structured light high-dimensional image and the uniform illumination image of the target sample.

[0024] Furthermore, the DeepSpNet model includes multiple one-dimensional convolutional layers, pooling layers, fully connected layers, and output layers; a one-dimensional batch normalization layer is added after each convolutional layer, and a normalization layer is added between the fully connected layers; the fully connected layer corresponds to the classification result output through the softmax function.

[0025] Further, in step S3, the size of the convolution kernel, the step size of each convolution kernel advance, and the round of each training are preset; the hyperparameters include: learning rate, batch size, number of channels of the convolution layer, negative slope of LeakyReLU, and Dropout value; the hyperparameter combination is a set of possible values ​​of all hyperparameters.

[0026] Furthermore, in step S4, during the training of the DeepSpNet model, other hyperparameters are automatically searched through itertools.product and param_grid dictionaries; an early stopping mechanism is introduced by defining EarlyStopping; and the training progress is monitored and recorded in real time.

[0027] Compared with the prior art, the present invention has the following significant effects:

[0028] 1. The present invention enhances the detection of specific defects inside objects by introducing structured light illumination mode. At the same time, the design of automated processing and model optimization process solves the computational complexity problem existing in the existing vibration spectroscopy analysis model. The automated process of the present invention not only reduces human intervention, but also improves the accuracy and efficiency of hyperspectral image analysis, and is particularly suitable for non-destructive detection of internal diseases of samples such as fruits; the present invention uses multi-phase structured light hyperspectral imaging technology, which can more accurately capture the defect characteristics inside the sample by de-modeling and synthesizing illumination data of different phases; compared with the vibration spectroscopy analysis method, the present invention can effectively improve the depth detection capability;

[0029] 2. Use the de-modeling method to synthesize structured light high-dimensional images and uniform illumination images. On the basis of determining the specific defect area, distinguish normal and abnormal pixels, extract the one-dimensional hyperspectral classification data set, train the 1D-CNN classification model DeepSpNet, and realize automatic optimization of model parameters while saving computing power through grid search and introducing early stopping mechanism. At the same time, the training process can be recorded in real time. Finally, the trained model can be used to realize early abnormal judgment of pixels and regions in hyperspectral images, and early prediction of diseases can be realized before the disease characteristics appear.

[0030] 3. The universal workflow of the present invention avoids the tedious manual operation and subjective judgment of data analysis, and solves the problem of lack of systematic analysis workflow and automatic model optimization in traditional spectral analysis software. BRIEF DESCRIPTION OF THE DRAWINGS

[0031] Figure 1 is a workflow flowchart of the present invention;

[0032] Figure 2(a) is the spectrum of the mango sinusoidal pattern stripes with a phase offset of 0°, (b) is the spectrum of the mango sinusoidal pattern stripes with a phase offset of 120°, and (c) is the spectrum of the mango sinusoidal pattern stripes with a phase offset of 240°.

[0033] Figure 3 Schematic diagram of extracting the characteristic area of ​​mango lesions with a custom circle center. DETAILED DESCRIPTION

[0034] The present invention is further described in detail below in conjunction with the accompanying drawings and specific implementation methods.

[0035] like Figure 1 The flowchart of the workflow-based method for automated analysis and model optimization of hyperspectral data is shown. The main steps are as follows:

[0036] Step 1: Collect image data of the target sample through a hyperspectral imaging device;

[0037] In the present invention, a structured light illumination mode is used for the image acquisition of the target sample. This structured light illumination mode can control the projection mode of light on the object to enhance the detection of the target features. For each target sample, sinusoidal pattern spectral images with three phases of 0°, 120° and 240° are obtained respectively to ensure that different angles and lighting conditions of the target sample are covered. Compared with traditional hyperspectral images, structured light enhances the penetration of the internal features of the target by modulating the incident angle and intensity of light, and can present the internal structure more clearly. After the multi-phase data of structured light is synthesized, it can provide more complete hyperspectral information. After the image acquisition is completed, these image data are stored in a computer.

[0038] Structured light illumination technology enhances the penetration of light inside an object by modulating the specific pattern of light on the object, and can more deeply detect defects in specific areas inside the object. Structured light modulates the sinusoidal pattern and uses the spectral data of multiple phases to generate structured light high-dimensional images. Compared with ordinary hyperspectral images, it can more clearly present the details and specific defects inside the object. Therefore, hyperspectral imaging combined with structured light illumination can effectively improve the depth and accuracy of detection.

[0039] The target sample number of the collected hyperspectral images is evenly distributed according to the number of classifications.

[0040] Because structured light illumination can better control the penetration of light in the object's tissue, it can enhance the detection of certain depth-specific defects in the object. Structured light illumination imaging modulates the light irradiated on the object according to a specific pattern through a pattern generating device (for example, a projector). Generally, the sinusoidal pattern can be used as the modulation pattern because of its good harmonic characteristics.

[0041] Step 2, performing demodulation and synthesis processing on the collected image data;

[0042] By applying the de-modeling formula, the image data of different phases are synthesized to generate a high-dimensional structured light image (AC) and a uniform illumination image (DC) for each sample. These high-dimensional images contain rich spectral feature information, providing an important data basis for subsequent analysis and model building.

[0043] Step three, for the area to be detected in the spectral image, determine the position and size that need to be paid attention to, and form a feature area; in the present invention, manual annotation or automatic detection algorithm is used to ensure that the feature area is accurate.

[0044] Step 4: extract the image data in the feature area and save it in the form of an array. Each one-dimensional sample data is the hyperspectral band data of a pixel in the feature area.

[0045] This not only facilitates data management and processing, but also ensures data traceability and overall organization. Aiming at the problem of repeated information expression in each characteristic parameter of the hyperspectral spectrum, the present invention performs dimensionality reduction processing on the hyperspectral data of the extracted AC and DC image characteristic areas, extracts the spectral data of a specific band, and then stores it.

[0046] Step 5: Repeat steps 2 to 4 for all target samples to ensure that the feature area of ​​each target sample is accurately extracted and the data in the feature area is stored in a standardized manner. The data is labeled and classified, with the data in the disease-free area labeled 0 and the data in the diseased area labeled 1.

[0047] The data within the characteristic region of all target samples constitute a one-dimensional spectral data set.

[0048] Step 6: Build a one-dimensional convolutional neural network (1D-CNN) classification model - DeepSpNet, which includes several convolutional layers, pooling layers and fully connected layers to process hyperspectral data and output classification results: early or healthy.

[0049] The one-dimensional spectral dataset was randomly divided into a training set and a test set in a ratio of 4:1. The DeepSpNet model was trained with the training set, and the model was optimized by evaluating the classification accuracy, precision, recall and other indicators on the test set. The optimized model was used for the classification and discrimination of hyperspectral images.

[0050] The DeepSpNet model includes multiple one-dimensional convolutional layers (conv1, conv2, conv3, etc.), multiple optional pooling layers, multiple fully connected layers (FC1, FC2, etc.), and one output layer. In order to accelerate the training process of the network and improve the convergence and generalization ability of the model, a one-dimensional batch normalization layer (Batch Normalization, BN layer) is added after each convolution layer, and a BN layer is also added between two fully connected layers. The fully connected layer FC is finally mapped to the classification result output through the softmax function. The convolution, pooling and other calculations in the DeepSpNet model are all performed on one-dimensional data, and the convolution kernel is a one-dimensional weight matrix. The model hyperparameters such as the size, number, and step size of the convolution kernel are optimized and selected according to the characteristics of the data set.

[0051] The BN normalization layer performs a linear transformation on the data so that its mean is 0 and its variance is 1. By normalizing each feature in the data set, the overall distribution of the data set can be made more consistent with the form of a standard normal distribution. For a given feature x, the calculation formula for BN normalization is as follows:

[0052]

[0053] in, is the normalized eigenvalue, is the mean of the feature, is the standard deviation of the feature.

[0054] Step 7: Set the DeepSpNet model parameters;

[0055] Some values ​​of the hyperparameters in the present invention have been confirmed, such as the size of the convolution kernel (kernell), the step size (stirde) of the convolution kernel each time has been set to 1, and the number of epochs for each training is also set to 50 in advance. The following parameters are changed during the training process: learning rate (lr), batch size (batch_size), the number of channels of the convolution layer (channel1, channel2, channel3), the negative slope (negative_slope) of LeakyReLU, and the Dropout value (dp).

[0056] In order to improve the accuracy of model prediction, the manually set hyperparameter values ​​remain unchanged (kernell1=1, kernell2=1, kernell3=1, stride1=1, stride2=1, stride3=1, epochs=50).

[0057] Step 8: Determine the optimal hyperparameters;

[0058] The collected one-dimensional spectral data set is divided into a training set and a test set. The training set is put into the constructed DeepSpNet model, and the automatic search of other hyperparameters is realized through itertools.product and param_grid dictionary.

[0059] Table 1 Model hyperparameters

[0060]

[0061] The hyperparameter space of each hyperparameter in Table 1 is a set of possible values, and each hyperparameter is independent of each other. A hyperparameter combination is established for the set of possible values ​​of all hyperparameters under the combination in Table 1, and each hyperparameter combination is used to train the DeepSpNet model.

[0062] Using classification accuracy as the evaluation index, the model accuracy of each combination of hyperparameters in the hyperparameter space is calculated to determine the optimal hyperparameters.

[0063] Step 9: Train the DeepSpNet model for all hyperparameter combinations;

[0064] After training all hyperparameter combinations, the code will automatically save and verify the DeepSpNet model parameters with the highest accuracy in the test set, which is convenient for subsequent DeepSpNet model deployment and application. Automated search for other hyperparameters is achieved through itertools.product, tolist, and param_grid dictionaries, which makes the model training process no longer rely on manual adjustment of hyperparameters. Automated search can cover a wide range of parameter combinations, increasing the possibility of finding the optimal hyperparameter combination. At the same time, EarlyStopping is defined to stop model training of the current parameters in time when model training is not ideal. This model training method saves a lot of time and computing power, and avoids the need to manually test each hyperparameter combination one by one.

[0065] The code records and displays the loss and accuracy of training and validation after each epoch, providing real-time feedback on model performance. Performance monitoring not only helps developers understand the performance of the model during training, but can also be used for debugging and optimization. By monitoring performance, developers can adjust hyperparameters or model structure when necessary to achieve better training results.

[0066] The set hyperparameters will be combined, and finally the DeepSpNet model training of all hyperparameter combinations will be completed.

[0067] Step 10: output the numerical value of the best hyperparameter combination of the DeepSpNet model, and save the DeepSpNet model trained with the best hyperparameter combination for subsequent disease prediction.

[0068] Step 11: Repeat steps 1 to 4. Import the data of the points to be predicted into the DeepSpNet model, and the model outputs the predicted results of healthy or early stage to achieve the prediction of the disease.

[0069] In this embodiment, the early non-destructive detection of mango anthracnose is taken as an example. The detection of other internal diseases of fruits can refer to the method of this embodiment. The present invention realizes the non-destructive detection of early anthracnose lesions in mangoes through a set of hyperspectral analysis workflows. The main steps are as follows:

[0070] Step 1, select a batch of healthy mangoes as research objects, and inoculate a suspension of Colletotrichum gloeosporioides spores into the surface of the fruit pulp of a part of the mangoes at a specific location by artificial inoculation to obtain diseased fruits, which are left for seven days.

[0071] Step 2: Collect hyperspectral data for all mango samples (with and without inoculation) every 24 hours after inoculation. Based on the imaging mode of structured light illumination, use a spectral camera to shoot the specified object, obtain hyperspectral images of three phases of 0°, 120°, and 240°, name and store them in the computer. The storage content includes the sample code of the mango, the number of storage days, the spectral stripe format, etc. Shoot and extract the hyperspectral raw data file and .har text file in .raw format. The .hdr file is used to describe all metadata information of the data in the .raw file, so that the software can correctly interpret the data in the .raw file.

[0072] The structured light illumination modulates the light irradiated on the mango in a sinusoidal pattern. The modulated sinusoidal pattern is expressed as:

[0073] Where (x, y) is the spatial coordinate; , , , They represent the background or direct component (DC), amplitude component (AC), spatial frequency (along the x-axis) and phase offset respectively.

[0074] Finally, the sinusoidal mode fringe spectra with three different phase offsets of 0°, 120°, and 240° were obtained.

[0075] Step 3, read the spectral data of each mango sample at different phases, perform de-modeling and synthesis processing, and calculate the structured light high-dimensional image (AC) and uniform illumination image (DC) of each mango sample. The formulas are as follows:

[0076] Among them, I 1 ,I 2 ,I 3 They are the three phase images of 0°, 120°, and 240° respectively. Figure 2 As shown in (a), (b), and (c); represents a structured light high-dimensional image, Represents a uniformly illuminated image.

[0077] Step 4: On the AC and DC images of the diseased mango sample after demolding, manually determine the position coordinates and diameter of the lesion feature area to be extracted, such as Figure 3 As shown in the figure, the extracted lesion feature area is a custom circular diameter or rectangular area centered on the selected position coordinates.

[0078] Step 5, based on the location of the lesion feature area obtained in step 4, the AC and DC image data of the same part of the same sample mango in the non-disease period are extracted and used as the healthy data set for modeling.

[0079] Step 6, after extracting the data of the characteristic area of ​​the mango sample inoculated with anthrax, a custom storage path is set and the data is saved in the form of an array table. The extracted data includes AC and DC images of the characteristic area of ​​the mango sample lesion and AC and DC images of the same part in the non-disease period.

[0080] Step 7, repeating steps 3 to 6, storing the high-dimensional data of the characteristic regions of all anthracnose mango samples and disease-free mango samples respectively, forming a one-dimensional spectral data set, which includes the high-dimensional data of the characteristic regions of anthracnose mango samples and the same characteristic regions of disease-free mango samples; each one-dimensional sample data is the high-spectral band data of one pixel in the characteristic region of one mango. At the same time, the data is labeled, the data of the disease-free region is labeled 0, and the data of the diseased region is labeled 1.

[0081] Step 8: Divide the one-dimensional spectral data set obtained in step 7 into a training set and a test set in a ratio of 4:1.

[0082] Step 9: Build and train the 1D-CNN classification model DeepSpNet. The classification results are 0 and 1, where 0 represents healthy mangoes and 1 represents early diseased mangoes.

[0083] Step 10, set the automated hyperparameter search range for the DeepSpNet model, and use itertools.product, tolis, and param_grid dictionaries to implement automated hyperparameter search.

[0084] Step 11: Define EarlyStopping to implement the early stopping function of model training, saving computing power and model training time.

[0085] Step 12: Use the training set to train the DeepSpNet model for each hyperparameter combination, evaluate the classification accuracy, precision, recall and other indicators of the DeepSpNet model on the test set, optimize the model, and determine the optimal hyperparameter combination as shown in Table 2.

[0086] Table 2 Optimal hyperparameter combinations

[0087]

[0088] Step 13: output the value of the best hyperparameter combination, and save the DeepSpNet model trained with the best hyperparameter combination for subsequent disease prediction.

[0089] Step 14, repeat steps 2 and 3. Obtain new individual spectral data, import the data of the points to be predicted into the DeepSpNet model, and the model outputs the predicted results of healthy or early stage to achieve the prediction of the disease.

Claims

1. A workflow-based method for automated analysis and model optimization of hyperspectral data, characterized in that: The steps include: S1, use imaging equipment to obtain the hyperspectral image of the target sample, process the hyperspectral image, extract and save the spectral data of the characteristic area, and construct a one-dimensional hyperspectral data set of all target samples; the implementation steps are as follows: S11, using a hyperspectral imaging device to respectively acquire images of the target sample at three phases of 0°, 120° and 240°, and storing the image data; S12, the collected image data is processed by demodulation and synthesis to obtain a structured light high-dimensional image and a uniform illumination image of the target sample; the calculation formula of the demodulation and synthesis processing is as follows: , Among them, I1, I2, and I3 are the three phase images of 0°, 120°, and 240° respectively; I AC Represents the structured light high-dimensional image, I DC Represents a uniformly illuminated image; S13, after determining a feature area that needs attention on the structured light high-dimensional image and the uniform illumination image, extracting image data of the feature area; S14, saving the image data of the feature area in an array form, wherein each one-dimensional sample data in the array is the hyperspectral band data of a pixel in the feature area; S15, repeatedly executing steps S12 to S14 for all target samples, extracting and storing the structured light high-dimensional image and uniform illumination image of each target sample, and performing label classification on the data to obtain a one-dimensional hyperspectral data set of all target samples; S2, builds DeepSpNet model based on one-dimensional convolutional neural network; S3, set the training parameters of the DeepSpNet model and establish a hyperparameter combination; S4, divide the one-dimensional hyperspectral dataset into a training set and a test set in proportion, use the training set to train the DeepSpNet model, evaluate the accuracy, precision, and recall of the DeepSpNet model through the test set, and automatically select the best hyperparameter combination; S5, saves the DeepSpNet model with the best hyperparameter combination for target prediction.

2. The workflow-based hyperspectral data automated analysis and model optimization method according to claim 1, characterized in that: In step S11, the target sample numbers of the collected hyperspectral images are evenly distributed according to the number of categories.

3. The workflow-based hyperspectral data automated analysis and model optimization method according to claim 1, characterized in that: The DeepSpNet model includes multiple one-dimensional convolutional layers, pooling layers, fully connected layers, and output layers; a one-dimensional batch normalization layer is added after each convolutional layer, and a normalization layer is added between the fully connected layers; the fully connected layer corresponds to the classification result output through the softmax function.

4. The workflow-based hyperspectral data automated analysis and model optimization method according to claim 1, characterized in that: In step S3, the size of the convolution kernel, the step size of each convolution kernel advance, and the number of rounds of each training are preset; the hyperparameters include: learning rate, batch size, number of channels of the convolution layer, negative slope of LeakyReLU, and Dropout value; the hyperparameter combination is a set of possible values ​​of all hyperparameters.

5. The workflow-based hyperspectral data automated analysis and model optimization method according to claim 1, characterized in that: In step S4, during the training of the DeepSpNet model, other hyperparameters are automatically searched through itertools.product and param_grid dictionaries; the early stopping mechanism is introduced by defining EarlyStopping; and the training progress is monitored and recorded in real time.

Images