A method for identifying the origin of traditional Chinese medicine based on LIBS spectrum and image fusion

Abstract

The present invention provides a method for identifying the origin of Chinese herbal medicine slices based on fusion of LIBS spectrum and image features, the method comprising: extracting features from LIBS spectrum based on the RC value of the PLS-DA model, extracting features from LIBS spectrum and LIBS image using a convolutional neural network, fusing the extracted spectral features and image features at the feature layer and at the decision layer, placing the fused features into a classifier for classification, and obtaining a result of identifying the origin of Chinese herbal medicine slices based on atlas fusion. The present invention combines LIBS with multivariate information fusion technology, and through the classification of the origins of two herbal medicine slices, white peony root and white angelica root, it is confirmed that it has great potential in the field of Chinese herbal medicine identification.

Description

Technical Field

[0001] The present invention relates to the field of Chinese medicinal material origin identification, and in particular to a method for Chinese medicinal material origin classification based on LIBS spectrum and image fusion. Background Art

[0002] Modern research shows that Chinese herbal medicines have anti-inflammatory, analgesic, liver-protecting, and antioxidant effects, and are widely used in clinical practice, with broad development prospects. The chemical composition of Chinese herbal medicines is complex and diverse, and their main chemical substances differ in exerting their efficacy and treating diseases. The quality of herbal medicines is closely related to their origin. The content of various components in Chinese herbal medicines directly affects the exertion of their pharmacological effects, and the content of the components is affected by the origin.

[0003] There are many traditional methods for identifying the origin of Chinese herbal medicines. However, due to factors such as complex sample preparation procedures, high sample loss, poor identification efficiency, and low degree of automation, rapid detection has not yet been achieved in the practice of identifying the origin of Chinese herbal medicines. Laser-Induced Breakdown Spectroscopy (LIBS) technology forms plasma by focusing ultrashort pulse lasers on the sample surface, and then analyzes the plasma emission spectrum to determine the material composition and content of the sample. LIBS has the advantages of simple sample preparation, low sample loss, high detection efficiency, and the ability to achieve online detection. It has been widely used in clinical medicine, archaeology, industry and other fields.

[0004] The paper "Distinguish Fritillaria cirrhosa and non-Fritillariacirrhosa using laser-induced breakdown spectroscopy" by Wei et al. proposed the use of LIBS combined with learning vector quantization to distinguish between Fritillaria cirrhosa and non-Fritillaria cirrhosa, and proved the feasibility of LIBS in the classification of traditional Chinese medicines through experiments.

[0005] The paper "Geographical authenticity evaluation of Mentha haplocalyx by LIBS coupled with multivariate analyzes" by Liu et al. uses LIBS combined with principal component analysis and least squares support vector machine to classify and identify mint, and classifies Chinese medicinal materials through machine learning methods.

[0006] The above methods illustrate the application prospects and great potential of LIBS in the field of Chinese medicinal materials classification, but their research only focuses on LIBS spectra and has not yet involved LIBS images, making recognition difficult and the accuracy low. Summary of the invention

[0007] The purpose of the present invention is to propose a method based on LIBS spectrum and image fusion to achieve rapid identification of the origin of Chinese herbal medicine pieces and improve the accuracy of origin identification.

[0008] To achieve the above object, the present invention provides the following technical solutions:

[0009] A method for identifying the origin of Chinese herbal medicine pieces based on LIBS spectrum and image fusion, comprising the following steps:

[0010] Step 1: Use the RC value based on partial least squares discriminant analysis (PLS-DA) to perform feature screening on LIBS spectral data to obtain the screened spectral features;

[0011] Step 2: Using a convolutional neural network to extract features from LIBS spectral data and LIBS image data, and obtaining spectral features and image features after feature extraction;

[0012] Step 3: Fusing the spectral features and image features at the feature level and decision level;

[0013] Step 4: Put the fused features into different classifiers for classification;

[0014] Step 5: Obtain classification results based on fusion features.

[0015] Furthermore, the RC value based on PLS-DA is used to extract LIBS spectral features in step 1 as follows: input spectral data of size (m×n), denoted as X; manually add response value Y to the spectral data, where Y is a (m×1) matrix; establish a PLS-DA classification model to obtain the corresponding wavelengths (λ1, λ2, λ3…λ m )'s RC value (w1.w2,w3...w m ), and the characteristic wavelengths of the LIBS spectrum are screened out by comparing the RC values.

[0016] Furthermore, in step 2, the residual network (ResNet18, ResNet50) in the convolutional neural network is selected as the network for extracting spectral and image features; after the input data is subjected to convolution, batch normalization, activation, and pooling operations, the features of the last fully connected layer are selected as the extracted features.

[0017] Furthermore, the LIBS image data in step 2 is the overall image of the Chinese herbal medicine slices after laser dotting, and the image is captured by a camera in the LIBS system through an autofocus program, and the autofocus program uses a hill climbing method as a search function.

[0018] Further, the specific steps for obtaining the image data are as follows: 1. Obtain the image sharpness value at the current position and denote it as Ten0; 2. Move the X-Y-Z sample stage for placing the sample along the specified Z direction by a given step size s0, and obtain the current image sharpness Ten1. If Ten1 > Ten0, move in the original direction by the same step size s0. Otherwise, multiply the step size by the convergence coefficient p to obtain the converged step size s1 = p × s0; 3. Compare the size of the converged step size s1 with the given threshold t. If s1 < t, the focusing is completed. If s1 > t, the sample stage moves in the reverse direction by the converged step size s1, and repeat steps 2-3 until the focusing is completed.

[0019] Further, the Ten value of the image sharpness is calculated using the Tenengrad function, and the Tenengrad function uses the Sobel operator to extract the gradient values in the horizontal and vertical directions. The specific process is as follows: 1. Let the sober convolution kernel be G x ,G y , then the gradient of the image I at the point (x, y) is Define the Ten value of this image as n is the total number of pixels in the image.

[0020] Further, in step 3, the extracted features are normalized respectively to obtain spectral data with a size of (m1×1) and image data with a size of (m2×1). The two are concatenated to obtain features with a size of ((m1 + m2)×1), and the concatenated features are used as the final fusion features and input into different classifiers for classification respectively.

[0021] Further, the decision-level fusion in step 3 is to input different modal information into the trained classifiers respectively to output scores or decisions for fusion

[0022] Further, the decision-level fusion is specifically as follows: Put the spectral and image features into their respective classifiers for training. The probability of each origin for the output of the spectral classifier is (P1, P2, P3), and the probability of each origin for the output of the image classifier is (K1, K2, K3). Calculate their arithmetic mean Take the class with the highest arithmetic mean as the final decision on the origin attribution of the Chinese herbal pieces.

[0023] Beneficial effects: The present invention first performs data fusion on LIBS spectral information and LIBS image information, and establishes a model based on the fusion data to identify the origin of Chinese herbal pieces. By comprehensively integrating information from different sources, it realizes multi-angle analysis of the same sample, further improves the accuracy of LIBS classification, and effectively improves the accuracy rate of identifying the origin of Chinese medicinal materials. Description of the Drawings

[0024] In order to better illustrate the technical solutions, specific implementation steps and experimental results of the present invention, the following briefly introduces the drawings required in the implementation process.

[0025] Figure 1 The figure is a detailed flow chart of the origin identification of Chinese herbal medicine slices based on LIBS spectrum fusion in the present invention.

[0026] Figure 2 This is a schematic diagram of an average spectrum example of a white peony sample in the dataset.

[0027] Figure 3 This is a schematic diagram of an average spectrum example of a Angelica dahurica sample in the data set.

[0028] Figure 4 It is the LIBS image of the white peony root sample in the dataset.

[0029] Figure 5 It is the LIBS image of the Chinese herbal medicine Angelica dahurica sample in the dataset.

[0030] Figure 6 It is a schematic diagram of the fusion process of the spectral features extracted by ResNet18 and the image features extracted by ResNet50 at the feature level.

[0031] Figure 7 It is a schematic diagram of the fusion process of spectral information and image information decision layer. DETAILED DESCRIPTION

[0032] The embodiments of the present invention are described in detail below with reference to the accompanying drawings.

[0033] The purpose of the present invention is to propose a method for identifying the origin of Chinese herbal medicine slices based on LIBS spectrum and image fusion.

[0034] In order to make the above-mentioned objects, features and advantages of the present invention more clearly understood, the present invention is further described in detail below with reference to pictures and examples.

[0035] As shown in the figure, the method for identifying the origin of traditional Chinese medicine based on LIBS spectrum and image fusion of the present invention has the following specific steps:

[0036] Step 1, obtain LIBS spectral data and image data. The LIBS instrument detects the surface of the Chinese herbal medicine sample according to the preset 4×4 dot matrix. A total of 16 positions are detected, and each point is hit nine times. The first four times are for sample surface cleaning, and the spectrum obtained by the last five hits is used for specific research. This experiment was carried out under normal pressure conditions. The parameters of the LIBS instrument need to be adjusted before use. The laser pulse energy is 100mJ, the pulse frequency is 1Hz, the acquisition delay time is 2μs, and the integration time is 10μs. This acquisition method can further collect more sample surface information and reduce the influence of other factors. The acquired LIBS spectral data needs to be preprocessed. The preprocessing methods include area normalization and spectral averaging. Area normalization is to divide the intensity of the spectrum by the integrated area of ​​the spectrum. Spectral averaging is to average the spectra of the last five times as the spectrum of a point, and then average the spectra of 16 points as the final spectrum of a sample. Spectral preprocessing can reduce signal noise and improve model performance. The obtained LIBS image is the full image of the sample after laser ablation, and the image format is grayscale. The image after ablation contains more information than the image before ablation. The volume, area, and depth of the ablation pit have a certain impact on the result of origin identification. The Chinese medicinal materials used in the experiment here are white peony root and white angelica root. Both samples come from three different origins, marked as 0, 1, and 2 respectively.

[0037] Step 2: Use the regression coefficient value RC based on PLS-DA to perform feature screening on the LIBS spectrum. The RC value is used to measure the relationship between the variable and the response value. The variable with a larger RC value has a greater contribution to the model. First, a PLS-DA classification model is established for the spectral data, and the parameters are adjusted to obtain the optimal model. The results of different wavelengths (λ1, λ2, λ3...λ m ) corresponding to the RC value (w1.w2,w3...w m ). Comparing the size of the RC values ​​corresponding to different wavelengths, for white peony, 37 characteristic wavelengths and the peak intensities of the two adjacent wavelengths before and after were selected, a total of 111 features, for Angelica dahurica, 30 characteristic wavelengths and the peak intensities of the two adjacent wavelengths before and after were selected, a total of 90 features, and the selected spectral characteristic wavelengths are shown in Table 1 and Table 2:

[0038]

[0039] Table 1: Characteristic wavelengths of white peony spectrum screened by PLS-DA.

[0040] Note: I represents the atomic spectral line of an element; II represents the primary ion spectral line of an element; - represents that the characteristic spectral line has no corresponding element.

[0041]

[0042] Table 2: Characteristic wavelengths of Angelica dahurica spectrum screened by PLS-DA.

[0043] Note: I represents the atomic spectral line of an element; II represents the primary ion spectral line of an element; - represents that the characteristic spectral line has no corresponding element.

[0044] Step 3: Use ResNet18 to extract spectral features. The initial learning rate of the model is 0.0001, the optimizer is the SGDM optimizer, the number of iterations is 500, and the mini-batch size is 50. Input the spectral data into the ResNet18 network, use the convolution operation to extract high-dimensional features, and use the 256 features output by the last fully connected layer as the spectral fusion features.

[0045] Step 4: Use ResNet50 to extract image features. The initial learning rate of the model is 0.0001, the optimizer is Adam optimizer, the number of iterations is 300, and the mini-batch size is 8. ResNet50 is used to extract the high-dimensional information of the image, and the 512 features output by the last fully connected layer are used as image fusion features.

[0046] Step 5: Fuse the spectral features extracted by PLS-DA and ResNet18 and the image features extracted by ResNet50 at the feature layer. First, normalize the two types of features to obtain white peony spectrum data with sizes of (111×1) and (256×1), white peony spectrum data with sizes of (90×1) and (256×1), and white peony and white peony image data with sizes of (512×1). The spectral and image features of the two types of Chinese medicinal materials are spliced ​​to obtain white peony fusion features with sizes of (623×1) and (768×1) and white peony fusion features with sizes of (602×1) and (768×1). Input the fusion features into different classifiers to obtain the results of feature layer fusion modeling. The classifiers used here are support vector machines (SVM), PLS-DA and ResNet18.

[0047] Step 6. Put the spectral features and image features extracted above into different classifiers for modeling. The classifiers used for spectral feature modeling are PLS-DA, SVM and ResNet18, and the classifier used for image feature modeling is ResNet50. Here, the decision layer fusion of PLS-DA and ResNet50 of white peony sample is used as an example to illustrate: the probabilities of the three categories of different origins (0, 1, 2) output by the spectral classifier PLS-DA are (P1, P2, P3), and the probabilities of the three categories of different origins (0, 1, 2) output by the image classifier ResNet50 are (K1, K2, K3). Calculate the arithmetic mean of the probabilities of the three categories of different origins (0, 1, 2) of the spectral classifier and the image classifier The category with the highest arithmetic mean is taken as the final result of the decision-making layer fusion PLS-DA-ResNet50 white peony root slice origin attribution. and The final classification result is origin 0. The final result is randomly selected from category 0 and category 1.

[0048] Step 7: Obtain the origin recognition results of white peony root and white angelica root based on feature fusion. Compared with single features, the model established based on fusion features has a higher recognition accuracy. The specific results are shown in Tables 3 and 4:

[0049]

[0050] Table 3: Identification results of origin of white peony root.

[0051] Note: IMG: image features of the complete sample image after ablation RC: spectral features filtered out according to the RC value of the PLS-DA model ResNet18: spectral features filtered out according to the ResNet18 model ResNe50: image features filtered out according to ResNet50.

[0052]

[0053] Table 4: Identification results of the origin of Angelica dahurica.

[0054] Note: IMG: image features of the complete sample image after ablation RC: spectral features filtered out according to the RC value of the PLS-DA model ResNet18: spectral features filtered out according to the ResNet18 model ResNe50: image features filtered out according to ResNet50.

Claims

1. A method for identifying the origin of Chinese herbal medicine pieces based on LIBS spectrum and image fusion, characterized in that It includes the following steps: Step 1: Use the RC value based on the partial least squares discriminant analysis method PLS-DA to screen the features of the LIBS spectral data, and obtain the screened spectral features; Step 2: Use a convolutional neural network to extract the features of the LIBS spectral data and the LIBS image data, and obtain the spectral features and image features after feature extraction; Step 3: Fuse the spectral features and image features at the feature level and the decision level; Step 4: Put the fused features into different classifiers for classification; Step 5: Obtain the classification result based on the fused features; The specific steps of extracting LIBS spectral features using RC values ​​based on PLS-DA in step 1 are as follows: input spectral data of size (m×n), denoted as X; manually add response value Y to the spectral data, where Y is a (m×1) matrix; establish a PLS-DA classification model, and obtain the corresponding spectral data of different wavelengths (λ1, λ2, λ3, ... λ m )'s RC value (w1.w2,w3...w m ), the characteristic wavelengths of LIBS spectra are screened out by comparing the RC values; The fusion at the decision level in step 3 is to input different modal information into the trained classifiers respectively, and output the scores or decisions for fusion; the fusion at the decision level is specifically as follows: put the spectrum and image features into their respective classifiers for training, and obtain the probability of each category of origin output by the spectrum classifier as (P1, P2, P3), and the probability of each category of origin output by the image classifier as (K1, K2, K3), and calculate their arithmetic mean The category with the highest arithmetic mean is taken as the final decision on the origin of the Chinese herbal medicine pieces.

2. The method for identifying the origin of traditional Chinese medicine decoction pieces based on the fusion of LIBS spectrum and image according to claim 1, wherein: In step 2, the residual network in the convolutional neural network is selected as the network for extracting spectral and image features; after the input data undergoes convolution, batch normalization, activation, and pooling operations, the features of the last fully connected layer are selected as the extracted features.

3. The method for identifying the origin of traditional Chinese medicine decoction pieces based on the fusion of LIBS spectrum and image according to claim 1, wherein: In step 2, the LIBS image data is the overall image of the traditional Chinese medicine decoction pieces after laser dotting, and the image is obtained by the camera in the LIBS system through an autofocus program, and the autofocus program uses the hill climbing method as the search function.

4. The method for identifying the origin of traditional Chinese medicine decoction pieces based on the fusion of LIBS spectrum and image according to claim 1, wherein: The specific steps for obtaining the image data are as follows:

1. Obtain the image sharpness value at the current position and record it as Ten0; 2. Move the X-Y-Z sample stage for placing the sample along the specified Z direction by a given step size s0, and obtain the current image sharpness Ten1. If Ten1 > Ten0, move in the original direction by the same step size s0, otherwise multiply the step size by the convergence coefficient p to obtain the converged step size s1 = p×s0; 3. Compare the size of the converged step size s1 with the given threshold t. If s1 < t, the focusing is completed. If s1 > t, the sample stage moves in the reverse direction by the converged step size s1, and repeat steps 2-3 until the focusing is completed.

5. The method for identifying the origin of traditional Chinese medicine decoction pieces based on the fusion of LIBS spectrum and image according to claim 4, wherein: The image clarity Ten value is calculated using the Tenengrad function, which uses the Sobel operator to extract the gradient values ​​in the horizontal and vertical directions. The specific process is as follows:

1. Let the Sobel convolution kernel be G x ,G y , then the gradient of image I at point (x, y) is Define the Ten value of the image as n is the total number of pixels in the image.

6. The method for identifying the origin of traditional Chinese medicine decoction pieces based on the fusion of LIBS spectrum and image according to claim 1, wherein: In step 3, the extracted features are respectively normalized to obtain spectral data with a size of (m1×1) and image data with a size of (m2×1), and the two are spliced to obtain features with a size of ((m1 + m2)×1). The spliced features are used as the final fused features and are respectively input into different classifiers for classification.

Images