A hyperspectral rod recognition method based on incremental learning

Through the hyperspectral rod recognition method based on incremental learning, the feature center is obtained by using convolutional neural networks to solve the problems of low rod recognition efficiency and cumbersome switching in the existing technology, and achieve efficient and accurate Identification and training process.

CN116778217BActive Publication Date: 2024-11-22NANJING FORESTRY UNIV
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202310117786.4
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2023-02-15
Publication Date
2024-11-22
Estimated Expiration
2043-02-15

AI Technical Summary

Technical Problem

The prior art has huge differences in spectral characteristics due to differences in the variety of origin when training and identifying models, resulting in low rod recognition efficiency and cumbersome switching of multiple models, which increases production costs and reduces efficiency.

Method used

The hyperspectral rod-like object recognition method based on incremental learning is adopted to obtain the feature centers of different substances through convolutional neural networks, and the incremental learning is used to identify and learn objects to be recognized to improve recognition accuracy and training efficiency. The specific steps include collecting hyperspectral images, calculating feature centers, determining the category of substances to be identified and updating the feature centers, and creating a new category.

Benefits of technology

It improves the recognition accuracy of different types of rod-shaped objects, saves time in training models, reduces production costs, simplifies the model switching process, and improves training efficiency and recognition accuracy.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN116778217B_ABST
    Figure CN116778217B_ABST
Patent Text Reader

Abstract

The present invention discloses a method for recognizing rod-shaped objects based on a hyperspectral image and incremental learning, and belongs to the field of hyperspectral imaging and deep learning. According to the hyperspectral image and the category label, each category is first calculated by a neural network to obtain the confidence level and characteristic information of different categories of substances, and the characteristic center of the substance is calculated. New spectral data is input into the trained neural network model, and the category of the unknown substance is determined to be a known or unknown category based on the distance between the characteristic information and the characteristic center of each type of substance. Finally, the characteristic center of the object is updated and a new category is created by incremental learning, thereby improving the recognition accuracy and training efficiency of the network for unknown rod-shaped objects. The present invention uses the output of the convolutional layer of the convolutional neural network as the characteristic information, and the output of the fully connected layer as the confidence level, determines the characteristic center of the object, provides a standard for subsequent recognition and learning, and improves the accuracy of spectral image recognition based on deep learning.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention belongs to the field of hyperspectral imaging and deep learning technology, and specifically relates to a hyperspectral rod-shaped object recognition method based on incremental learning. Background Art

[0002] In the field of rod-shaped object identification, there are often many unknown substances that are unknown when training the identification model, including some rod-shaped objects with huge spectral characteristics due to differences in origin and variety. It takes a long time and is costly to train a new identification model suitable for new and old rod-shaped objects based on the new rod-shaped object data. In actual production, multiple different models are often trained to be applicable to different types of rod-shaped objects. This can certainly solve the problem of some rod-shaped objects being unable to be identified, but it takes a long time to train a neural network model, and the switching of multiple different models is too cumbersome, which is inconvenient in industrial production, increases the cost of production, and reduces production efficiency.

[0003] Existing technologies usually use anomaly detection to identify unknown material types, but the general usage scenarios of anomaly detection are mostly defect detection, such as identifying defects on workpieces. It is impossible to accurately identify a single discrete object, nor can it distinguish between two or more unknown categories of objects of different categories. Anomaly detection can only identify known categories and unknown categories, and cannot further distinguish unknown categories.

[0004] Unknown material types can be identified in the production and identification process through incremental learning. The unknown categories and the improvement of the data model of known categories can effectively improve the recognition accuracy of different types of rods, save the time spent on training models, and reduce the cost of the production process. Therefore, building a hyperspectral rod recognition method based on incremental learning is of great significance for improving the recognition accuracy and training efficiency of different types of rods. Summary of the invention

[0005] The technical problem solved by the present invention is to provide a hyperspectral deep learning recognition method for rod-shaped objects, which first uses a convolutional neural network to obtain the characteristic centers of different substances; then uses incremental learning to recognize and learn the objects to be recognized, thereby improving recognition accuracy and training efficiency.

[0006] Technical solution: In order to solve the above problems, the technical solution adopted by the present invention is as follows:

[0007] A hyperspectral rod-shaped object recognition method based on incremental learning, based on hyperspectral images and category labels, firstly calculates each category through a neural network to obtain the confidence level and characteristic information of different categories of substances, and calculates the characteristic center of the substance through its confidence level and characteristic information; inputs new spectral data into the trained neural network model, and determines whether the category of the unknown substance is known or unknown based on the distance between the characteristic information and the characteristic center of each substance; finally, updates the characteristic center of the object and creates a new category through incremental learning. The following steps are included:

[0008] S1: Collect hyperspectral images of different substances, correct and reduce noise of the data, and annotate the spectral data;

[0009] S2: Put the spectral data and category labels into the neural network for training to obtain the characteristic centers of different categories of substances;

[0010] S3: Put the hyperspectral image of the substance to be identified into the trained neural network model, and determine whether the category of the substance to be identified is known or unknown based on the feature information;

[0011] S4: For substances whose identified categories are unknown, new categories are created.

[0012] Furthermore, the specific implementation method of step S2 is as follows:

[0013] S21: First, build a convolutional neural network model. The convolutional neural network model contains 2 convolutional layers, 1 pooling layer, and 2 fully connected layers, and contains two outputs;

[0014] S22: Input the spectral data and category labels into the neural network for training and output the characteristic information of each category and confidence level , the feature information expression is as follows:

[0015] ;

[0016] In the formula, x j represents the feature information of the jth dimension, Indicates the substance of category c l Characteristic information of a substance;

[0017] S23: Taking the confidence level as a weight, weighted averaging is performed on the characteristic information to obtain the characteristic center of each category of substances.

[0018] Furthermore, in step S23, the characteristic center of each category of substances is calculated as follows:

[0019] ;

[0020] Where, T C represents the characteristic center of substances in category c, and n represents the number of substances in category c.

[0021] Furthermore, in step S3, the category of the substance to be identified is determined using the following method:

[0022] S31, collecting a hyperspectral image of the substance to be identified, and correcting and reducing noise on the data;

[0023] S32, putting the corrected data into the neural network model trained in step S2, obtaining the characteristic information and confidence level of the substance to be identified, and calculating its characteristic position. The calculation formula is as follows:

[0024] L=fX ;

[0025] In the formula, L is the feature position, f is the confidence level, and X is the feature information;

[0026] S33, calculating the multimodal confidence variable moments between the characteristic positions of the substance to be identified and the characteristic centers of each known category;

[0027] S34, comparing the calculated multimodal confidence variable moment with a threshold value to determine the category of the substance.

[0028] S35, the substances that meet the comparison conditions are classified as known categories and step S32 is repeated to update the characteristic center of the substances. When the comparison conditions of multiple categories are met, it is handed over to manual judgment.

[0029] Furthermore, in step S33, the calculation formula of the multimodal confidence variable moment is as follows:

[0030] ;

[0031] Where D is the multimodal confidence variable moment, T C The covariance matrix of .

[0032] Furthermore, in step S34, the multimodal confidence variable moment is compared with the threshold under the following conditions:

[0033] ;

[0034] Furthermore, in step S4, the unknown category substances are processed as follows:

[0035] S41, when the category is unknown substance, its unknown substance characteristic information X q and the confidence level f for the unknown substance q Store in database;

[0036] S42, counting the number of unknown substances in the database, if it is greater than 100, proceed to step S43, if it does not meet the condition, continue to step S3;

[0037] S43, taking an appropriate amount of data information from the database and calculating its elastic discrete value.

[0038] S44, repeat step S43 until the elastic discrete value S is less than 1. After completion, the acquired data are of the same category and do not belong to any known category, that is, incremental learning is completed and the recognition of new categories can be realized.

[0039] Furthermore, in step S43, the formula for calculating the elastic discrete value is as follows:

[0040] ;

[0041] In the formula, k is the sample size, q Indicates q samples, is the average of the data.

[0042] Beneficial effects: Compared with the prior art, the advantages of the present invention include:

[0043] (1) The present invention uses the output of the convolutional layer of the convolutional neural network as feature information and the output of the fully connected layer as the confidence level, determines the feature center of the object, provides a standard for subsequent recognition and learning, and improves the accuracy of spectral image recognition based on deep learning.

[0044] (2) In the process of determining the object category, the present invention utilizes feature space information and uses multimodal confidence variable moments and threshold judgments to further constrain the prediction results based on the convolutional neural network, thereby improving the accuracy of the prediction.

[0045] (3) The present invention uses an incremental learning approach to construct a data set of unknown material categories, learns during the recognition process, and constrains the same unknown category through its elastic discrete value, thereby simplifying the process of adding a new data set and optimizing the model, improving the recognition accuracy of unknown category substances, and improving training efficiency; saving model training time for new categories, and improving the model's recognition accuracy for multiple different categories of rod-shaped objects. BRIEF DESCRIPTION OF THE DRAWINGS

[0046] Figure 1 It is a convolutional neural network model;

[0047] Figure 2 This is a partially annotated hyperspectral data legend;

[0048] Figure 3It is the characteristic information of some rods output by the convolutional neural network;

[0049] Figure 4 is the original image and the predicted result image;

[0050] Figure 5 are the original image with plastic film and the predicted result image. DETAILED DESCRIPTION

[0051] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, the specific implementation methods of the present invention are described in detail below in conjunction with specific embodiments.

[0052] The hyperspectral rod recognition method based on incremental learning of the present invention mainly includes the following steps:

[0053] Step 1: Collect some hyperspectral images of different substances, correct and reduce noise, and annotate the spectral data. Some annotated hyperspectral data are as follows: Figure 2 As shown, the two marked substances in the figure are two different types of rod-shaped objects;

[0054] The reflectance spectrum images of rod-shaped objects in the range of 400 to 1000 nm were obtained using the FX10E series hyperspectral imager of SPECIM of Finland;

[0055] Step 2: Put the spectral data and category labels into the neural network for training to obtain the characteristic centers of different categories of substances. The specific implementation steps are as follows:

[0056] Step 21: First, build a convolutional neural network model, which includes 2 convolutional layers, 1 pooling layer, and 2 fully connected layers, and contains two outputs. The structure is as follows Figure 1 as shown.

[0057] Step 22: Input the spectral data and category labels into the neural network for training and output the characteristic information of each category and confidence level , some feature information such as Figure 3 As shown, the feature information expression is as follows:

[0058] ;

[0059] In the formula, x j represents the feature information of the jth dimension, Indicates the substance of category c l Characteristic information of a substance.

[0060] Step 23: Use the confidence level as the weight to perform weighted averaging on the feature information to obtain the feature center of each category of substance. The calculation method is as follows:

[0061] ;

[0062] In the formula, T C represents the characteristic center of substances of category c, and n represents the number of substances of category c;

[0063] Step 3: Place the hyperspectral image of the substance to be identified into the trained neural network model, and determine whether the category of the substance to be identified is known or unknown based on the feature information. The specific method for determination is as follows:

[0064] Step 31, collecting a hyperspectral image of the substance to be identified, and correcting and reducing noise on the data;

[0065] Step 32, put the corrected data into the neural network model trained in step 2, obtain the characteristic information and confidence level of the substance to be identified, and calculate its characteristic position. The calculation formula is as follows:

[0066] L=fX ;

[0067] Where L is the feature position, f is the confidence level, and X is the feature information.

[0068] Step 33, calculate the multimodal confidence variable moment between the characteristic position of the substance to be identified and the characteristic center of each known category, and the calculation formula is as follows:

[0069] ;

[0070] Where D is the multimodal confidence variable moment, T C The covariance matrix of

[0071] Step 34, compare the calculated multimodal confidence variable moment with the threshold to determine the category of the substance, and the comparison conditions are as follows:

[0072] .

[0073] In step 35, the substances that meet the comparison conditions are classified as known categories and step 32 is repeated to update the characteristic center of the substances. When the comparison conditions of multiple categories are met, it is handed over to manual judgment.

[0074] Step 4: For substances whose categories are unknown, a new category is created. The specific steps for creating a new category are as follows:

[0075] Step 41, when the category is unknown substance, the unknown substance characteristic information Xq and the confidence level f for the unknown substance q Store in database;

[0076] Step 42, counting the number of unknown substances in the database, if it is greater than 100, proceed to step S43, if it does not meet the condition, continue to step 3;

[0077] Step 43, take a proper amount of data information from the database and calculate its elastic discrete value. The calculation formula is as follows:

[0078] ;

[0079] In the formula, k is the sample size, q Indicates q samples, is the average of the data.

[0080] Step 44, repeat step 43 until the elastic discrete value S is less than 1. After completion, the acquired data are of the same category and do not belong to any known category, that is, incremental learning is completed, and the recognition of new categories can be realized.

[0081] like Figure 4 As shown, from left to right in the figure are the visualization of the spectral graph and the recognition results of new rods of unknown categories predicted by incremental learning. It can be seen that although the initial model does not have the spectral information of this kind of rod, it is not misidentified as any known category, and can also be accurately identified as the same unknown category.

[0082] like Figure 5 As shown in the figure, in the initial model, only the category data of rods, plastic sheets, cigarette butts, and hemp ropes were added, but a piece of plastic film was still identified as the same category as a whole, and was not misidentified as a plastic sheet with similar features. This shows that this method is more accurate in identifying unknown categories and has good practical value.

[0083] The above is only a preferred embodiment of the present invention. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principle of the present invention. These improvements and modifications should also be regarded as the scope of protection of the present invention.

Claims

1. A hyperspectral rod recognition method based on incremental learning, characterized in that: According to the hyperspectral image and category label, each category is first calculated through the neural network to obtain the confidence level and characteristic information of different categories of substances, and the characteristic center of the substance is calculated through its confidence level and characteristic information; new spectral data is input into the trained neural network model, and the category of the unknown substance is determined to be known or unknown based on the distance between the characteristic information and the characteristic center of each substance; finally, the characteristic center of the object is updated and a new category is created through incremental learning, including the following steps: S1: Collect hyperspectral images of different substances, correct and reduce noise of the data, and annotate the spectral data; S2: Put the spectral data and category labels into the neural network for training to obtain the characteristic centers of different categories of substances; the specific implementation method is as follows: S21: First, build a convolutional neural network model. The convolutional neural network model contains 2 convolutional layers, 1 pooling layer, and 2 fully connected layers, and contains two outputs; S22: Input the spectral data and category labels into the neural network for training and output the characteristic information of each category and confidence level , the feature information expression is as follows: ; In the formula, x j represents the feature information of the jth dimension, Indicates the substance of category c l Characteristic information of a substance; S23: Take the confidence level as the weight and perform weighted average on the characteristic information to obtain the characteristic center of each category of substances; the characteristic center of each category of substances is calculated as follows: ; In the formula, T C represents the characteristic center of substances of category c, and n represents the number of substances of category c; S3: Put the hyperspectral image of the substance to be identified into the trained neural network model, and determine whether the category of the substance to be identified is known or unknown through the feature information; determine the category of the substance to be identified, and use the following method to achieve: S31, collecting a hyperspectral image of the substance to be identified, and correcting and reducing noise on the data; S32, putting the corrected data into the neural network model trained in step S2, obtaining the characteristic information and confidence level of the substance to be identified, and calculating its characteristic position. The calculation formula is as follows: L=fX ; In the formula, L is the feature position, f is the confidence level, and X is the feature information; S33, calculating the multimodal confidence variable moment between the characteristic position of the substance to be identified and the characteristic center of each known category; the calculation formula of the multimodal confidence variable moment is as follows: ; Where D is the multimodal confidence variable moment, T C The covariance matrix of S34, comparing the calculated multimodal confidence variable moment with a threshold value to determine the category of the substance; S35, the substances that meet the comparison conditions are classified as known categories and step S32 is repeated to update the characteristic center of the substances. When the comparison conditions of multiple categories are met, it is handed over to manual judgment; S4: For substances whose identified categories are unknown, new categories are created.

2. The method for rod recognition based on hyperspectral incremental learning according to claim 1, characterized in that: In step S34, the multimodal confidence variable moment is compared with the threshold under the following conditions: 。 3. The method for rod recognition based on hyperspectral incremental learning according to claim 1, characterized in that: In step S4, the unknown category substances are processed as follows: S41, when the category is unknown substance, the characteristic information of the unknown substance X q and unknown substance confidence level f q Stored in the database, q Indicates q samples; S42, counting the number of unknown substances in the database, if it is greater than 100, proceed to step S43, if it does not meet the condition, continue to step S3; S43, taking an appropriate amount of data information from the database and calculating its elastic discrete value; S44, repeat step S43 until the elastic discrete value S is less than 1. After completion, the acquired data are of the same category and do not belong to any known category, that is, the incremental learning is completed and the recognition of the new category is realized.

4. The method for rod recognition based on hyperspectral incremental learning according to claim 3, characterized in that: In step S43, the formula for calculating the elastic discrete value is as follows: ; In the formula, k is the sample size, q Indicates q samples, is the average of the data.