Rice panicle setting rate measuring method based on deep learning and map fusion

Abstract

The invention provides a rice panicle setting rate measuring method based on deep learning and map fusion. According to the method, an end-to-end prediction process is constructed through the steps of hyperspectral image acquisition, hyperspectral image labeling, data preprocessing, extraction of reflection data of rice grains, feature extraction, classification model establishment and the like on the rice grains. Firstly, depth features of rice grains are extracted through SS-CNN, then classification prediction is performed by using an SVM model, and finally accurate prediction of the rice setting rate is realized. The deep learning model in the research effectively connects hyperspectral data preprocessing and feature extraction, and the generalization ability of the model and the stability in practical application are ensured by comparing a prediction result with an actual setting rate.

Description

Technical Field

[0001] The present application relates to agricultural hyperspectral imaging technology combined with machine learning and deep learning applications, and in particular to a rice fruit set rate prediction method based on hyperspectral image processing and deep learning models, and belongs to the cross-technical field of agricultural intelligence, precision agriculture, and crop phenotyping analysis. Background Art

[0002] Rice is an important food crop in the world, and its yield and quality are directly related to food security. The rice fruiting rate is one of the important indicators to measure rice yield. Traditional rice fruiting rate assessment relies on manual measurement and observation, which is not only time-consuming and labor-intensive, but also has a high degree of subjectivity. With the development of intelligent agriculture, intelligent assessment methods based on hyperspectral imaging technology and deep learning models provide an efficient and accurate means for predicting rice fruiting rate.

[0003] Hyperspectral imaging technology is a remote sensing technology that integrates images and spectra. It can capture the spectral information of objects in multiple bands, thereby revealing subtle features that cannot be observed with traditional images. Hyperspectral technology has been widely used in the agricultural field in recent years, especially in crop health monitoring, disease detection, and yield prediction, and has good prospects. However, although hyperspectral technology can provide rich data resources, how to effectively process these high-dimensional data is a challenge. Traditional machine learning methods are prone to "dimensionality disaster" on high-dimensional data, which affects the performance of the model. In addition, the complexity and diversity of hyperspectral image data place higher requirements on the generalization ability of the model. Summary of the invention

[0004] In order to overcome the problems existing in the related technologies, this application provides a rice fruiting rate prediction method based on hyperspectral imaging technology combined with SS-CNN and SVM models. This method innovatively combines hyperspectral imaging technology and deep learning models, and processes and analyzes the hyperspectral data of rice to achieve accurate prediction of rice fruiting rate without destroying the plants.

[0005] The present application provides a rice seed setting rate prediction method based on hyperspectral imaging technology combined with SS-CNN and SVM model, comprising the following steps:

[0006] Hyperspectral image acquisition: Use hyperspectral imaging equipment to collect spectral images of rice grains and obtain hyperspectral data in multiple bands;

[0007] Hyperspectral image annotation: Annotate the collected hyperspectral images to generate annotation data containing the target area;

[0008] Data preprocessing: Perform smoothing and first-order derivative processing on the generated hyperspectral data;

[0009] Extract hyperspectral data: Extract the hyperspectral information of the corresponding area in the hyperspectral image based on the annotated data, and generate spectral data and spectral line graphs in standard data format for subsequent processing;

[0010] Band feature extraction: Based on the data characteristics, the spectral-spatial convolutional neural network (SS-CNN) is used to extract features from the preprocessed spectral data, extracting deep features layer by layer to form a high-dimensional feature vector;

[0011] Classification and prediction of seed setting rate: Use support vector machine (SVM) for training to obtain a classification model, predict the seed setting rate of rice grains, compare the actual seed setting rate with the predicted seed setting rate, and generate a comparison result of the seed setting rate;

[0012] In one embodiment, collecting a hyperspectral image of a whole rice plant includes:

[0013] Spectral images of rice grains are collected through hyperspectral imaging equipment to obtain hyperspectral data in multiple bands. The collected hyperspectral data covers the spectral range from 800nm ​​to 1700nm, recording the spectral reflectance information of rice grains. The spatial resolution of the spectral data is 320×333 pixels, and the spectral resolution is 256 bands, ensuring the accuracy and clarity of the spectral image.

[0014] In one embodiment, after obtaining the hyperspectral image data, labeling the hyperspectral image further includes:

[0015] Labelme software was used to annotate the collected hyperspectral images and generate annotation data containing the target area (such as rice grain outline or specific area). The generated annotation data was saved as a JSON format file, and the annotation information was aligned with the spectral image.

[0016] In one embodiment, data preprocessing includes:

[0017] Firstly, the raw hyperspectral data were preprocessed simply using SpecView data processing software. Then, the Savitzky-Golay filter was used to smooth the data and remove random noise. The parameters were set as window length of 31 and second-order polynomial fitting. The first-order derivative of the hyperspectral data was calculated to enhance the changes in spectral features and extract more significant features. Finally, the preprocessed spectral data was standardized to ensure that the data of all bands were within the same scale range to avoid the data values ​​of some bands being too large or too small to affect the classification results.

[0018] In one embodiment, extracting and visualizing hyperspectral data includes:

[0019] Based on the JSON file generated by labeling data using labelme software, the spectral data of the corresponding area in the hyperspectral image is extracted to form the corresponding spectral feature matrix, ensuring that each annotated area has corresponding hyperspectral information, and the spectral data of the annotated area is converted into a standard data format and a spectral line graph is generated for subsequent processing.

[0020] In one embodiment, using the SS-CNN model to extract band features includes:

[0021] According to the characteristics of the data, a spectral-spatial convolutional neural network (SS-CNN) is used to extract features of the preprocessed spectral data. A one-dimensional convolution kernel with a length of 5 and a ReLU activation function are used. The convolution layer is followed by a pooling layer, and the pooling layer adopts the maximum pooling operation. Then a second convolution layer, a batch normalization layer and a maximum pooling layer are added. The number of feature extraction layers is 2, and deep features are extracted layer by layer to form a high-dimensional feature vector.

[0022] In one embodiment, using a support vector machine classifier (SVM) to predict rice seed setting rate according to band characteristics includes:

[0023] The most representative and discriminative feature subset in the data set was obtained by combining the extracted high-dimensional features. A linear kernel function was used with a regularization parameter C of 1.0 and a random seed of 42 to initialize the random number generator to obtain a support vector machine (SVM) classification model to predict the fruit setting rate of rice grains. The actual fruit setting rate was compared with the predicted fruit setting rate to generate a comparison result of the fruit setting rate.

[0024] The technical solution provided in the present application has the following beneficial effects: a rice fruit set rate prediction method based on hyperspectral imaging technology combined with SS-CNN and SVM models, which realizes accurate prediction of rice fruit set rate through the steps of hyperspectral image acquisition of rice grains, hyperspectral image annotation, data preprocessing and extraction of reflectance data of rice grains, feature extraction and classification model establishment, thereby avoiding the time-consuming and labor-intensive problems of traditional manual detection and greatly improving analysis efficiency.

[0025] It should be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the present application. BRIEF DESCRIPTION OF THE DRAWINGS

[0026] The above and other objects, features and advantages of the present application will become more apparent through a more detailed description of exemplary embodiments of the present application in conjunction with the accompanying drawings, wherein the same reference numerals generally represent the same components in the exemplary embodiments of the present application.

[0027] Figure 1It is a flow chart of a method for predicting rice fruit set rate based on hyperspectral imaging technology combined with SS-CNN and SVM models as shown in Example 1 of the present application.

[0028] Figure 2 This is a hyperspectral image of a sample of the present invention.

[0029] Figure 3 It is a spectrum curve diagram generated after extracting the spectrum information of the target area in the present invention. DETAILED DESCRIPTION

[0030] The preferred embodiments of the present application will be described in more detail below with reference to the accompanying drawings. Although the preferred embodiments of the present application are shown in the accompanying drawings, it should be understood that the present application can be implemented in various forms and should not be limited by the embodiments described herein. On the contrary, these embodiments are provided to make the present application more thorough and complete, and to fully convey the scope of the present application to those skilled in the art.

[0031] The terms used in this application are for the purpose of describing specific embodiments only and are not intended to limit this application. The singular forms of "a", "said" and "the" used in this application and the appended claims are also intended to include plural forms unless the context clearly indicates other meanings. It should also be understood that the term "and / or" used herein refers to and includes any or all possible combinations of one or more associated listed items.

[0032] It should be understood that although the terms "first", "second", "third", etc. may be used in this application to describe various information, this information should not be limited to these terms. These terms are only used to distinguish the same type of information from each other. For example, without departing from the scope of this application, the first information may also be referred to as the second information, and similarly, the second information may also be referred to as the first information. Thus, the features defined as "first" and "second" may explicitly or implicitly include one or more of the features. In the description of this application, the meaning of "multiple" is two or more, unless otherwise clearly and specifically defined.

[0033] Embodiment 1

[0034] Rice is an important food crop in the world, and its yield and quality are directly related to food security. Rice fruit set rate is one of the important indicators to measure rice yield. Traditional rice fruit set rate assessment relies on manual measurement and observation, which is not only time-consuming and labor-intensive, but also has a high degree of subjectivity.

[0035] Therefore, this application proposes a rice fruit set rate prediction method based on hyperspectral imaging technology combined with SS-CNN and SVM models. This method constructs an end-to-end prediction process by collecting hyperspectral images of rice grains, annotating hyperspectral images, preprocessing data and extracting reflectance data of rice grains, extracting features and establishing classification models. First, the deep features of rice grains are extracted through SS-CNN, and then the SVM model is used for classification prediction, finally achieving accurate prediction of rice fruit set rate.

[0036] The technical solution of the embodiments of the present application is described in detail below with reference to the accompanying drawings.

[0037] Figure 1 It is a flow chart of a method for predicting rice fruit set rate based on hyperspectral imaging technology combined with SS-CNN and SVM models as shown in Example 1 of the present application.

[0038] Figure 2 This is the hyperspectral image acquisition of a certain sample shown in the embodiment of the present application.

[0039] Figure 3 It is a spectral curve diagram generated after extracting the spectral information of the target area shown in the embodiment of the present application.

[0040] See also Figure 1 , Figure 2 and Figure 3 .

[0041] A rice fruit set rate prediction method according to an embodiment of the present application comprises the following steps:

[0042] S101 uses hyperspectral imaging equipment to collect spectral images of rice grains and obtain hyperspectral data in multiple bands. The collected hyperspectral data covers the spectral range from 800nm ​​to 1700nm and records the spectral reflectance information of rice grains.

[0043] Furthermore, the embodiments of the present application do not limit the specific types of samples, which can be selected according to actual conditions.

[0044] S102 uses Labelme software to annotate the collected hyperspectral images.

[0045] Exemplarily, the specific content of the annotation data is: generating annotation data containing a target area (such as a rice grain outline or a specific area), saving the generated annotation data as a JSON format file, and aligning the annotation information with the spectral image.

[0046] The embodiment of the present application does not impose any necessary restrictions on the accuracy of the outline of the marked area, and it can be roughly accurate based on actual conditions.

[0047] S103 extracts and visualizes hyperspectral data: extracts spectral data of the corresponding area in the hyperspectral image based on the annotated data, converts the spectral data of the annotated area into a standard data format, and generates a spectral line graph for subsequent processing. Before obtaining data in the above steps, the raw data needs to be preprocessed, and the spectral data is smoothed using a Savitzky-Golay filter and first-order derivative processing.

[0048] S104 uses the SS-CNN model to obtain characteristic bands: an optimized shallow convolutional neural network model is used to extract features from the preprocessed spectral data, extracting deep features layer by layer to form a high-dimensional feature vector.

[0049] S105 uses SVM classifier to predict the rice grain setting rate based on characteristic bands: the most representative and discriminative feature subset in the data set is obtained by combining the extracted high-dimensional features, and SVM is used for training to obtain a classification model to predict the rice grain setting rate.

[0050] Beneficial effects of the embodiments of the present application: The rice fruiting rate prediction method based on hyperspectral imaging technology combined with SS-CNN and SVM models constructs an end-to-end prediction process by collecting hyperspectral images of rice grains, annotating hyperspectral images, preprocessing data and extracting reflectance data of rice grains, extracting features and establishing classification models. First, the deep features of rice grains are extracted by SS-CNN, and then the SVM model is used for classification prediction, and finally the accurate prediction of rice fruiting rate is achieved, which avoids the time-consuming and laborious problems of traditional manual detection and greatly improves the analysis efficiency.

[0051] The embodiments of the present application have been described above, and the above description is exemplary, not exhaustive, and is not limited to the disclosed embodiments. Many modifications and changes will be apparent to those of ordinary skill in the art without departing from the scope and spirit of the described embodiments. The selection of terms used herein is intended to best explain the principles of the embodiments, practical applications, or improvements to the technology in the market, or to enable other persons of ordinary skill in the art to understand the embodiments disclosed herein.

Claims

1. A rice ear setting rate measurement method based on deep learning and graph fusion, characterized in that: The method comprises the following steps: (1) Hyperspectral image acquisition: Spectral images of rice grains are acquired through hyperspectral imaging equipment to obtain hyperspectral data in multiple bands. The acquired hyperspectral data covers the spectral range from 800 nm to 1700 nm and records the spectral reflectance information of rice grains. (2) Hyperspectral image annotation: Labelme software is used to annotate the collected hyperspectral images and generate annotation data containing the target area (such as the outline of rice grains or specific areas). The generated annotation data is saved as a JSON format file, and the annotation information is aligned with the spectral image. (3) Data preprocessing: The generated hyperspectral data is preprocessed, including the following steps: a. Smoothing: Use Savitzky-Golay filter to smooth the spectral data and reduce data noise; b. First-order derivative processing: Calculate the first-order derivative of the spectral data to enhance the changes in spectral features and extract more significant features. (4) Extract and visualize hyperspectral data: Extract the spectral data of the corresponding area in the hyperspectral image based on the annotated data to form a corresponding spectral feature matrix, ensure that each annotated area has corresponding hyperspectral information, and convert the spectral data of the annotated area into a standard data format and generate a spectral line graph for subsequent processing. (5) Band feature extraction: Based on the data characteristics, the spectral-spatial convolutional neural network (SS-CNN) is optimized and modified to only care about the spectral features. The optimized shallow convolutional neural network model is used to extract features from the preprocessed spectral data, extracting deep-level features layer by layer to form a high-dimensional feature vector. (6) Classification and prediction of rice grain setting rate: The most representative and discriminative feature subset in the data set is obtained by combining the extracted high-dimensional features. The support vector machine (SVM) is used for training to obtain a classification model, predict the rice grain setting rate, compare the actual setting rate with the predicted setting rate, and generate a comparison result of the setting rate.

2. The method according to claim 1, characterized in that In the hyperspectral image acquisition step, the spatial resolution of the spectral data is 320×333 pixels, and the spectral resolution is 256 bands, ensuring the accuracy and clarity of the spectral image.

3. The method according to claim 1, characterized in that The hyperspectral image annotation uses the annotation data generated by Labelme to accurately extract the hyperspectral information of the rice grain area. The annotation data is saved in JSON format and contains annotation information such as polygon outlines and labels.

4. The method according to claim 1, characterized in that: In the data preprocessing step, the parameters of the Savitzky-Golay filter are set to a window length of 31 and a second-order polynomial fitting to ensure the effects of spectral smoothing and noise elimination.

5. The method according to claim 1, characterized in that The convolution layer of the SS-CNN model uses a one-dimensional convolution kernel with a length of 5 and a ReLU activation function. The convolution layer is followed by a pooling layer, which uses a maximum pooling operation. Then, a second convolution layer, a batch normalization layer, and a maximum pooling layer are added, and the number of feature extraction layers is 2.

6. The method according to claim 1, characterized in that The SVM model uses a linear kernel function, the regularization parameter C is 1.0, and the random seed 42 is used to initialize the random number generator.

7. A rice fruiting rate prediction system based on hyperspectral technology, characterized in that: The system includes: (1) Hyperspectral image acquisition module: used to collect spectral images of rice grains through hyperspectral imaging equipment, with a spectral range of 800nm ​​to 1700nm and a resolution of 320×333 pixels; (2) Image annotation module: Label the collected hyperspectral images through Labelme software to generate annotation files corresponding to the hyperspectral images for extracting spectral data of the target area; (3) Data extraction module: Generates the spectral feature matrix of rice grains based on the annotated data, and integrates the spectral data of the annotated area to generate a standard data format; (4) Data preprocessing module: smoothing and first-order derivative calculation of the generated hyperspectral data to reduce noise and enhance spectral features; (5) Deep feature extraction module: Use the SS-CNN model to extract high-dimensional features of rice grain spectra; (6) Classification and prediction module: Use the SVM model to classify the extracted high-dimensional feature data subset and predict the rice grain setting rate, compare the actual setting rate with the predicted setting rate, and generate a comparison result of the setting rate; 8. The system according to claim 7, characterized in that The data preprocessing module uses a Savitzky-Golay filter to smooth the spectral data, and the smoothing parameters are a window length of 31 and a second-order polynomial fitting.

9. The system according to claim 7, characterized in that The SS-CNN model in the deep feature extraction module includes 2 convolutional layers and a pooling layer, wherein there are one-dimensional convolution kernels with lengths of 5 and 3 respectively, and a one-dimensional pooling kernel with a length of 2.

10. The system according to claim 7, characterized in that The SVM model in the classification and prediction module uses a linear kernel function, the regularization parameter C is 1.0, and the random number generator is initialized.

Images