Processing device, processing method, and processing program

Abstract

To provide a technique capable of non-destructively, easily, and highly accurately predicting the elemental content of an object such as a plant.SOLUTION: A processing device 1 includes a regression analysis unit (learning unit) 14 that performs learning of a prediction model for predicting the elemental content using one or more feature quantities related to data of an object contained in high-dimensional images, and the elements and the elemental content included in the object.SELECTED DRAWING: Figure 1

Description

[Technical field]

[0001] The present disclosure relates to a processing device, a processing method, and a processing program. [Background technology]

[0002] To understand the growth state of a plant, it is important to know the elemental content of the plant. For example, there is elemental analysis using the combustion method, in which the plant is crushed and burned to measure the elemental content (Non-Patent Document 1). However, since this requires destructive testing of the plant, it is difficult to continuously measure the elemental content while the plant is growing. Therefore, there is a method in which the spectrum of the plant is observed with a hyperspectral camera and the spectral data is used to predict the elemental content of the plant non-destructively (Non-Patent Document 2). [Prior art documents] [Non-patent literature]

[0003] [Non-Patent Document 1] Matsumoto, et al., "About Elemental Analysis of Organic Compounds -The Process of Determining Molecular Formulas and Its Current Status-", Chemistry and Education, Vol. 63, No. 12, 2015, pp. 608-611 [Non-Patent Document 2] Katherine Meacham-Hensold, 10 others, “Plot-level rapid screening for photosynthetic parameters using proximal hyperspectral imaging”, Journal of Experimental Botany, Vol.71, No.7, 2020, p.2312-p.2328, Available at: https: / / doi.org / 10.1093 / jxb / eraa068 Summary of the Invention [Problem to be solved by the invention]

[0004] In Non-Patent Document 2, the average value of the spectral data of an object region contained in a hyperspectral image is used as the feature of the spectral data in that object region, and this is performed at multiple wavelengths to use the feature of the spectral data for each wavelength.

[0005] However, in hyperspectral images, the reflectance of specific wavelengths varies depending on the part of the plant, so there is variation in the spectral data (spectral intensity) within a wide object area (within the entire plant). Therefore, if the average spectral intensity is used as a feature of the spectral data in that object area, the accuracy of predicting elemental content could be compromised.

[0006] The present disclosure has been made in consideration of the above circumstances, and aims to provide a technology that can predict the elemental content of a target object such as a plant non-destructively, easily, and with high accuracy. [Means for solving the problem]

[0007] A processing device according to one embodiment of the present disclosure includes a learning unit that learns a predictive model for predicting elemental content using one or more features relating to data of an object contained in a high-dimensional image and the elements and elemental content contained in the object.

[0008] A processing method according to one embodiment of the present disclosure is a processing method performed by a processing device, in which a predictive model for predicting elemental content is trained using one or more features relating to data of an object contained in a high-dimensional image, and the elements and elemental content contained in the object.

[0009] A processing program according to one embodiment of the present disclosure causes a computer to function as the processing device. Effect of the Invention

[0010] According to the present disclosure, it is possible to provide a technology that can predict the elemental content of a target object such as a plant non-destructively, easily, and with high accuracy. [Brief description of the drawings]

[0011] [Figure 1] FIG. 1 is a diagram illustrating an example of the configuration of a processing device. [Diagram 2] FIG. 2 is a diagram showing a method for predicting elemental contents in plants. [Diagram 3] FIG. 3 is a diagram showing an example of a hyperspectral image of a plant. [Figure 4] FIG. 4 shows the predicted results of elemental contents in plants. [Diagram 5] FIG. 5 is a diagram showing the prediction error of elemental contents in plants. [Figure 6] FIG. 6 is a diagram illustrating a hardware configuration of the processing device. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0012] Hereinafter, embodiments of the present disclosure will be described with reference to the drawings.

[0013] [overview] High-dimensional images such as hyperspectral images are characterized by a wide wavelength range and high resolution. Some components that make up plants, such as chlorophyll, have absorption at specific wavelengths, and by analyzing images of reflected light (hyperspectral images, etc.), it is possible to detect the components inside the plant.

[0014] This disclosure discloses a technique for predicting the elemental content of plants and the like by utilizing this characteristic.

[0015] Specifically, a predictive model for predicting the element content is trained using one or more feature amounts related to the data of an object contained in a high-dimensional image, and the elements and element content contained in the object.

[0016] In this way, a predictive model for predicting elemental content is trained using one or more feature quantities related to the data of the object contained in the high-dimensional image, and the elements and elemental content contained in the object, so that the elemental content of an object such as a plant can be predicted non-destructively, easily, and with high accuracy.

[0017] In addition, values ​​based on high-dimensional image data are selected as the feature quantities of the data used as training data. For example, if the feature quantities of the data used as training data are the feature quantities of spectral data, values ​​based on hyperspectral image data are selected.

[0018] Specifically, among the features of the spectral intensity of each pixel at each wavelength, such as the average, maximum, minimum, median, first quartile, third quartile, variance, kurtosis, and skewness, one feature other than the average value or two or more different feature values ​​including the average value are selected. At this time, the selected feature value may be modified.

[0019] In this way, since one feature other than the average value is selected as the feature value of the data used for the training data, the prediction accuracy of the element content can be further improved. Also, since two or more different feature values ​​including the average value are selected as the data feature value of the training data, the prediction accuracy of the element content can be further improved.

[0020] In other words, by selecting one feature other than the average value or two or more different features including the average value, the training data takes into account the distribution of spectral intensity, such as differences in reflectance of different parts of the plant in the hyperspectral image, and the prediction accuracy of the elemental content can be further improved.

[0021] Regarding the selection of the feature quantity, any predetermined feature quantity may be selected, or the type and number of feature quantities may be automatically selected based on the evaluation of the prediction model. In the latter case, for example, each prediction model combining one or more feature quantities is evaluated using the prediction error (root mean square error, mean absolute error, etc.) for the prediction of the element content by the prediction model, a predetermined information criterion (Akaike information criterion, Bayesian information criterion, etc.), a predetermined variable selection method (stepwise method, etc.), etc., and the type and number of the optimal feature quantities are selected based on the evaluation results of each prediction model.

[0022] In addition, a hyperspectral image is an example of a high-dimensional image. For example, a high-dimensional image expressed based on an arbitrary feature space, such as an image divided by wavelength or a color space captured by a metalens, a multispectral camera, a visible light camera, or the like, may be extracted.

[0023] Hereinafter, a hyperspectral image will be used as an example of a high-dimensional image.

[0024] [Configuration of element content prediction device] FIG. 1 is a diagram showing an example of the configuration of a processing device 1 according to the present embodiment.

[0025] In addition to the processing device 1, the system is equipped with a hyperspectral camera for photographing the plant, an input terminal for inputting elemental content data of the plant obtained by elemental analysis using a conventional combustion method, and lighting such as a halogen light for irradiating the plant with light.

[0026] The processing device 1 includes a spectrum information acquisition unit 11 , a spectrum feature quantity generation unit 12 , an elemental content information acquisition unit 13 , a regression analysis unit 14 , and an elemental content prediction unit 15 .

[0027] The spectral information acquisition unit 11 has a function of acquiring a hyperspectral image (spectral data of the plant sample) of a plant sample from a hyperspectral camera.

[0028] The spectral feature generating unit 12 has a function of generating a plurality of different features in the plant sample region by using the spectral data of each pixel of the plant sample included in the hyperspectral image.

[0029] For example, the spectral feature generation unit 12 converts the values ​​of the spectral data of the plant sample into various features such as the mean, maximum, minimum, median, first quartile, third quartile, variance, kurtosis, and skewness of the spectral intensity of each pixel at each wavelength.

[0030] In addition, the spectral feature generation unit 12 has a function of selecting, from the various features after conversion, one feature other than the average value or two or more different feature values ​​including the average value as feature data of the spectral data in the plant sample area.

[0031] The spectral feature generation unit 12 may make a predetermined change to the selected feature. For example, the spectral feature generation unit 12 may divide the selected average value by the variance, or may square the selected average value.

[0032] The element content information acquisition unit 13 has a function of acquiring element content data (elements and element contents) of a plant sample obtained by elemental analysis using a conventional combustion method.

[0033] The regression analysis unit (learning unit) 14 has a function of using one or more selected features related to the spectral data of the plant sample contained in the hyperspectral image and the element content data of the plant sample obtained by elemental analysis using a conventional combustion method as training data, performing regression analysis (an example of machine learning) using the training data, and generating and learning a regression model (prediction model) for predicting the element content.

[0034] The regression analysis unit 14 also has a function of evaluating each prediction model in which the type and number of features have been changed. At this time, the spectral feature generation unit 12 selects the optimal type and number of features based on the evaluation result of each prediction model.

[0035] The element content prediction unit (prediction unit) 15 has a function of predicting the element content of a plant sample using a trained regression model.

[0036] [Method of predicting element content] FIG. 2 is a diagram showing a method for predicting element contents in a plant according to the present embodiment.

[0037] In this embodiment, the carbon content of lettuce leaves was predicted. In advance, a user photographed about 1 to 10 lettuce leaves using a hyperspectral camera with a wavelength range of 350 nm to 1100 nm (151 wavelengths). In addition, the sample was subjected to elemental analysis by a combustion method to measure the carbon content of the lettuce leaves.

[0038] First, the spectral information acquisition unit 11 acquires a hyperspectral image (FIG. 3) of a lettuce leaf captured by the hyperspectral camera (step S1).

[0039] Next, the elemental content information acquiring unit 13 acquires carbon content data (elements and elemental contents) of the lettuce leaves obtained by elemental analysis using the combustion method (step S2).

[0040] Next, the spectral feature generation unit 12 cuts out only the lettuce leaf region from the hyperspectral camera image acquired in step S1 (step S3).

[0041] In this case, the hyperspectral image is high-dimensional and the amount of information per image is very large, so statistical processing methods and multivariate analysis methods cannot be used as is, and the background region cannot be successfully removed. Therefore, for example, the image is reduced to three dimensions using PCA (Principal Component Analysis), and the background region is removed from the reduced hyperspectral image. PCA and three dimensions after reduction are just examples, and methods other than PCA and other numbers of dimensions may also be used.

[0042] Next, the spectral feature generator 12 normalizes the spectral data of each pixel of the hyperspectral image of only the lettuce leaf region by the spectral data of each pixel of a white board photographed under the same photographing conditions to obtain spectral data of relative reflectance. After that, the spectral data feature data is generated from the spectral data of relative reflectance, and one feature other than the average value or two or more different feature values ​​including the average value are selected (step S4).

[0043] In this embodiment, nine features of the spectral reflectance (spectral intensity) of each pixel at each wavelength, namely, the mean, maximum, minimum, median, first quartile, third quartile, variance, kurtosis, and skewness, were generated, and from among the nine features, one feature other than the mean value or two or more distinct features including the mean value were selected as feature data of the spectral data in the lettuce leaf region.

[0044] Note that normalization of the spectral data is performed to obtain the relative reflectance, thereby reducing the influence on the data due to differences in the unique shooting conditions that appear for each image, and improving accuracy, and is not necessarily required.

[0045] Next, the regression analysis unit 14 uses the feature data of the spectral data in the lettuce leaf region obtained in step S4 as explanatory variables and the carbon content data of the lettuce leaves obtained in step S2 as objective variables, combines the training data, and performs PLS (Partial Least Squares) regression analysis (machine learning) using the training data to generate a regression model (carbon content prediction model) for predicting the carbon content of plants (step S5).

[0046] Finally, the element content prediction unit 15 predicts the carbon element content of the lettuce leaves using the carbon content prediction model (step S6). Specifically, the spectral data of the lettuce leaves, which are the prediction target plant, is used as prediction explanatory variables, and the prediction explanatory variables are input to the carbon content prediction model to obtain a predicted value of the carbon content of the prediction target plant.

[0047] The predicted results of the carbon content of plants are shown in Fig. 4. Fig. 4(a) shows the predicted results when only the average value of the spectral intensity is used. Fig. 4(b) shows the predicted results when the average value of the spectral intensity and the third quartile value are used. It can be seen that Fig. 4(b) according to this embodiment shows a predicted result closer to the fitted line (solid line) to the measured values ​​than the conventional Fig. 4(a).

[0048] The average Root Mean Squared Error (RMSE) value of each prediction result is shown in Figure 5. The average RMSE value of the regression model using the mean and the third quartile value was lower than the average RMSE value of the regression model using the mean value.

[0049] The above prediction results improve the accuracy of prediction of carbon content, making it possible to predict the amount of elements contained in plants non-destructively, easily and with high accuracy.

[0050] (Variation 1) In this embodiment, the carbon content data obtained by elemental analysis using the combustion method is acquired at the timing of step S2, but may be acquired at any timing before the regression analysis of step S5.

[0051] (Variation 2) In this embodiment, the process of predicting the carbon element content has been described, but after that, the type and number of features to be used may be changed to evaluate each prediction model, and the optimal type and number of features may be selected (determined) based on the evaluation results of each prediction model. The selection may be performed for each type of plant.

[0052] (Variation 3) In this embodiment, plants are used as an example, but even butterflies, insects, etc. may have different reflectances at specific wavelengths depending on the part, so the present invention can also be applied to animals if there is such a possibility. In addition, the present invention can be applied not only to plants and animals, but also to other living things and non-living things if there is a possibility that the elements contained in the part to be predicted and the amount of the elements differ from the reflectance at a specific wavelength.

[0053] (Variation 4) In this embodiment, the PLS regression analysis has been used for explanation, but a regression method other than PLS regression or a prediction method other than regression may be used. An example of a regression method other than PLS regression is support vector regression. An example of a prediction method other than regression is Random Forest or LightGBM (Light Gradient Boosting Machine). These machine learning methods may be used.

[0054] [Effects of the embodiment] According to this embodiment, a prediction model for predicting elemental content is trained using one or more feature quantities related to data of an object contained in a high-dimensional image, and the elements and elemental content contained in the object, so that the elemental content of an object such as a plant can be predicted non-destructively, easily, and with high accuracy.

[0055] [others] The present disclosure is not limited to the above-described embodiment, and various modifications are possible within the scope of the present disclosure.

[0056] The processing device 1 of this embodiment described above can be realized, for example, using a general-purpose computer system including a CPU 901, a memory 902, a storage 903, a communication device 904, an input device 905, and an output device 906, as shown in FIG. 6.

[0057] The memory 902 and the storage 903 are storage devices. In the computer system, the CPU 901 executes a predetermined program loaded onto the memory 902, whereby each function of the processing device 1 is realized.

[0058] The processing device 1 may be implemented in one computer. The processing device 1 may be implemented in multiple computers. The processing device 1 may be a virtual machine implemented in a computer.

[0059] The program for the processing device 1 can be stored in a computer-readable recording medium such as a HDD, SSD, USB memory, CD, or DVD. The computer-readable recording medium is, for example, a non-transitory recording medium. The program for the processing device 1 can also be distributed via a communication network. [Explanation of symbols]

[0060] 1 Processing equipment 11 Spectral information acquisition unit 12 Spectral feature generation unit 13 Element content information acquisition department 14 Regression Analysis Section 15 Elemental Content Prediction Section 901 CPU 902 Memory 903 Storage 904 Communication equipment 905 Input Device 906 Output Device

Claims

1. A processing device having a learning unit that trains a prediction model for predicting elemental content using one or more feature amounts related to data of an object contained in a high-dimensional image and the elements and elemental content contained in the object.

2. The processing device according to claim 1 , wherein the feature amount is one feature amount other than an average value of feature amounts related to the high-dimensional image, or two or more different feature amounts.

3. The processing device according to claim 1 , further comprising a prediction unit that predicts an element content by using the trained prediction model.

4. In a processing method performed by a processing device, A processing method for learning a predictive model for predicting elemental content using one or more features related to data of an object contained in a high-dimensional image, and the elements and elemental content contained in the object.

5. A processing program that causes a computer to function as the processing device according to any one of claims 1 to 3.

Images