Information processing apparatus, information processing method, and program

Abstract

Predict the measured information without increasing the processing effort. 【Solution means】An information processing apparatus comprising: a growth index derivation unit that derives a crop growth index based on a satellite image obtained by imaging farmland from a satellite; and a yield prediction unit that derives a predicted value of the measured information of the crop by inputting the growth index into a learned model, wherein the learned model is created based on the crop growth-related information generated based on an aerial image obtained by imaging the farmland from above and the measured information of the crop that is the source of the growth-related information, and the output value of a regression model that outputs a predicted value of the measured information of the crop when the crop growth-related information is input is used as teacher data, and a crop growth index created based on the satellite image is used as learning data, and it is learned to output a predicted value of the measured information of the crop when the growth index is input.

Description

Technical Field

[0001] The present invention relates to an information processing apparatus, an information processing method, and a program.

Background Art

[0002] In crop cultivation, due to the influence of soil, topography, weather, pests and diseases, damage by birds and beasts, etc., there may be a large difference in the cultivation plan and the actual growth and yield for each field. As a result, the harvest of crops may be carried out by a contractor or the like on a regional basis, and there may be a shortage or surplus of personnel, machinery, and facilities related to the harvesting and preparation work, leading to work losses. Therefore, in order to formulate an appropriate harvesting and preparation work plan and efficiently allocate personnel and machinery, a technology that can accurately predict the yield within a region before the harvesting work is required. The conventionally used method is to manually conduct a mowing survey of a part of the field, measure the plant height and weight, and predict the yield of the entire field from the results. In this method, the yield of the entire field is predicted based on the mowing survey results of only a very small part of a large field. However, the mowing work for each field is labor-intensive and burdensome for workers. In addition, due to the low representativeness of the sampling data, the error is large. Furthermore, since the validity of the survey location selection and the accuracy of the growth status diagnosis result depend on the proficiency of the investigator, the variation in data and the difficulty of technology inheritance are problems.

[0003] In recent years, a technology has been developed to estimate the plant height from a three-dimensional image created from an image obtained by drone aerial photography, then calculate the above-ground volume from the integral value of the estimated grass height, and finally predict the yield from the relational expression between the above-ground volume and the measured yield (for example, Patent Document 1). In addition, a method is also known in which the NDVI value is calculated from the color information obtained from satellite images, and the yield is predicted from the relational expression with the yield (for example, Patent Document 2).

Prior Art Documents

Patent Documents

[0004]

Patent Document 1

Patent Document 2

Summary of the Invention

Problems to be Solved by the Invention

[0005] In the method using drone aerial photography images, due to the high labor and cost associated with drone aerial photography (the shootable range is about 10 to 50 [ha] / day), it takes an enormous amount of time and labor to complete the aerial photography of all target fields within a region, and it has not been put into social implementation in a vast area. Also, in the method using satellite images, due to the long shooting interval (= low shooting frequency) and low resolution by satellites, it may not be possible to make a timely and accurate prediction at the field level. Further, when predicting the yield from the relational expression between satellite images and measured yields, measured values adapted to the resolution of satellite images (about 1 to 10 [m]) are required, but the sampling work at this scale is extremely labor-intensive, and it is also a problem that it takes a long time to collect the necessary measured values. Thus, the conventional methods have problems in terms of labor and accuracy in promoting practical application.

[0006] The present invention has been made in consideration of such circumstances, and one of its objectives is to provide an information processing apparatus, an information processing method, and a program capable of predicting measured information without increasing the processing labor.

Means for Solving the Problems

[0007] An information processing apparatus according to an aspect of the present invention includes a growth index derivation unit that derives a growth index of a crop based on a satellite image obtained by imaging farmland from a satellite, and a yield prediction unit that derives a predicted value of the measured information of the crop by inputting the growth index into a learned model. The learned model is created based on the growth-related information of the crop generated based on an aerial image obtained by imaging the farmland from above and the measured information of the crop that is the source of the growth-related information, and outputs a predicted value of the measured information of the crop when at least the growth-related information of the crop is input. The output value of a regression model is used as teacher data, and the growth index of the crop created based on the satellite image is used as learning data, and it is learned to output a predicted value of the measured information of the crop when the growth index is input.

[0008] An information processing method according to another aspect of the present invention includes a process in which an information processing apparatus derives a growth index of a crop based on a satellite image obtained by imaging farmland from a satellite, and a process in which the information processing apparatus derives a predicted value of the measured information of the crop by inputting the growth index into a learned model. The learned model is created based on the growth-related information of the crop generated based on an aerial image obtained by imaging the farmland from above and the measured information of the crop that is the source of the growth-related information, and outputs a predicted value of the measured information of the crop when at least the growth-related information of the crop is input. The output value of a regression model is used as teacher data, and the growth index of the crop created based on the satellite image is used as learning data, and it is learned to output a predicted value of the measured information of the crop when the growth index is input.

[0009] A program according to another aspect of the present invention causes a processor of an information processing apparatus to execute a process of deriving a growth index of a crop based on a satellite image obtained by imaging farmland from a satellite, and a process of inputting the growth index into a learned model to derive a predicted value of the measured information of the crop. The learned model is created based on the growth-related information of the crop generated based on an aerial image obtained by imaging the farmland from above and the measured information of the crop that is the source of the growth-related information, and outputs a predicted value of the measured information of the crop when at least the growth-related information of the crop is input. The output value of the regression model is used as teacher data, and the growth index of the crop created based on the satellite image is used as learning data. The model is learned to output a predicted value of the measured information of the crop when the growth index is input.

Advantages of the Invention

[0010] According to each of the above aspects, it is possible to predict the measured information without increasing the processing labor.

Brief Description of the Drawings

[0011]

Figure 1

Figure 2

Figure 3

Figure 4

Figure 5

Figure 6

Embodiments for Carrying Out the Invention

[0012] Hereinafter, with reference to the drawings, embodiments of the information processing apparatus, information processing method, and program of the present invention will be described. The information processing apparatus of the present invention derives a predicted value of actual measurement information of crops by inputting a growth index of crops derived based on a satellite image obtained by imaging farmland from a satellite into a learned model. The actual measurement information is, for example, the yield, but may also be, for example, the nutritional value (per unit yield or per unit area), the presence or absence of diseases, the proportion of diseases, the proportion of weeds, etc. The learned model is created based on growth-related information of crops generated based on the aerial images taken from above. The growth-related information is, for example, the height information of the crops, but is not limited to this, and may be color information, the number of seedlings emerged, etc., or may include a plurality of them. The information processing apparatus aims to improve the accuracy of yield prediction without increasing the processing labor. In the following description, the growth-related information is the height information of the crops, and the crops are assumed to be corn with a high correlation between the plant height (height) and the yield. The information processing apparatus is realized by one or more processors. This processor may be a rental server on the cloud or a dedicated product for implementing the invention.

[0013] FIG. 1 is a diagram showing an example of the usage environment and configuration of the information processing apparatus 100. The information processing apparatus 100 acquires, for example, an image (satellite image, more specifically, a satellite constellation image of a plurality of times provided in cooperation from a plurality of satellites, which may be simply referred to as a satellite image hereinafter) captured by the satellite 10 from one or more satellites 10 via the satellite image provider server 20 and the network NW, and also acquires an image (aerial image) captured by the drone 30 from the drone 30 via the relay device 40 and the network NW. The satellite image is an image of the farmland N on the ground. The aerial image is an image captured by the drone 30 while flying over the farmland N a plurality of times and changing the angle. In the present invention, the aerial image is used exclusively in the generation stage (learning stage) of the regression model and the learned model, and the input data used in yield prediction is exclusively the growth index generated from the satellite image.

[0014] The information processing apparatus 100 provides output information to the terminal device 50 via, for example, the network NW. The network NW is an arbitrary network such as, for example, a LAN, a WAN, or an Internet line, and may be wired or wireless. Note that the method of acquiring an image is not limited to this, and an image may be acquired by mounting an image stored in a storage medium on the drive device of the information processing apparatus 100. Further, the image may be pre-processed before being passed to the information processing apparatus 100.

[0015] The satellite 10 orbits, for example, on an orbit around the Earth and periodically images the ground. The satellite image provider server 20 provides the images captured by the satellite 10 to the information processing apparatus 100 and the like.

[0016] The drone 30 is an unmanned aerial vehicle equipped with various imaging devices. The drone 30 repeatedly images the ground while flying in response to an instruction from the relay device 40. The drone 30 stores a plurality of drone images together with the camera parameters of the drone images and position information such as GPS in the memory card mounted on the drone 30 or transmits them to the relay device 40.

[0017] The relay device 40 is a terminal device such as, for example, a tablet terminal. The relay device 40 is equipped with an application for operating the drone 30 and receiving and displaying the images captured by the drone 30. The relay device 40 displays the images captured by the drone 30 as RGB images or transmits them to the information processing apparatus 100 via the network NW.

[0018] The terminal device 50 is a computer device such as, for example, a personal computer, a smartphone, or a tablet terminal. The terminal device 50 communicates with the information processing apparatus 100 via the network NW and displays the output information received from the information processing apparatus 100 described later. The relay device 40 and the terminal device 50 may be integrated into one.

[0019] The information processing apparatus 100 has, for example, the function of a web server. The information processing apparatus 100 includes, for example, an acquisition unit 110, a growth index derivation unit 120, a learning unit 130, a yield prediction unit 140, a provision unit 150, and a storage unit 170. The learning unit 130 includes, for example, a height information generation unit 132, a regression model generation unit 134, and a machine learning processing unit 136. Components other than the storage unit 170 are realized, for example, by a processor such as a CPU (Central Processing Unit) executing a program (software). Some or all of these components may be realized by hardware (including a circuit unit; circuitry) such as an LSI (Large Scale Integration), an ASIC (Application Specific Integrated Circuit), an FPGA (Field-Programmable Gate Array), or a GPU (Graphics Processing Unit), or may be realized by cooperation between software and hardware. The program may be stored in advance in a storage device (a storage device having a non-transitory storage medium) such as an HDD (Hard Disk Drive) or a flash memory, or may be stored in a removable storage medium (a non-transitory storage medium) such as a DVD or a CD-ROM, and may be installed in the storage device when the storage medium is mounted on a drive device. The storage unit 170 stores information such as a regression model 172 and a learned model 174 generated by the learning unit 130.

[0020] FIG. 2 is a diagram schematically showing the content of processing of each part of the information processing apparatus 100 in the learning stage. The acquisition unit 110 acquires an aerial image, a satellite image, and information on the actual yield of crops (actual measured yield). The aerial image includes a first aerial image of the farmland N in the state of bare land taken from above and a second aerial image of the farmland N before the crop is harvested (in a state where the plant height before harvesting has grown). The acquisition unit 110 acquires this information via the network NW or via an input device (not shown).

[0021] The growth index derivation unit 120 generates a growth index image (hereinafter simply referred to as the growth index) with the growth index as the pixel value based on each pixel value of the satellite image. The growth index is, for example, the NDVI (Normalized Difference Vegetation Index) represented by Equation (1). In the equation, Ref NIR is the reflectance in the near-infrared wavelength band (hereinafter referred to as the band), and Ref Red is the reflectance in the red light band. Instead of this, the growth index may be GNDVI (Green Normalized Difference Vegetation Index), NDRE (Normalized Difference Red Edge), LCI (Leaf Chlorophyll Index), etc. In the following description, it is assumed that the growth index is NDVI.

[0022] NDVI =(Ref NIR -Ref Red ) / (Ref NIR +Ref Red ) …(1)

[0023] The height information generation unit 132 of the learning unit 130 calculates the depth (distance from the imaging position) for each pixel in the image using a three-dimensional reconstruction technique that applies the principle of a stereo camera to, for example, a plurality of aerial images taken from different imaging positions. Then, a three-dimensional model is generated by converting the distance from the imaging position into absolute coordinates on the ground. The height information generation unit 132 generates a first numerical surface model based on the first aerial image and a second numerical surface model based on the second aerial image, respectively. Further, the height information generation unit 132 generates height information indicating the plant height before crop harvest by performing a vertical difference calculation on the first numerical surface model and the second numerical surface model. FIG. 3 is a diagram showing the height information generated for a certain farmland N (field).

[0024] Returning to FIG. 2, the regression model generation unit 134 generates a regression model based on the height information generated by the height information generation unit 132 and the actual measured yield of the crop from which the height information was derived (the yield at the time of actual harvesting after the second aerial image was acquired). For example, the regression model generation unit 134 generates a regression model (a function that outputs the yield when height information is input) 172 by fitting the relationship between the height information and the actual measured yield to a geometric figure such as a quadratic curve or a straight line based on the least squares method or the like. FIG. 4 is a diagram schematically showing how the regression model 172 is generated.

[0025] Returning to FIG. 2, when the regression model 172 is generated, the machine learning processing unit 136 uses, as teacher data, the predicted aerial yield value (the output value of the regression model) obtained by inputting the height information generated at another timing at a later date into the regression model 172, and, as learning data, for example, time-series data of the growth index of the crop created based on the satellite image, and learns (trains) the parameters of the machine learning model so that a predicted value of the crop yield is output when the time-series data of the growth index is input, thereby generating a learned model 174. The learned model 174 is learned by, for example, a DNN (Deep Neural Network), but there are no particular restrictions on the form of the learned model 174. The machine learning processing unit 136 generates the learned model 174 by, for example, the backpropagation method.

[0026] FIG. 5 is a diagram arranging the processing of each part of the information processing apparatus 100 in time series including the learning stage and the inference stage. First, in the first period for generating the regression model 172, the regression model 172 is generated based on the first aerial image, the second aerial image, and the actual measured yield. In this first period, for example, the regression model 172 is generated using information acquired in accordance with multiple plantings and harvests on the same farmland.

[0027] In the subsequent second phase, a learned model 174 is generated based on the first aerial image, the second aerial image, the predicted aerial yield obtained using the regression model 172, and the growth index based on the satellite image acquired during the same period. Similar to the first phase, in this second phase, the learned model 174 may be generated using the information obtained as multiple plantings and harvests are carried out on the same farmland.

[0028] In the third phase, which is the inference phase, the yield prediction unit 140 predicts the yield of the crop from which the satellite image was derived (satellite yield prediction value) by inputting the satellite image into the learned model 174. The providing unit 150 outputs information on the satellite yield prediction value to the terminal device 50 or the like.

[0029] By performing such processing, it is possible to predict the yield without the trouble of flying a drone for aerial photography every time the yield is predicted. Aerial photography by a drone requires skilled techniques and it is difficult to acquire information over a wide area at once. In contrast, satellite constellation images are taken over a wide area at high frequencies, so they can be applied to various regions. That is, once the regression model 172 and the learned model 174 are generated, it is possible to predict the yield simply by inputting the growth index based on the satellite image that can be acquired over a wide area easily thereafter. Therefore, it is possible to predict the yield without increasing the processing effort.

[0030] The inventor of the present application conducted an empirical experiment on the correlation between the predicted aerial yield value and the predicted satellite yield value. That is, while obtaining the predicted satellite yield value by the above method, it was determined what would happen if the predicted aerial yield value was obtained for the same farmland N at the same time, and a correlation diagram of them was created. FIG. 6 is a diagram showing the results of the empirical experiment of the present invention. As shown in the figure, although there are some outliers, it was confirmed that a high correlation is shown as a whole.

[0031] According to each of the embodiments described above, it is possible to predict the measured information without increasing the processing effort.

[0032] As described above, the embodiments for carrying out the present invention have been described using embodiments. However, the present invention is not limited to such embodiments, and various modifications and substitutions can be made without departing from the gist of the present invention. For example, the input information to the regression model may include not only the growth-related information of the crop but also other information such as weather information, variety information, or cultivation management information such as planting density and fertilization amount.

Explanation of Signs

[0033] 10 Satellite 30 Drone 40 Relay Device 50 Terminal Device 100 Information Processing Device 110 Acquisition Unit 120 Growth Index Derivation Unit 130 Learning Unit 132 Height Information Generation Unit 134 Regression Model Generation Unit 136 Machine Learning Processing Unit 140 Yield Prediction Unit 150 Provision Unit

Claims

1. A growth index derivation unit that derives a crop growth index based on a satellite image obtained by imaging farmland from a satellite, A yield prediction unit that derives a predicted value of the actual measurement information of the crop by inputting the growth index into a learned model, Comprising, The learned model is created based on the growth-related information of the crop generated based on an aerial image obtained by imaging the farmland from above and the actual measurement information of the crop that is the source of the growth-related information, and at least when the growth-related information of the crop is input, the output value of a regression model that outputs a predicted value of the actual measurement information of the crop is used as teacher data, and the crop growth index created based on the satellite image is used as learning data, and it is learned to output a predicted value of the actual measurement information of the crop when the growth index is input, The growth-related information is at least one of crop height information, color information, and seedling count information, The actual measurement information is yield, yield per unit, or nutritional value per unit area, presence or absence of disease, proportion of disease, or proportion of weeds, The crop is a crop with a high correlation between the growth-related information and the actual measurement information, The height information is generated based on the difference between a first aerial image obtained by imaging the farmland in a bare land state from above and a second aerial image obtained by imaging the farmland before harvesting the crop, An information processing device.

2. The growth index is time-series data derived for a plurality of time points, The learned model is learned using the output value of the regression model as teacher data and the time-series data of the growth index as learning data, The information processing device according to Claim 1.

3. Further comprising a learning unit that generates the learned model, The information processing device according to Claim 1.

4. The information processing device, Executes a process of deriving a crop growth index based on a satellite image obtained by imaging farmland from a satellite, And a process of deriving a predicted value of the actual measurement information of the crop by inputting the growth index into a learned model. The learned model is created based on the growth-related information of the crop generated based on the aerial image of the farmland taken from above and the measured information of the crop that is the source of the growth-related information, and when at least the growth-related information of the crop is input, the output value of the regression model that outputs the predicted value of the measured information of the crop is used as teacher data, and the growth index of the crop created based on the satellite image is used as learning data, and it is learned to output the predicted value of the measured information of the crop when the growth index is input. The growth-related information is at least one of the height information of the crop, the color information, and the information on the number of seedlings. The measured information is the yield, the yield per unit, or the nutritional value per unit area, the presence or absence of diseases, the proportion of diseases, or the proportion of weeds. The crop is a crop with a high correlation between the growth-related information and the measured information. The height information is generated based on the difference between the first aerial image of the farmland in the bare land state taken from above and the second aerial image of the farmland before harvesting the crop taken from above. Information processing method.

5. To the processor of the information processing device, Based on the satellite image of the farmland taken by the satellite, perform the process of deriving the growth index of the crop, and By inputting the growth index into the learned model, perform the process of deriving the predicted value of the measured information of the crop. The learned model is created based on the growth-related information of the crop generated based on the aerial image of the farmland taken from above and the measured information of the crop that is the source of the growth-related information, and when at least the growth-related information of the crop is input, the output value of the regression model that outputs the predicted value of the measured information of the crop is used as teacher data, and the growth index of the crop created based on the satellite image is used as learning data, and it is learned to output the predicted value of the measured information of the crop when the growth index is input. The growth-related information is at least one of the height information of the crop, the color information, and the information on the number of seedlings. The measured information is the yield, the yield per unit, or the nutritional value per unit area, the presence or absence of diseases, the proportion of diseases, or the proportion of weeds. The crop is a crop with a high correlation between the growth-related information and the measured information. The height information is generated based on the difference between the first aerial image of the farmland in the bare land state taken from above and the second aerial image of the farmland before harvesting the crop taken from above. Program.

Images