Information processing device, information processing method, and program

Abstract

To enable prediction of measured information without increasing processing effort.SOLUTION: An information processing device includes: a growth index derivation unit that derives a growth index of a crop based on a satellite image captured of a farmland by a satellite; and a yield prediction unit that outputs a predicted value of measured information of the crop by inputting the growth index to a trained model, in which the trained model is constructed based on growth-related information of the crop generated from aerial images of the farmland taken from above and measured information of the crop on which the growth-related information is based, and the trained model is one that has been trained such that: output values from a regression model, which outputs predicted values of measured information of the crop in response to input of crop growth-related information, are used as teacher data; crop growth indices created based on the satellite image are used as training data; and predicted values of measured information of the crop are output when the growth index is input.SELECTED DRAWING: Figure 2

Description

[Technical Field]

[0001] The present invention relates to an information processing device, an information processing method, and a program. [Background technology]

[0002] When cultivating crops, soil, topography, weather, pests, and wildlife damage can significantly affect the actual growth and yield of crops. As a result, crop harvesting is often carried out locally by contractors, resulting in shortages or excesses of personnel, machinery, and facilities for harvesting and processing, leading to lost work. Therefore, to formulate appropriate harvesting and processing plans and efficiently allocate personnel and machinery, a technology is needed that can accurately predict yields within an area before harvesting begins. The conventional approach involves manually conducting a field-by-field survey and measuring plant height and weight to predict the yield for the entire field. This method predicts the yield for the entire field based on the results of a field-by-field survey of only a small portion of a large field. However, the labor-intensive process of field-by-field surveys places a heavy burden on workers. In addition, the sampling data is not representative enough, resulting in large errors. Furthermore, the validity of the selection of survey locations and the accuracy of the results of the growth status diagnosis depend on the skill level of the surveyor, which means that there is variability in the data and it is difficult to transfer the techniques.

[0003] In recent years, a technology has been developed that estimates plant height from three-dimensional images created from images taken by aerial photography using drones, then calculates above-ground volume from the integrated value of the estimated plant height, and then predicts yield from the relationship between above-ground volume and measured yield (for example, Patent Document 1). Also known is a method that calculates NDVI values from color information obtained from satellite images and predicts yield from the relationship with yield (for example, Patent Document 2). [Prior art documents] [Patent documents]

[0004] [Patent Document 1] Japanese Patent Publication No. 2021-189763 [Patent Document 2] Japanese Patent Application Laid-Open No. 2010-166851 Summary of the Invention [Problem to be solved by the invention]

[0005] In methods using drone aerial images, the high effort and cost required for drone aerial photography (the maximum capture area is approximately 10–50 ha per day) requires a significant amount of time and effort to complete aerial photography of all target fields in a region, preventing their widespread adoption. Furthermore, in methods using satellite images, the long satellite capture intervals (i.e., low capture frequency) and low resolution often prevent timely and accurate predictions at the field level. Furthermore, when predicting yields based on the relationship between satellite images and measured yields, actual measurements adapted to the resolution of the satellite images (approximately 1–10 m) are required. However, sampling at this scale is extremely labor-intensive, and collecting the necessary measurements takes many years, which presents challenges for practical application. Thus, conventional methods pose challenges in terms of labor and accuracy.

[0006] The present invention has been made in consideration of the above circumstances, and one of its objects is to provide an information processing device, an information processing method, and a program that can predict actual measurement information without increasing the processing effort. [Means for solving the problem]

[0007] An information processing device according to the present invention comprises a growth index derivation unit that derives a crop growth index based on satellite images of farmland taken from a satellite, and a yield prediction unit that derives a predicted value of the crop's actual measurement information by inputting the growth index into a trained model. The trained model is created based on the crop growth-related information generated based on aerial images of the farmland taken from above and the actual measurement information of the crop that is the source of the growth-related information, and uses the output value of a regression model that outputs a predicted value of the actual measurement information of the crop when at least the crop growth-related information is input as training data, and a crop growth index created based on the satellite image as training data, and is trained to output a predicted value of the crop's actual measurement information when a growth index is input.

[0008] Another aspect of the information processing method of the present invention is an information processing device that performs the following processes: deriving a crop growth index based on satellite images of agricultural land taken from a satellite; and deriving a predicted value of actual measurement information of the crop by inputting the growth index into a trained model; the trained model is created based on crop growth-related information generated based on aerial images of the agricultural land taken from above and the actual measurement information of the crop that is the source of the growth-related information; the trained model uses the output value of a regression model that outputs a predicted value of the actual measurement information of the crop when at least the crop growth-related information is input as training data, and a crop growth index created based on the satellite image as training data; and is trained to output a predicted value of the actual measurement information of the crop when a growth index is input.

[0009] Another aspect of the present invention is a program that causes a processor of an information processing device to execute a process of deriving a crop growth index based on satellite images of agricultural land taken from a satellite, and a process of deriving a predicted value of actual measurement information of the crop by inputting the growth index into a trained model, wherein the trained model is created based on crop growth-related information generated based on aerial images of the agricultural land taken from above and the actual measurement information of the crop that is the source of the growth-related information, and uses the output value of a regression model that outputs a predicted value of the actual measurement information of the crop when at least the crop growth-related information is input as training data, and a crop growth index of the crop created based on the satellite image as training data, and is trained to output a predicted value of the actual measurement information of the crop when a growth index is input. [Effects of the Invention]

[0010] According to each of the above aspects, it is possible to predict actual measurement information without increasing the processing effort. [Brief explanation of the drawings]

[0011] [Figure 1] 1 is a diagram illustrating an example of a usage environment and configuration of an information processing device 100. FIG. [Figure 2] FIG. 2 is a diagram schematically illustrating the content of processing by each unit of the information processing device 100 in the learning stage. [Figure 3] FIG. 10 is a diagram showing height information generated for a certain farmland N (field). [Figure 4] FIG. 10 is a diagram schematically illustrating how a regression model 172 is generated. [Figure 5] FIG. 1 is a diagram showing the processing of each part of the information processing device 100 in chronological order, including the learning stage and the inference stage. [Figure 6] FIG. 10 is a diagram showing the results of a demonstration experiment of the present invention. DETAILED DESCRIPTION OF THE INVENTION

[0012] Hereinafter, with reference to the drawings, embodiments of an information processing device, an information processing method, and a program according to the present invention will be described. The information processing device of the present invention derives predicted values of actual crop measurement information by inputting crop growth indicators derived from satellite images of farmland captured from a satellite into a trained model. The actual measurement information may be, for example, yield, but may also include nutritional value (per unit yield or unit area), the presence or absence of disease, the proportion of diseased plants, the proportion of weeds, etc. The trained model is created based on crop growth-related information generated based on aerial images captured from above. The growth-related information may be, for example, crop height information, but is not limited to this, and may also include color information, the number of seedlings, or a combination of these. The information processing device aims to improve the accuracy of yield prediction without increasing processing effort. In the following description, the growth-related information is crop height information, and the crop is assumed to be corn, which has a high correlation between plant height (height) and yield. The information processing device is implemented by one or more processors. This processor may be a rental server on the cloud or a dedicated product for implementing the invention.

[0013] FIG. 1 is a diagram illustrating an example of the usage environment and configuration of an information processing device 100. For example, the information processing device 100 acquires images (satellite images, more specifically, satellite constellation images from multiple periods provided in cooperation with multiple satellites, hereinafter simply referred to as satellite images) captured by one or more satellites 10 via a satellite image provider server 20 and a network NW, and also acquires images (aerial images) captured by a drone 30 via a relay device 40 and a network NW. The satellite images are images of farmland N on the ground. The aerial images are images captured multiple times at different angles while the drone 30 is flying over the farmland N. In the present invention, the aerial images are used exclusively in the generation stage (learning stage) of the regression model and the trained model, and the growth indexes generated from the satellite images are used exclusively as input data for yield prediction.

[0014] The information processing device 100 provides output information to the terminal device 50 via, for example, a network NW. The network NW is any network such as a LAN, a WAN, or an internet line, and may be wired or wireless. Note that the method of acquiring an image is not limited to this, and an image may be acquired by loading an image stored on a storage medium into a drive device of the information processing device 100. Furthermore, the image may have been preprocessed before being passed to the information processing device 100.

[0015] The satellite 10, for example, orbits the Earth and periodically captures images of the ground. The satellite image provider server 20 provides the images captured by the satellite 10 to the information processing device 100 and the like.

[0016] The drone 30 is an unmanned aerial vehicle equipped with various imaging devices. The drone 30 repeatedly captures images of the ground while flying in accordance with instructions from the relay device 40. The drone 30 stores multiple drone images, along with camera parameters and location information such as GPS, on a memory card installed in the drone 30 or transmits the images to the relay device 40.

[0017] The relay device 40 is, for example, a terminal device such as a tablet terminal. The relay device 40 is equipped with an application for operating the drone 30 and for receiving and displaying images captured by the drone 30. The relay device 40 displays the images captured by the drone 30 as RGB images and transmits them to the information processing device 100 via the network NW.

[0018] The terminal device 50 is a computer device such as a personal computer, a smartphone, a tablet terminal, etc. The terminal device 50 communicates with the information processing device 100 via a network NW and displays output information received from the information processing device 100 (described later). The relay device 40 and the terminal device 50 may be integrated into one device.

[0019] The information processing device 100 has, for example, the functionality of a web server. The information processing device 100 includes, for example, an acquisition unit 110, a growth index derivation unit 120, a learning unit 130, a yield prediction unit 140, a provision unit 150, and a storage unit 170. The learning unit 130 includes, for example, a height information generation unit 132, a regression model generation unit 134, and a machine learning processing unit 136. The components other than the storage unit 170 are realized by, for example, a processor such as a CPU (Central Processing Unit) executing a program (software). Some or all of these components may be realized by hardware (including circuitry) such as an LSI (Large Scale Integration), an ASIC (Application Specific Integrated Circuit), an FPGA (Field-Programmable Gate Array), or a GPU (Graphics Processing Unit), or may be realized by a combination of software and hardware. The program may be stored in advance in a storage device (a storage device having a non-transitory storage medium) such as an HDD (Hard Disk Drive) or flash memory, or may be stored in a removable storage medium (a non-transitory storage medium) such as a DVD or CD-ROM, and installed in the storage device by inserting the storage medium into a drive device. The storage unit 170 stores information such as the regression model 172 and the trained model 174 generated by the learning unit 130.

[0020] 2 is a diagram schematically illustrating the processing content of each unit of the information processing device 100 in the learning stage. The acquisition unit 110 acquires aerial images, satellite images, and information on the actual yield of the crop (actually measured yield). The aerial images include a first aerial image taken from above of bare farmland N, and a second aerial image taken from above of farmland N before the crop is harvested (when the grass has fully grown before harvesting). The acquisition unit 110 acquires this information via the network NW or via an input device (not shown).

[0021] The growth index derivation unit 120 generates a growth index image (hereinafter simply referred to as growth index) with growth indexes as pixel values based on the pixel values of the satellite image. The growth index is, for example, NDVI (Normalized Difference Vegetation Index) expressed by the following formula (1). In the formula, Ref NIR is the reflectance of the near-infrared wavelength band (hereinafter referred to as the band), Ref Red is the reflectance of the red light band. Alternatively, the growth index may be the Green Normalized Difference Vegetation Index (GNDVI), the Normalized Difference Red Edge (NDRE), the Leaf Chlorophyll Index (LCI), etc. In the following description, the growth index is assumed to be NDVI.

[0022] NDVI =(Ref NIR -Ref Red ) / (Ref NIR +Ref Red ) …(1)

[0023] The height information generation unit 132 of the learning unit 130 calculates the depth (distance from the imaging position) for each pixel in a plurality of aerial images captured from different imaging positions, for example, using a three-dimensional reconstruction technique that applies the principles of a stereo camera. Then, a three-dimensional model is generated by converting the distance from the imaging position into absolute coordinates on the ground. The height information generation unit 132 generates a first digital surface model based on the first aerial image and a second digital surface model based on the second aerial image. Furthermore, the height information generation unit 132 generates height information indicating the height of the crops before harvest by performing a vertical difference calculation between the first digital surface model and the second digital surface model. Figure 3 is a diagram showing height information generated for a certain farmland N (field).

[0024] Returning to Fig. 2, the regression model generation unit 134 generates a regression model based on the height information generated by the height information generation unit 132 and the actual yield of the crop that is the source of the height information (the yield when the crop was actually harvested after the second aerial image was acquired). For example, the regression model generation unit 134 generates a regression model 172 (a function that outputs the yield when height information is input) by fitting the relationship between the height information and the actual yield to a geometric figure such as a quadratic curve or a straight line based on the least squares method or the like. Fig. 4 is a diagram schematically showing how the regression model 172 is generated.

[0025] Returning to FIG. 2 , once the regression model 172 is generated, the machine learning processing unit 136 uses, as training data, the aerial yield prediction value (the output value of the regression model) obtained by inputting height information generated at a different time into the regression model 172 at a later date, and time-series data of a crop growth index created based on satellite images, for example, as training data, and learns (trains) the parameters of the machine learning model so that a predicted value of crop yield is output when the time-series data of the growth index is input, thereby generating a trained model 174. The trained model 174 is trained, for example, by a deep neural network (DNN), but there are no particular restrictions on the form of the trained model 174. The machine learning processing unit 136 generates the trained model 174 by, for example, a backpropagation technique.

[0026] 5 is a diagram showing the chronological order of the processing of each part of information processing device 100, including the learning stage and the inference stage. First, in the first stage for generating regression model 172, regression model 172 is generated based on the first aerial image, the second aerial image, and the measured yield. In this first stage, regression model 172 is generated using information acquired as, for example, planting and harvesting are performed multiple times on the same farmland.

[0027] In the subsequent second period, a trained model 174 is generated based on the first aerial image, the second aerial image, and a predicted aerial yield obtained using the regression model 172, as well as growth indices based on satellite images acquired during the same period. As in the first period, the trained model 174 in this second period may be generated using information acquired as planting and harvesting are performed multiple times on the same farmland.

[0028] In the third stage, which is the inference stage, the yield prediction unit 140 predicts the yield of the crop that is the source of the satellite image (satellite yield prediction value) by inputting the satellite image into the trained model 174. The providing unit 150 outputs information on the satellite yield prediction value to the terminal device 50 or the like.

[0029] By performing this processing, yield predictions can be made without the hassle of flying a drone to take aerial photographs each time a yield prediction is made. Aerial photography using a drone requires skilled techniques, and it is difficult to obtain wide-area information at once. In contrast, satellite constellation images are captured frequently over a wide area, making them applicable to a variety of regions. In other words, once the regression model 172 and trained model 174 are generated, yield predictions can be made simply by inputting growth indices based on satellite images that can be easily obtained over a wide area. Therefore, yield predictions can be made without increasing the processing effort.

[0030] The inventors of the present application conducted a demonstration experiment to examine the correlation between aerial photography yield predictions and satellite-based yield predictions. Specifically, satellite-based yield predictions were calculated using the above-described method, and a correlation diagram was created to determine what would have happened if aerial photography yield predictions had been calculated for the same farmland N at the same time. Figure 6 shows the results of the demonstration experiment of the present invention. As shown in the figure, although there were some outliers, it was confirmed that the results showed a high correlation overall.

[0031] According to each of the embodiments described above, it is possible to predict actual measurement information without increasing the processing effort.

[0032] While the present invention has been described above using embodiments, it is not limited to these embodiments and various modifications and substitutions can be made without departing from the spirit of the present invention. For example, the input information to the regression model may include not only information related to crop growth, but also other information such as meteorological information, variety information, or cultivation management information such as planting density and fertilizer amount. [Explanation of symbols]

[0033] 10 satellites 30 Drone 40 Relay Device 50 Terminal Equipment 100 Information processing device 110 Acquisition Department 120 Growth index derivation part 130 Learning Department 132 Height information generation unit 134 Regression model generation unit 136 Machine Learning Processing Unit 140 Yield Forecasting Department 150 Provision Department

Claims

1. a growth index deriving unit that derives a crop growth index based on a satellite image of farmland captured by a satellite; a yield prediction unit that derives a predicted value of the actual measurement information of the crop by inputting the growth index into a trained model; Equipped with The trained model is created based on the crop growth-related information generated based on aerial images of the farmland taken from above and the actual measurement information of the crop that is the source of the growth-related information, and uses the output value of a regression model that outputs a predicted value of the actual measurement information of the crop when at least the crop growth-related information is input as training data, and a crop growth index created based on the satellite image as training data, and is trained to output a predicted value of the actual measurement information of the crop when a growth index is input. Information processing device.

2. The growth index is time series data derived for multiple time points, The trained model is trained using the output value of the regression model as training data and the time series data of the growth index as training data.

2. The information processing device according to claim 1.

3. The growth-related information is crop height information, The actual measurement information is yield.

2. The information processing device according to claim 1.

4. The height information is generated based on a difference between a first aerial image of the farmland in a bare state taken from above and a second aerial image of the farmland before the crops are harvested taken from above.

4. The information processing device according to claim 3.

5. The crop is corn.

2. The information processing device according to claim 1.

6. Further comprising a learning unit that generates the trained model.

2. The information processing device according to claim 1.

7. The information processing device A process of deriving a crop growth index based on satellite images of farmland taken from a satellite; and deriving a predicted value of the actual measurement information of the crop by inputting the growth index into a trained model; The trained model is created based on the crop growth-related information generated based on aerial images of the farmland taken from above and the actual measurement information of the crop that is the source of the growth-related information, and uses the output value of a regression model that outputs a predicted value of the actual measurement information of the crop when at least the crop growth-related information is input as training data, and a crop growth index created based on the satellite image as training data, and is trained to output a predicted value of the actual measurement information of the crop when a growth index is input. Information processing methods.

8. The processor of the information processing device A process of deriving a crop growth index based on satellite images of farmland taken from a satellite; and executing a process of deriving a predicted value of the actual measurement information of the crop by inputting the growth index into a trained model; The trained model is created based on the crop growth-related information generated based on aerial images of the farmland taken from above and the actual measurement information of the crop that is the source of the growth-related information, and uses the output value of a regression model that outputs a predicted value of the actual measurement information of the crop when at least the crop growth-related information is input as training data, and a crop growth index created based on the satellite image as training data, and is trained to output a predicted value of the actual measurement information of the crop when a growth index is input. program.

Images