Information processing apparatus, information processing method, and program

Abstract

To enable the prediction of a yield even in the middle of growth while considering a change with time in a growth state by comprehensively using data affecting the growth of a crop.SOLUTION: An acquisition unit configured to acquire a time-series image obtained by imaging a farm field from above and time-series growth-related data related to growth of a crop in the farm field, and a derivation unit configured to derive one or more predicted yields of the crop in one or more predicted target sections of the farm field by inputting the time-series image and the time-series growth-related data to a trained model; Wherein the learned model is obtained by learning in advance a machine learning model including a convolutional neural network and a recurrent neural network so that a loss function including an error between the one or more predicted yields at each time point in the time series and a value indicated by the acquired yield data becomes small for one or more learning target sections in the agricultural field.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present invention relates to an information processing device, an information processing method, and a program. [Background technology]

[0002] Conventionally, there are known techniques for predicting the yield of crops in a target plot of farmland. For example, Patent Document 1 discloses a technique for deriving the expected yield of crops in a target plot by inputting data representing the state of vegetation obtained from time-series images of farmland taken from above and time-series weather information into a trained model. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Application Publication No. 2024-064151 Summary of the Invention [Problem to be solved by the invention]

[0004] However, the conventional techniques do not take into account changes in the state of crop growth over time and do not consider yield prediction during the growth of the crop. Furthermore, in many cases, the conventional techniques predict yield using only images, and do not comprehensively use data that affects crop growth.

[0005] The present invention has been made in consideration of the above circumstances, and one of its objects is to provide an information processing device, an information processing method, and a program that can comprehensively use data that affects crop growth, take into account changes in growth conditions over time, and make it possible to predict yield even during growth. [Means for solving the problem]

[0006] An information processing device according to one embodiment of the present invention comprises an acquisition unit that acquires time-series images of agricultural land taken from above and time-series growth-related data relating to the growth of crops in the agricultural land, and a derivation unit that derives one or more predicted yields of the crops for one or more prediction target sections of the agricultural land by inputting the time-series images and the time-series growth-related data into a trained model, wherein the trained model is a machine learning model that has been trained in advance, including a convolutional neural network and a recurrent neural network, so as to reduce a loss function that includes an error between the one or more predicted yields at each point in the time series and the value indicated by the acquired yield data for one or more training target sections of the agricultural land.

[0007] Another aspect of the information processing method of the present invention involves a computer acquiring time-series images of farmland taken from above and time-series growth-related data relating to the growth of crops in the farmland, and inputting the time-series images and the time-series growth-related data into a trained model to derive one or more predicted yields of the crops for one or more prediction target sections of the farmland, wherein the trained model is a machine learning model that has been pre-trained, including a convolutional neural network and a recurrent neural network, so as to reduce a loss function that includes an error between the one or more predicted yields at each point in the time series and the value indicated by the acquired yield data for one or more training target sections of the farmland.

[0008] Another aspect of the present invention provides a program that causes a computer to acquire time-series images of farmland taken from above and time-series growth-related data regarding the growth of crops in the farmland, and inputs the time-series images and the time-series growth-related data into a trained model to derive one or more predicted yields of the crops for one or more prediction target sections of the farmland, wherein the trained model is a machine learning model that has been trained in advance, including a convolutional neural network and a recurrent neural network, so as to reduce a loss function that includes an error between the one or more predicted yields at each point in the time series and the value indicated by the acquired yield data for one or more training target sections of the farmland.

[0009] Another aspect of the present invention provides an information processing device that includes an acquisition unit that acquires time-series images of agricultural land taken from above, time-series growth-related data regarding the growth of crops in the agricultural land, and crop yield data for one or more target plots of the agricultural land, and a learning unit that generates a trained model that derives one or more expected yields of the crops in the one or more target plots by inputting the time-series images and the time-series growth-related data, and the learning unit generates the trained model by learning parameters of a machine learning model that includes a convolutional neural network and a recurrent neural network so as to reduce a loss function that includes an error between the one or more expected yields at each point in the time series and the value indicated by the crop yield data for the one or more target plots.

[0010] Another aspect of the information processing method of the present invention involves acquiring time-series images of agricultural land taken from above, time-series growth-related data related to crop growth in the agricultural land, and crop yield data for one or more target plots of the agricultural land, and generating a trained model that derives one or more predicted yields of the crops in the one or more target plots by inputting the time-series images and the time-series growth-related data, and generating the trained model by learning parameters of a machine learning model that includes a convolutional neural network and a recurrent neural network so as to reduce a loss function that includes an error between the one or more predicted yields at each point in the time series and the value indicated by the crop yield data for the one or more target plots.

[0011] Another aspect of the present invention provides a program that causes a computer to acquire time-series images of agricultural land taken from above, time-series growth-related data related to crop growth in the agricultural land, and crop yield data for one or more target plots of the agricultural land, and by inputting the time-series images and the time-series growth-related data, generates a trained model that derives one or more predicted yields of the crops in the one or more target plots, and generates the trained model by learning parameters of a machine learning model that includes a convolutional neural network and a recurrent neural network so as to reduce a loss function that includes an error between the one or more predicted yields at each point in the time series and the value indicated by the crop yield data for the one or more target plots. [Effects of the Invention]

[0012] According to each of the above aspects, it is possible to comprehensively use data that affects crop growth, and to predict yield even during growth while taking into account changes in growth conditions over time. [Brief explanation of the drawings]

[0013] [Figure 1] FIG. 1 is a diagram illustrating an example of a usage environment of an information processing device 100. [Figure 2]1 is a configuration diagram of an information processing device 100. FIG. [Figure 3] 10 is a diagram for explaining the processing at the inference stage executed by the information processing device 100. FIG. [Figure 4] FIG. 1 is a diagram illustrating the configuration of a trained model 152. [Figure 5] FIG. 10 is a diagram for explaining the processing of a learning unit 140. [Figure 6] FIG. 10 is a diagram illustrating an example of an interpolation method for a farm field image. [Figure 7] 10 is a flowchart showing an example of the flow of processing at an inference stage executed by the information processing device 100. [Figure 8] 10 is a flowchart showing an example of the flow of processing in a learning stage executed by the information processing device 100. [Figure 9] FIG. 10 is a diagram illustrating the configuration of a trained model 152 according to the second embodiment. [Figure 10] FIG. 10 is a diagram illustrating the processing of a learning unit 140 according to the second embodiment. [Figure 11] 10 is a flowchart showing an example of the flow of processing at the inference stage executed by the information processing device 100 according to the second embodiment. [Figure 12] 10 is a flowchart showing an example of the flow of processing in a learning stage executed by the information processing device 100 according to the second embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0014] Hereinafter, an information processing device, an information processing method, and a program according to the present invention will be described with reference to the drawings. Note that in the drawings, vectors are shown in bold, but in the specification, a symbol with a "#" after or within it indicates a vector.

[0015] [composition] FIG. 1 is a diagram illustrating an example of an environment in which an information processing device 100 is used. The information processing device 100 acquires, for example, images (satellite images) of farmland F captured by one or more satellites 10 via a satellite image provider server 20 and a network NW, and also acquires various information from one or more public information providing servers 30 via the network NW, such as plot polygon data, meteorological information for the farmland F (e.g., ground surface temperature, difference between daily maximum and minimum temperatures, and global solar radiation), soil characteristics for the farmland F (e.g., soil type, humus content, ground surface reflectance for a specific month of the previous year prior to the harvest year), and the growth status of crops on the farmland F (e.g., leaf area index, photosynthetically active radiation absorption rate). The satellite images are images of the farmland F captured repeatedly on the ground. Time-series satellite images are an example of "time-series images of farmland captured from above." The time-series meteorological information, soil characteristics, and growth status are also examples of "growth-related data." Instead of satellite images, images of the farmland F taken by a drone or a camera attached to a steel tower may be used.

[0016] Note that satellite images do not need to be acquired frequently (e.g., daily); satellite images may be interpolated from a predetermined number of satellite images using the interpolation method described below, or images and data corresponding to period averages or each growth stage may be used. Furthermore, soil properties may be estimated using known methods based on data separately obtained from a satellite 10, or may be values ​​obtained from a digital soil map or the like. Furthermore, growth conditions may be obtained from data separately obtained from a satellite 10 or calculations using a crop growth model.

[0017] The information processing device 100 provides information to the terminal device 50 via a network NW. The network NW is, for example, any network such as a LAN, a WAN, or an internet line, and may be wired or wireless. Note that the method of acquiring satellite images is not limited to this, and satellite images may also be acquired by loading images stored on a storage medium into a drive device of the information processing device 100. Furthermore, the satellite images may have been preprocessed before being passed to the information processing device 100.

[0018] The satellite 10, for example, orbits the Earth and periodically captures images of the ground. The satellite image provider server 20 provides the images captured by the satellite 10 to the information processing device 100 and the like.

[0019] The public information providing server 30 is a server operated by a public institution such as the Ministry of Agriculture, Forestry and Fisheries or the Japan Meteorological Agency, a network distributor requested by such an institution, or a private company that secondarily provides information provided by the public institution.

[0020] The terminal device 50 is a computer device such as a personal computer, a smartphone, a tablet terminal, etc. The terminal device 50 communicates with the information processing device 100 via the network NW, and displays information received from the information processing device 100.

[0021] FIG. 2 is a configuration diagram of an information processing device 100. The information processing device 100 has, for example, the functionality of a web server. The information processing device 100 includes, for example, an acquisition unit 110, a derivation unit 120, an information provision unit 130, a learning unit 140, and a storage unit 150. Information such as a trained model 152 is stored in the storage unit 150. The components other than the storage unit 150 are realized, for example, by a processor such as a CPU (Central Processing Unit) executing a program (software). Some or all of these components may be realized by hardware (including circuitry) such as an LSI (Large Scale Integration), an ASIC (Application Specific Integrated Circuit), an FPGA (Field-Programmable Gate Array), or a GPU (Graphics Processing Unit), or may be realized by a combination of software and hardware. The program may be stored in advance in a storage device such as an HDD (Hard Disk Drive) or flash memory (a storage device with a non-transitory storage medium), or may be stored in a removable storage medium (a non-transitory storage medium) such as a DVD or CD-ROM, and installed in the storage device by inserting the storage medium into a drive device.

[0022] The functions of each unit of the information processing device 100 will be described below with reference to FIG. 3. FIG. 3 is a diagram for explaining the processing of the inference stage executed by the information processing device 100. The acquisition unit 110 acquires, as described above, brush polygon data, time-series satellite images from time t=1 to time t=n, meteorological information, soil characteristics, growth conditions, and other information using a communication interface (not shown). Here, time t=1 represents the start time when acquisition of this data regarding the farmland F begins, and time t=n represents the inference time when yield estimation is performed. The acquisition unit 110 may acquire this information automatically, or may acquire it by accepting an operation from an operator as necessary. For example, the operator may download data from a website, store the data at a specified address in the storage unit 150 of the information processing device 100, and the acquisition unit 110 may then read the downloaded data from the storage unit 150.

[0023] [Inference stage] First, the acquisition unit 110 performs a process of dividing the satellite image into fields using satellite images and brush polygon data to acquire field images. Brush polygon data is land division data provided by the Ministry of Agriculture, Forestry and Fisheries. Incidentally, each crop (wheat, paddy rice, wheat, soybeans, sugar beets, potatoes, etc.) exhibits unique temporal changes in vegetation conditions. Therefore, for example, fields containing target crops can be extracted using data representing time-series vegetation conditions generated from satellite images divided into fields. The data representing vegetation conditions is index data indicating the characteristics of light reflection by plants, and includes data such as NDVI (Normalized Difference Vegetation Index) and reflectance. The satellite images of the fields containing the target crops are used for subsequent processing. These fields are an example of "target divisions." Similarly, the acquisition unit 110 can acquire weather information, soil characteristics, and growth conditions for each field by collating the plot data and brush polygon data from which this information was obtained, as necessary. Note that the acquisition unit 110 may distinguish each field in a single satellite image and acquire the weather information, soil characteristics, and growth conditions associated with each field, without dividing the satellite image into multiple field images.

[0024] Furthermore, the acquiring unit 110 may generate data representing the state of vegetation over time from the time-series satellite images. The data representing the state of vegetation is index data indicating the characteristics of light reflection by plants, and includes data such as NDVI (Normalized Difference Vegetation Index) and reflectance. The acquiring unit 110 may perform processing to exclude areas estimated to be free of crops from the farm field image. In this case, the acquiring unit 110 excludes, for example, pixels whose NDVI indicates an abnormal value (such as a value outside the range of plus or minus 2σ relative to the average of the entire image).

[0025] The derivation unit 120 inputs time-series field images (which may include NDVI and reflectance data), meteorological information, soil properties, and growth conditions into the trained model 152 to derive a predicted crop yield for each of one or more prediction target plots of farmland. As described below, the trained model 152 is a machine learning model that has been trained in advance, including a convolutional neural network (CNN) and a recurrent neural network (RNN), so as to minimize a loss function, which includes an error between one or more predicted yields at each time point (time point t = 1 to n) in the time series and the values ​​indicated by the acquired yield data (actual measured values), for one or more training target plots of farmland. The RNN includes a neural network architecture, such as a long short-term memory (LSTM), that can take into account information forgetting in long-term memory. Note that the prediction target plot and the training target plot described above may be different plots.

[0026] The information providing unit 130 provides information based on the processing results of the derivation unit 120 to the terminal device 50. The information providing unit 130 may provide the expected yield as numerical data as is, or may provide it as image data in which each field in a satellite image is linked to the expected yield (for example, by superimposing the expected yield value on each field). While conventional techniques predict the yield at the final stage of crop growth, the trained model 152 according to this embodiment can output the expected yield at any inference time point n. Therefore, the provided expected yield data can be used, for example, as evidence for formulating a crop cultivation management method for each period. That is, according to this embodiment, the expected yield data can be provided to the terminal device 50 at any timing during crop growth, improving convenience for information users. Furthermore, unlike conventional techniques, it is possible to eliminate the need to build a model for each period. The configuration of the trained model 152 according to this embodiment is described below.

[0027] [Trained model] FIG. 4 is a configuration diagram of the trained model 152. The configuration diagram of the trained model 152 shown in FIG. 4 shows the internal configuration of the trained model 152 shown in FIG. 3. In addition, in FIG. 4, the weather information, soil properties, and growth conditions shown in FIG. 3 are collectively shown as growth-related data. As shown in FIG. 4, the trained model 152 includes a CNN that receives time-series field images and growth-related data as input. The CNN extracts features by convolving (reducing the resolution of) the input field images and growth-related data at each time point, and outputs an intermediate feature map. The value of the intermediate feature map output for a certain time point (t=k) is passed by the RNN to the CNN for the next time point (t=k+1), where it is input together with the field images and growth-related data for the next time point (t=k+1).

[0028] Meanwhile, the intermediately output feature map is further convolved to finally obtain the expected yield y k #=[y k1 ,y k2 ,…,y km ] where the expected yield y k # denotes the predicted yield at harvest time (e.g., several months later) predicted based on the time-series information available up to each time point k. Furthermore, symbols 1 to m denote the identifiers of each field. That is, the predicted yield y k # is a vector value representing the expected yield at the harvest time for each field, predicted at each time point k. The expected yields y1#, y2#, y n #, y n # is the expected yield taking into account all the time-series information available up to the inference time point n. Therefore, the information providing unit 130 calculates the expected yields y1#, y2#, . . . y n # of expected yield y n Alternatively, only information relating to # may be provided to the terminal device 50.

[0029] As described above, the trained model 152 according to this embodiment is composed of both CNN and RNN, and is therefore capable of taking into account changes in growth variability within a field over time. Furthermore, the trained model 152 according to this embodiment can take into account spatial variability in the growth conditions of crops within a field by outputting a predicted yield for each field included in the farmland. This improves the accuracy of inferences made by the trained model 152.

[0030] [Learning stage] The learning unit 140 will be described below. In this embodiment, the learning unit 140 is a part of the information processing device 100, but the learning process and inference process in machine learning may be implemented by a separate device. Therefore, the learning unit 140 may be a function of a device separate from the derivation unit 120, etc. In this case, the separate device stores, in addition to the learning unit 140, a machine learning model from which the trained model 152 is generated and yield data that serves as training data.

[0031] 5 is a diagram for explaining the processing of the learning unit 140. In FIG. 5, the left part represents the output of the machine learning model from which the trained model 152 is generated, and the right part represents yield data (actual measurement values) of the crop actually harvested at the harvest time. First, the learning unit 140 calculates the predicted yield y at each time point k output by the machine learning model. k Mean square error MSE between # and the actual measured value y# k is calculated according to the following formula (1).

[0032]

number

[0033] The learning unit 140 calculates the mean square error MSE k After calculating, these values ​​are used to define the loss function loss according to the following equation (2).

[0034]

number

[0035] In equation (2), r k is y k represents the correlation coefficient between # and y#, and w k represents the weight that takes into account the period from each time point k to the harvest date. k The higher the correlation coefficient between # and y#, the k Since k can approximate y# with higher accuracy, its influence on the magnitude of the loss function loss is evaluated to be smaller. Furthermore, the closer the time point k is to the harvest date, the smaller the predicted yield y output by the machine learning model. k Since the accuracy of # is expected to be high, the weight w k is set to a larger value the closer the time point k is to the harvest date.

[0036] The learning unit 140 learns the parameters of the machine learning model by backpropagation or the like so as to reduce (approach zero) the loss function loss defined by equation (2), and sets the machine learning model at the point when the convergence condition is satisfied or a certain number of learning rounds are completed as the trained model 152. The loss function loss is calculated from equation (2) by subtracting the term 1-r k However, verification by the inventors has revealed that the term 1-r relating to the correlation coefficient k It was confirmed that the accuracy of the trained model 152 was improved by using a loss function loss including

[0037] Interpolate Data Thus, in this embodiment, in both the learning stage and the inference stage, time-series field images of the farmland F captured by the satellite 10 are used as input to the pre-learning machine learning model and the trained model 152. However, for example, due to bad weather such as cloudy or rainy weather, the satellite 10 may fail to capture images of the farmland F or the captured images may not be of good quality. In such cases, the field images can be interpolated to obtain field images for learning or inference, as described below.

[0038] FIG. 6 is a diagram showing an example of a method for interpolating field images. The upper part of FIG. 6 shows the time-series field images before interpolation, and the lower part of FIG. 6 shows the time-series field images after interpolation. As an example, FIG. 6 shows a situation in which field images at time points t2, t3, and t4 are missing. If the reflectance of each point in the field image at time point t1 is represented as ref1 and the reflectance of each point in the field image at time point t5 is represented as ref5, the acquisition unit 110 acquires the reflectance of each point in the field image at time point t i Reflectance ref of each point in the field image at (i=2, 3, 4) i , ref i =ref1+(ref5-ref1)(i-1) / 4. i Reflectance ref at (i=2, 3, 4) i , the acquisition unit 110 calculates the reflectance ref i From time t i (i=2, 3, 4) can be generated. This makes it possible to prepare field images for learning or inference even if, for example, imaging of the farmland F fails or the captured images are of poor quality due to bad weather such as cloudy or rainy weather.

[0039] The interpolation of field images is not limited to the interpolation using a linear function based on two points described above, and any interpolation algorithm such as polynomial interpolation may be used if there are three or more correctly acquired field images. Furthermore, for example, crop growth can be simulated by inputting daily weather information into an existing crop growth model, and the growth can then be input into an existing plant canopy radiative transfer model to simulate reflectance and vegetation index. Therefore, field images may be interpolated based on the reflectance obtained in this manner.

[0040] [Processing flow] 7 and 8, the flow of processing at the inference stage and the learning stage executed by the information processing device 100 will be described. Fig. 7 is a flowchart showing an example of the flow of processing at the inference stage executed by the information processing device 100.

[0041] First, the acquisition unit 110 acquires time-series field images and growth-related data from the start time t1 to the inference time tn (step S100). Next, the derivation unit 120 inputs the acquired time-series field images and growth-related data into the trained model 152, and derives the expected yield for each field (step S102). Next, the information provision unit 130 provides information related to the derived expected yield to the terminal device 50. This completes the processing of this flowchart.

[0042] 8 is a flowchart showing an example of the flow of processing in the learning stage executed by the information processing device 100. First, the learning unit 140 acquires time-series field images and growth-related data, and actual measured values ​​of yield data (step S200). Next, the learning unit 140 calculates a loss function that represents the error between the predicted yield obtained by inputting the time-series field images and growth-related data into the machine learning model and the yield data (actual measured values) (step S202). Next, the learning unit 140 learns parameters of the machine learning model so as to reduce the loss function (step S204). This completes the processing of this flowchart.

[0043] According to the present embodiment described above, time-series images of farmland captured from above and time-series growth-related data related to crop growth on the farmland are acquired, and the time-series images and time-series growth-related data are input into a trained model to derive one or more predicted crop yields for one or more target plots of farmland. The trained model is a machine learning model including a convolutional neural network and a recurrent neural network that has been trained in advance so as to minimize a loss function that includes an error between one or more predicted yields at each time point in the time series and the values ​​indicated by the acquired yield data for one or more training target plots of farmland. This makes it possible to predict yields even during growth by comprehensively using data that affects crop growth and taking into account changes in growth conditions over time.

[0044] [Second embodiment] In the embodiment described above, the trained model 152 is trained to output only the predicted yield for each prediction target plot. However, the present invention is not limited to such a configuration, and the trained model 152 may also be trained to output other growth quantities that can be predicted at the same time as the yield. That is, in the second embodiment, the trained model 152 may be trained to output multidimensional values. For example, if the crop is wheat, the growth quantities that can be predicted at the same time as the yield include the grain protein content (the ratio of the amount of protein in the grain to the grain weight), the number of ears, and the grain weight (the weight of the wheat grain). In the following description, as an example, the crop is wheat and the growth quantity is the grain protein content. However, for crops other than wheat, growth quantities specific to each crop can also be simultaneously estimated.

[0045] FIG. 9 is a configuration diagram of a trained model 152 according to the second embodiment. The trained model 152 shown in FIG. 9 differs from the trained model 152 shown in FIG. 4 only in that it is configured to output a grain protein content. As in FIG. 4, the trained model 152 includes a CNN that receives time-series field images and growth-related data as input. The CNN extracts features by convolving (reducing the resolution of) the input field images and growth-related data at each time point, and outputs an intermediate feature map. The intermediate feature map values ​​output for a certain time point (t=k) are passed by the RNN to the CNN for the next time point (t=k+1), where they are input together with the field images and growth-related data for the next time point (t=k+1).

[0046] Meanwhile, the intermediately output feature map is further convolved to finally obtain the expected yield y k #=[y k1 ,y k2 ,…,y km ] and predicted grain protein content p k #=[p k1 ,p k2 ,…,p km ] where the predicted grain protein content p k# means the predicted grain protein content at harvest time (e.g., several months later) predicted based on the time series information available up to each time point k. That is, the predicted grain protein content p k # is a vector value representing the predicted grain protein content at harvest time for each field, predicted at each time point k. The predicted grain protein contents p1#, p2#, p n #, p n # is a predicted grain protein content taking into consideration all the time-series information available up to the inference time point n. Therefore, the information providing unit 130 calculates the predicted grain protein contents p1#, p2#, . . . p n Among #, predicted grain protein content p n Alternatively, only information relating to # may be provided to the terminal device 50.

[0047] FIG. 10 is a diagram for explaining the processing of the learning unit 140 according to the second embodiment. In FIG. 10, the left part represents the output (predicted grain protein content) of the machine learning model from which the trained model 152 was generated, and the right part represents grain protein content data (actual measured value) actually measured at harvest time. The grain protein content data may be, for example, the average grain protein content of crops harvested for each field. Furthermore, although not shown in FIG. 10, the machine learning model according to the second embodiment also outputs the predicted yield shown in FIG. 5.

[0048] First, the learning unit 140 calculates the predicted yield y at each time point k output by the machine learning model. k The mean square error MSE1 between # and the actual measured value y# k is calculated according to the following formula (3): Formula (3) is the same as formula (1) except for the notation.

[0049]

number

[0050] Furthermore, the learning unit 140 calculates the predicted grain protein content p at each time point k output by the machine learning model. k The mean square error MSE2 between # and the actual measured value p# k is calculated according to the following formula (4).

[0051]

number

[0052] The learning unit 140 calculates the mean square error MSE1 k and MSE2 k After calculating, these values ​​are used to define the loss function loss according to the following equation (5).

[0053]

number

[0054] In equation (5), r1 k is y k represents the correlation coefficient between # and y#, r2 k is p k represents the correlation coefficient between # and p#, and w k represents the weight that takes into account the period from each time point k to the harvest date. Correlation coefficient r1 k The higher the value, the more accurately yk# approximates y#, and the correlation coefficient r2 k The higher the k Since # can more accurately approximate p#, its influence on the magnitude of the loss function loss is evaluated to be smaller. Furthermore, the closer the time point k is to the harvest date, the smaller the predicted yield y output by the machine learning model. k # and predicted grain protein content p k Since the accuracy of # is expected to be high, the weight w k is set to a larger value the closer the time point k is to the harvest date.

[0055] The learning unit 140 learns the parameters of the machine learning model by backpropagation or the like so as to reduce (approach zero) the loss function loss defined by equation (5), and the machine learning model at the point when it satisfies the convergence condition or completes a certain number of learning rounds is designated as the trained model 152 according to the second embodiment.

[0056] [Processing flow] Next, the processing flow of the inference stage and the learning stage executed by the information processing device 100 according to the second embodiment will be described with reference to Fig. 11 and Fig. 12. Fig. 11 is a flowchart showing an example of the processing flow of the inference stage executed by the information processing device 100 according to the second embodiment.

[0057] First, the acquisition unit 110 acquires time-series field images and growth-related data from the start time t1 to the inference time tn (step S300). Next, the derivation unit 120 inputs the acquired time-series field images and growth-related data into the trained model 152, and derives the expected yield and expected growth amount for each field (step S302). Next, the information provision unit 130 provides information related to the derived expected yield and expected growth amount to the terminal device 50 (step S304). This completes the processing of this flowchart.

[0058] 12 is a flowchart showing an example of the flow of processing in the learning stage executed by the information processing device 100 according to the second embodiment. First, the learning unit 140 acquires time-series field images and growth-related data, and actual measured values ​​of yield data and growth amount data (step S400). Next, the learning unit 140 calculates a loss function that represents the error between the predicted yield and predicted growth amount obtained by inputting the time-series field images and growth-related data into the machine learning model, and the yield data and growth amount data (actual measured values) (step S402). Next, the learning unit 140 learns parameters of the machine learning model so as to reduce the loss function (step S404). This completes the processing of this flowchart.

[0059] According to the second embodiment described above, the information processing device derives not only the predicted yield of the crop but also other growth amounts that can be predicted at the same time as the yield, thereby providing the user of the information processing device with more useful information regarding the growth of the crop.

[0060] The above describes the form for carrying out the present invention using an embodiment, but the present invention is not limited to such an embodiment, and various modifications and substitutions can be made within the scope that does not deviate from the gist of the present invention. [Explanation of symbols]

[0061] 10 satellites 20 Satellite image provider server 30 Public Information Server 50 Terminal Equipment 100 Information processing device 110 Acquisition Department 120 Derivation part 130 Information Provision Department 140 Learning Department 150 Storage section 152 trained models

Claims

1. an acquisition unit that acquires time-series images of farmland captured from above and time-series growth-related data on the growth of crops in the farmland; a derivation unit that derives one or more predicted yields of the crop for one or more prediction target plots of the farmland by inputting the time-series images and the time-series growth-related data into a trained model, The trained model is a machine learning model including a convolutional neural network and a recurrent neural network that has been trained in advance for one or more training target plots of farmland so as to reduce a loss function including an error between the one or more predicted yields at each time point in the time series and the value indicated by the acquired yield data. Information processing device.

2. The growth-related data includes at least one of meteorological information about the farmland, soil characteristics about the farmland, and growth conditions about the crops. The information processing device according to claim 1 .

3. the loss function is defined by the error, a correlation coefficient between the one or more predicted yields at each time point in the time series and a value indicated by the yield data, and a weight according to a period from a time point at which the predicted yield is derived to a time point at which the crop is harvested. The information processing device according to claim 1 .

4. The weight is set to a larger value as the derivation time approaches the harvest time of the crop. The information processing device according to claim 3 .

5. The machine learning model inputs the feature map obtained by inputting the time-series images and the time-series growth-related data into the convolutional neural network, and inputs the feature map into the recurrent neural network. The information processing device according to claim 1 .

6. the derivation unit inputs the time-series images and the time-series growth-related data into a trained model, thereby deriving, for one or more prediction target plots of the farmland, in addition to one or more predicted yields of the crop, a growth amount that can be predicted at the same time as the predicted yield; The trained model is a machine learning model including a convolutional neural network and a recurrent neural network that has been trained in advance for one or more training target plots of farmland so as to reduce a loss function including an error between the one or more predicted yields and predicted growth amounts at each time point in the time series and values ​​indicated by the acquired yield data and growth amount data. The information processing device according to claim 1 .

7. The computer Obtaining time-series images of farmland taken from above and time-series growth-related data on the growth of crops in the farmland; By inputting the time-series images and the time-series growth-related data into a trained model, one or more predicted yields of the crop are derived for one or more prediction target plots of the farmland; The trained model is a machine learning model including a convolutional neural network and a recurrent neural network that has been trained in advance for one or more training target plots of farmland so as to reduce a loss function including an error between the one or more predicted yields at each time point in the time series and the value indicated by the acquired yield data. Information processing methods.

8. On the computer, acquiring time-series images of the farmland taken from above and time-series growth-related data relating to the growth of crops in the farmland; By inputting the time-series images and the time-series growth-related data into a trained model, one or more predicted yields of the crops are derived for one or more prediction target plots of the farmland; The trained model is a machine learning model including a convolutional neural network and a recurrent neural network that has been trained in advance for one or more training target plots of farmland so as to reduce a loss function including an error between the one or more predicted yields at each time point in the time series and the value indicated by the acquired yield data. program.

9. an acquisition unit that acquires time-series images of farmland taken from above, time-series growth-related data on the growth of crops in the farmland, and crop yield data for one or more target plots of the farmland; a learning unit that generates a trained model that derives one or more predicted yields of the crop in the one or more target plots by inputting the time-series images and the time-series growth-related data; the learning unit generates the learned model by learning parameters of a machine learning model including a convolutional neural network and a recurrent neural network so as to reduce a loss function including an error between the one or more predicted yields at each time point in the time series and a value indicated by crop yield data in the one or more target plots; Information processing device.

10. The computer acquiring time-series images of farmland taken from above, time-series growth-related data relating to the growth of crops in the farmland, and crop yield data for one or more target plots of the farmland; generating a trained model that derives one or more predicted yields of the crop in the one or more target plots by inputting the time-series images and the time-series growth-related data; generating the trained model by training parameters of a machine learning model including a convolutional neural network and a recurrent neural network so as to reduce a loss function including an error between the one or more predicted yields at each time point in the time series and values ​​indicated by crop yield data for the one or more target plots; Information processing methods.

11. On the computer, acquiring time-series images of the farmland taken from above, time-series growth-related data relating to the growth of crops in the farmland, and crop yield data for one or more target plots of the farmland; generating a trained model that derives one or more predicted yields of the crop in the one or more target plots by inputting the time-series images and the time-series growth-related data; generating the trained model by learning parameters of a machine learning model including a convolutional neural network and a recurrent neural network so as to reduce a loss function including an error between the one or more predicted yields at each time point in the time series and values ​​indicated by crop yield data for the one or more target plots; program.

Images