Information processor, information processing system, information processing method, and computer program

Abstract

To improve estimation accuracy of a growth state of a plant.SOLUTION: An information processor comprises a model acquisition part which acquires a learned model, and an estimation processing part. The learned model is a model created by performing predetermined machine learning by using first information including image information regarding an image obtained by capturing a specific kind of a plant to be cultivated and imaging time information showing a period of time in a year when the image was captured as an explanatory variable, and second information showing the growth state of the plant as an objective variable. The estimation processing part estimates the growth state of the plant by using the first information regarding the plant to be estimated and the learned model.SELECTED DRAWING: Figure 5

Description

[Technical Field]

[0001] The technology disclosed in this specification relates to information processing for estimating the growth state of a plant under cultivation. [Background technology]

[0002] In recent years, various techniques have been proposed for improving the efficiency and optimizing the cultivation of plants (leafy vegetables, fruit trees, ornamental plants, etc.). For example, a technique is known in which, in fruit tree cultivation carried out by a competent farmer, the size of the fruit and the integrated value of the leaf area are calculated from photographed images of the fruit tree, a growth curve that approximates the growth state of the fruit is determined based on the integrated value of the fruit size and the leaf area, the growth state of the cultivated fruit is evaluated using the growth curve, and the harvest date of the fruit is predicted based on the evaluation results (see, for example, Patent Document 1). [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Application Publication No. 2019-187259 Summary of the Invention [Problem to be solved by the invention]

[0004] The above-mentioned conventional techniques have room for improvement in the accuracy of estimating the growth state.

[0005] This specification discloses a technique that can solve the above-mentioned problems. [Means for solving the problem]

[0006] The technology disclosed in this specification can be realized, for example, in the following forms.

[0007] (1) The information processing device disclosed in this specification includes a model acquisition unit that acquires a trained model and an estimation processing unit. The trained model is a model created by performing predetermined machine learning using first information, including image information about an image of a specific type of cultivated plant and image capture time information indicating the time of year when the image was taken, as explanatory variables, and second information indicating the growth state of the plant as a target variable. The estimation processing unit estimates the growth state of the plant using the first information about the plant to be estimated and the trained model. This information processing device can realize estimation that reflects differences in plant growth at different times of the year, thereby improving the accuracy of estimating the growth state of the plant.

[0008] (2) In the information processing device, the photographing time information may be information indicating the month in which the photograph was taken. With this configuration, it is possible to realize estimation that reflects differences in plant growth from month to month in a year, and it is possible to further improve the accuracy of estimating the growth state of plants.

[0009] (3) In the information processing device, the photographing time information may be information indicating the season in which the photograph was taken. This configuration makes it possible to realize estimation that reflects differences in plant growth between seasons in a year, thereby improving the accuracy of estimating the growth state of plants while suppressing an increase in the amount of information.

[0010] (4) In the information processing device, the second information may be information indicating at least one of the size of the plant, the growth level of the plant, and the harvest period of the plant as the growth state. According to this configuration, at least one of the size of the plant, the growth level of the plant, and the harvest period of the plant can be estimated with high accuracy using the trained model.

[0011] (5) The information processing device disclosed in this specification includes a model acquisition unit that acquires a trained model and an estimation processing unit. The trained model is a model created by performing predetermined machine learning using first information, including image information about an image of a specific type of cultivated plant, as an explanatory variable and second information indicating the harvest period of the plant as a target variable. The estimation processing unit estimates the harvest period of the plant using the first information about the plant to be estimated and the trained model. This information processing device can estimate the harvest period of the plant. This allows, for example, agricultural workers to plan harvesting work while the plants are being cultivated, thereby improving the efficiency and optimization of plant cultivation.

[0012] (6) In the information processing device, the harvest period may be defined by a harvest start date and a harvest end date. According to this configuration, the trained model can be used to estimate the plant harvest start date and the plant harvest end date. This allows, for example, a farm worker to predict the amount of labor to be applied each day during the harvest period while cultivating plants, thereby improving the efficiency and optimization of plant cultivation.

[0013] (7) In the information processing device, the first information may further include elapsed time information indicating the elapsed time from the start of cultivation of the plant to the photographing. This configuration can further improve the accuracy of estimating the growth state of the plant using the trained model.

[0014] (8) In the information processing device, the image information may be information indicating a frequency distribution of each component of a color space in the image. With this configuration, compared to using the image itself, it is possible to reduce the amount of information while maintaining estimation accuracy, and to improve the learning speed when creating the trained model.

[0015] (9) The information processing device may further include a training data acquisition unit that acquires training data in which multiple images generated by photographing the plant during cultivation at multiple dates and times are associated with information indicating the growth state of the plant, and the model acquisition unit creates the trained model by performing the machine learning using the training data. With this configuration, it is possible to acquire a trained model for estimating the growth state of a plant without using another device, and to estimate the growth state of a plant using the model.

[0016] The technology disclosed in this specification can be realized in various forms, such as an information processing device, an information processing system, an information processing method, a computer program that realizes these methods, a non-transitory recording medium on which the computer program is recorded, etc. [Brief explanation of the drawings]

[0017] [Figure 1] FIG. 1 is an explanatory diagram showing a schematic configuration of a plant cultivation management system 10 according to an embodiment of the present invention. [Figure 2] 1 is a block diagram showing a schematic configuration of a plant cultivation management server 100. [Figure 3] 1 is a flowchart showing the details of an estimation model creation process according to this embodiment. [Figure 4] A conceptual diagram of growth rate G [Figure 5] An explanatory diagram showing an example of training data TD [Figure 6] FIG. 10 is an explanatory diagram showing another example of training data TD. [Figure 7] Illustrative diagram showing the results of Example 1 [Figure 8] Illustrative diagram showing the results of Example 1 [Figure 9] Illustrative diagram showing the results of Example 1 [Figure 10] Illustrative diagram showing the results of Example 2 [Figure 11] Illustrative diagram showing the results of Example 2 DETAILED DESCRIPTION OF THE INVENTION

[0018] (Configuration of plant cultivation management system 10) FIG. 1 is an explanatory diagram showing a schematic configuration of a plant cultivation management system 10 according to this embodiment. The plant cultivation management system 10 is a system for managing the cultivation of plants in a farm field. More specifically, the plant cultivation management system 10 is a system that makes it possible to grasp the growth state of plants under cultivation by using photographed images of the plants under cultivation, thereby improving the efficiency and optimizing plant cultivation. The plant cultivation management system 10 is an example of an information processing system.

[0019] In this embodiment, the field is a semi-open (sunlight-utilizing) plant factory PF. Here, a semi-open plant factory PF is a field surrounded by materials that do not block sunlight, such as a greenhouse. However, the plant cultivation management system 10 can also be applied to other types of fields (e.g., other types of plant factories, such as closed or open types, fields for open-field cultivation, etc.). Although FIG. 1 shows only one plant factory PF, the plant cultivation management system 10 can also be applied to managing plant cultivation in each of multiple plant factories PF. The plants cultivated in the plant factory PF may be any type of plant, such as leafy vegetables (e.g., spinach), fruit trees (e.g., tomatoes), or ornamental plants.

[0020] The plant cultivation management system 10 includes a plant cultivation management server 100, a terminal device 200, and a photographing device 300. The devices constituting the plant cultivation management system 10 are connected to each other via a communication network NET so as to be able to communicate with each other.

[0021] The photographing device 300 is installed in the plant factory PF and generates images (e.g., still image data expressed in RGB components) by photographing plants cultivated in the plant factory PF. In this embodiment, the photographing device 300 is a webcam that takes photographs of a preset range in the plant factory PF periodically (e.g., every hour) or continuously. The photographing device 300 has a communication interface (not shown) and transmits image data generated by photographing each time photographing is performed to the plant cultivation managing server 100 via the communication network NET. A single plant factory PF may be installed with multiple photographing devices 300 that photograph different ranges as their subjects. When there are multiple plant factories PFs that are managed by the plant cultivation management system 10, a photographing device 300 is installed in each plant factory PF.

[0022] The terminal device 200 is, for example, a smartphone or a tablet terminal. The terminal device 200 includes a display unit 210 configured, for example, by a liquid crystal display or the like. An application program for plant cultivation management is installed in the terminal device 200. By using the application program, a user of the terminal device 200 (such as a farm worker P1) can grasp the growth status of plants being cultivated in the plant factory PF.

[0023] The plant cultivation managing server 100 is a server device for managing plant cultivation in the plant factory PF. FIG. 2 is a block diagram schematically showing the configuration of the plant cultivation managing server 100. The plant cultivation managing server 100 includes a control unit 110, a storage unit 120, a display unit 130, an operation input unit 140, and an interface unit 150. These units are connected to each other via a bus 190 so as to be able to communicate with each other. The plant cultivation managing server 100 is an example of an information processing device.

[0024] The display unit 130 of the plant cultivation management server 100 is configured, for example, by a liquid crystal display or the like, and displays various images and information. The operation input unit 140 is configured, for example, by a keyboard, mouse, buttons, microphone, etc., and receives operations and instructions from the administrator. The display unit 130 may be equipped with a touch panel so as to function as the operation input unit 140. The interface unit 150 is configured, for example, by a LAN interface, a USB interface, etc., and communicates with other devices via wired or wireless connection.

[0025] The storage unit 120 of the plant cultivation management server 100 is configured with, for example, a ROM, a RAM, a hard disk drive (HDD), etc., and is used to store various programs and data, and as a work area when executing various programs, and as a temporary storage area for data. For example, the storage unit 120 stores a plant cultivation management program CP, which is a computer program for executing various processes described below. The plant cultivation management program CP is provided in a state stored in a computer-readable recording medium (not shown), such as a CD-ROM, a DVD-ROM, or a USB memory, or is provided in a state that can be obtained from an external device (another server or terminal device on the network) via the interface unit 150, and is stored in the storage unit 120 in a state that is operable on the plant cultivation management server 100.

[0026] During the execution of various processes described below, training data TD, an estimation model MO, and target image data I(o) are stored in the storage unit 120 of the plant cultivation management server 100. The contents of this information will be explained together with the explanation of the various processes described below.

[0027] The control unit 110 of the plant cultivation management server 100 is configured with, for example, a CPU, and controls the operation of the plant cultivation management server 100 by executing a computer program read from the storage unit 120. For example, the control unit 110 reads and executes a plant cultivation management program CP from the storage unit 120, thereby functioning as a plant cultivation management unit 111 that executes various processes described below. The plant cultivation management unit 111 includes a training data acquisition unit 112, a model acquisition unit 114, a target data acquisition unit 115, an estimation processing unit 116, and an information transmission unit 118. The functions of each of these units will be described in conjunction with the explanation of various processes described below.

[0028] (Estimation model creation process) Next, an estimation model creation process executed by the plant cultivation management server 100 of this embodiment will be described. Fig. 3 is a flowchart showing the contents of the estimation model creation process in this embodiment. The estimation model creation process is a process of creating an estimation model MO used to estimate the growth state of a plant by performing predetermined machine learning. The estimation model MO is a trained model created by performing predetermined machine learning using first information including image information related to an image captured of a specific type of cultivated plant as an explanatory variable and second information indicating the growth state of the plant as a target variable.

[0029] Examples of the growth state of a plant estimated by the estimation model MO include the size of the plant (plant height of leafy vegetables, diameter of fruit, etc.), the growth rate of the plant, and the harvest period of the plant.

[0030] Examples of the growth degree of a plant include the growth degree relative to the harvest start date Te1 and the growth degree relative to the harvest end date Te2. FIG. 4 is an explanatory diagram conceptually showing the growth degree G. As shown in FIG. 4, the growth degree G1 relative to the harvest start date Te1 is calculated using the following formula: In the formula below, To is the photography date, and Ts is the cultivation start date (planting date). G1 = (To-Ts) / (Te1-Ts) (where To≦Te1) G1=1 (however, To≧Te1)

[0031] As shown in FIG. 4, the growth rate G2 for the harvest end date Te2 is calculated by the following formula. G2=(To-Ts) / (Te2-Ts)

[0032] The harvest period of a plant is defined by, for example, a harvest start date Te1 and a harvest end date Te2. If the harvest is completed in one day, the harvest start date Te1 and the harvest end date Te2 are the same day (harvest date Te). The estimation model MO may directly estimate the harvest period of a plant, or may indirectly estimate the harvest period using the growth degree for the harvest start date Te1 and the growth degree for the harvest end date Te2 described above.

[0033] In the estimation model creation process (FIG. 3), first, the training data acquisition unit 112 (FIG. 2) of the plant cultivation management server 100 acquires training data TD (S110). The training data TD is data in which the above-mentioned explanatory variables and objective variables are associated with each other. The acquired training data TD is stored in the storage unit 120 of the plant cultivation management server 100.

[0034] Fig. 5 is an explanatory diagram showing an example of training data TD. In the example of Fig. 5, the plants cultivated in the plant factory PF are spinach, and photographed images of the plants and information indicating the month in the year in which the photographs were taken (hereinafter referred to as "month variables") are used as explanatory variables, and the average size AS of the plants is used as the objective variable. The photographed images are an example of image information, the month variables are an example of photographing time information, and the photographed images and month variables are examples of first information. The average size AS of the plants is an example of second information indicating the growth state of the plants.

[0035] As described above, the photographing device 300 installed in the plant factory PF periodically photographs, and each time photographing is performed, the image data I generated by the photographing is transmitted to the plant cultivation management server 100 via the communication network NET. The training data acquisition unit 112 of the plant cultivation management server 100 acquires the image data I transmitted from the photographing device 300 via the interface unit 150 and stores it in the storage unit 120 as training image data. The training data acquisition unit 112 may perform trimming on the image data I transmitted from the photographing device 300.

[0036] The training data acquisition unit 112 uses a known image recognition process to identify image data Is captured on the cultivation start date Ts and image data Ie captured on the harvest date Te from among the multiple image data stored in the storage unit 120. In the example of FIG. 5, it is assumed that the harvest is completed in one day, and the harvest start date and harvest end date are the same day (harvest date Te). The training data acquisition unit 112 selects image data I captured on a date between the cultivation start date Ts and the harvest date Te from among the multiple image data I stored in the storage unit 120. However, this embodiment targets a semi-open plant factory PF that uses sunlight for cultivation, and therefore only image data I generated by capturing images during a predetermined daytime period (e.g., 7:00 AM to 5:00 PM) is used. Image data I captured outside of these time periods is excluded because the plants may not be clearly visible in the image data I. In this way, the training data acquisition unit 112 selects, as training image data I(t), multiple image data Ii (Is, I1, I2, . . . , Ie) captured and generated at multiple dates and times Ti (Ts, T1, T2, . . . , Te) within the period from the cultivation start date Ts to the harvest date Te. In addition, the training data acquisition unit 112 identifies the month variable for each image data I from the information on the capture date attached to each image data I.

[0037] The training data acquisition unit 112 acquires the value of the average plant size AS (average plant height) for each of the multiple image data I stored in the storage unit 120. The average plant size AS can be measured, for example, by referring to an image of a ruler that appears together with the plant in the image data I. The average plant size AS may also be acquired by referring to a document showing actual measurements taken by agricultural workers on-site at the plant factory PF and recorded together with the date and time.

[0038] The training data acquisition unit 112 creates training data TD as labeled data by associating the training image data I(t) and month variables as explanatory variables with the average plant size AS as a response variable.

[0039] The training data acquisition unit 112 may acquire training data TD for one crop of the target plant, or may acquire training data TD for multiple crops of the target plant to improve the accuracy of the estimation model MO. When multiple image capture devices 300 are installed in one plant factory PF, the training data acquisition unit 112 may acquire the training data TD based on image data acquired from the multiple image capture devices 300. The training data acquisition unit 112 may acquire the training data TD based on image data acquired from the image capture devices 300 installed in each of the multiple plant factories PF. Training image data may be generated by performing data augmentation (data expansion) on the image data acquired from the image capture devices 300, such as by rotating, shifting, or adjusting brightness.

[0040] FIG. 6 is an explanatory diagram showing another example of training data TD. In the example of training data TD shown in FIG. 6, information indicating the frequency distribution of each component of the color space in the captured image is used as the explanatory variable, rather than the captured image itself. For example, if data in which each of the three RGB components is expressed in 256 gradations (0 to 255) is used as the captured image, information indicating the frequency distribution of each component of the color space is generated by aggregating the pixel frequencies for each of 256 × 3 = 768 sections (r0, r1, . . . , b254, b255). For example, section "r0" indicates the number of pixels whose "r" component value is "0" among the multiple pixels constituting the captured image. The color components used to aggregate the frequency distribution are not limited to RGB, but may be other components such as HSV. Note that in the example of training data TD shown in FIG. 6, a month variable and the number of days elapsed from the start date of cultivation to the time of photographing (number of days elapsed in cultivation) are also used as explanatory variables. The number of days elapsed in cultivation is an example of elapsed time information.

[0041] The configuration of the training data TD is not limited to the examples of FIGS. 5 and 6 and may be any configuration. For example, in the examples of FIGS. 5 and 6, instead of the month variable, other photographing time information indicating the time of year when the photograph was taken (e.g., information indicating the season, such as spring, summer, autumn, or winter) may be used as the explanatory variable. In the example of FIG. 5, the number of days since cultivation may be additionally used as the explanatory variable, or information indicating the frequency distribution of each component of the color space of the photographed image may be used instead of the photographed image itself. In the examples of FIGS. 5 and 6, the month variable and / or the number of days since cultivation may be omitted. In the examples of FIGS. 5 and 6, instead of the average size, other information indicating the growth state of the plant (e.g., the growth rate of the plant, the harvest period of the plant, etc.) may be used as the objective variable.

[0042] Next, the model acquisition unit 114 (FIG. 2) of the plant cultivation management server 100 performs predetermined machine learning using the training data TD to create an estimation model MO (S120 in FIG. 3). The model acquisition unit 114 uses, for example, the root mean square error (RMSE), the mean absolute error (MAE), the coefficient of determination (R 2) to create an estimation model MO. The created estimation model MO is stored in the storage unit 120 of the plant cultivation management server 100. Note that various known machine learning algorithms such as XGBoost, LightGBM, and Random Forest can be used to create the estimation model MO. The above steps complete the estimation model creation process for creating the estimation model MO.

[0043] The plant cultivation management server 100 can support the farmer P1 in cultivating plants by estimating the growth state of the plants using the estimation model MO. For example, the target data acquisition unit 115 of the plant cultivation management server 100 acquires photographed images of the plants being cultivated by the farmer P1. The estimation processing unit 116 of the plant cultivation management server 100 estimates the growth state of the plants (such as the size of the plants, the growth rate of the plants, and the harvest period of the plants) using the acquired images and the estimation model MO. The information transmission unit 118 of the plant cultivation management server 100 transmits the estimation results to the terminal device 200 of the farmer P1 and displays them on the display unit 210 of the terminal device 200. This allows the farmer P1 to understand the current or future growth state of the plants being cultivated in the plant factory PF without having to visit the plant factory PF.

[0044] (Effects of this embodiment) As described above, the plant cultivation managing server 100 of this embodiment includes a model acquisition unit 114 and an estimation processing unit 116. The model acquisition unit 114 acquires an estimation model MO, which is a trained model. The estimation model MO is a model created by performing predetermined machine learning using, for example, first information, including image information on an image of a specific type of cultivated plant and image capture time information indicating the time of year when the image was taken, as explanatory variables, and second information indicating the growth state of the plant as a target variable. The estimation processing unit 116 estimates the growth state of the plant using the first information about the plant to be estimated and the estimation model MO. As described above, the estimation model MO used by the plant cultivation managing server 100 of this embodiment is created by machine learning using information including image capture time information indicating the time of year when the plant was photographed as an explanatory variable. Therefore, the plant cultivation managing server 100 of this embodiment can realize estimation that reflects differences in plant growth depending on the time of year, thereby improving the accuracy of estimating the growth state of the plant.

[0045] In this embodiment, the photographing time information may be information indicating the month in which the photograph of the plant to be estimated was taken, which makes it possible to realize estimation that reflects differences in plant growth from month to month in a year, thereby further improving the accuracy of estimating the growth state of the plant.

[0046] In this embodiment, the photographing time information may be information indicating the season in which the photograph of the plant to be estimated was taken. In this way, it is possible to realize estimation that reflects differences in plant growth between seasons in a year, and improve the accuracy of estimating the growth state of the plant while suppressing an increase in the amount of information.

[0047] In this embodiment, the objective variable when creating the estimation model MO may be information indicating at least one of the plant size, the plant growth rate, and the plant harvest period. In this way, the estimation model MO can be used to estimate at least one of the plant size, the plant growth rate, and the plant harvest period with high accuracy.

[0048] Furthermore, in this embodiment, the estimation model MO is a model created by performing predetermined machine learning using, for example, first information including image information about images of a specific type of cultivated plant as an explanatory variable and second information indicating the plant's harvest period as an objective variable. Thus, the estimation model MO used by the plant cultivation managing server 100 of this embodiment is created using the second information indicating the plant's harvest period as an objective variable. Therefore, the plant cultivation managing server 100 of this embodiment can estimate the plant's harvest period. This allows, for example, a farm worker to plan harvesting work while the plant is being cultivated, thereby improving the efficiency and optimization of plant cultivation.

[0049] In this embodiment, the harvest period may be defined by a harvest start date and a harvest end date. In this way, the estimation model MO can be used to estimate the plant harvest start date and the plant harvest end date. This allows, for example, a farm worker to predict the labor required for each day during the harvest period while cultivating plants, thereby improving the efficiency and optimization of plant cultivation.

[0050] In this embodiment, the first information as an explanatory variable when creating the estimation model MO may further include elapsed time information indicating the time elapsed from the start of plant cultivation to the time of photographing, which can further improve the accuracy of estimating the growth state of the plant using the estimation model MO.

[0051] In this embodiment, the image information included in the explanatory variables when creating the estimation model MO may be information indicating the frequency distribution of each component in the color space of the image. This allows the amount of information to be reduced while maintaining estimation accuracy, compared to when the image itself is used, and improves the learning speed when creating the estimation model MO.

[0052] In this embodiment, the plant cultivation managing server 100 includes a training data acquiring unit 112 that acquires training data in which a plurality of images generated by photographing a plant under cultivation at a plurality of dates and times are associated with information indicating the growth state of the plant, and a model acquiring unit 114 creates an estimation model MO by performing machine learning using the training data. The plant cultivation managing server 100 of this embodiment can acquire a trained model for estimating the growth state of a plant without using any other device, and can estimate the growth state of a plant using the model.

[0053] The plant cultivation management system 10 of this embodiment includes a plant cultivation management server 100 and a terminal device 200 that can communicate with the plant cultivation management server 100 via a communication network. The plant cultivation management server 100 includes an information transmission unit 118 that transmits information indicating the estimation result of the growth state of the plant by the estimation processing unit 116 to the terminal device 200 via the communication network. As described above, in the plant cultivation management system 10 of this embodiment, the information transmission unit 118 of the plant cultivation management server 100 transmits the estimation result of the growth state of the plant to the terminal device 200. Therefore, the farm worker P1 who holds the terminal device 200 can grasp the growth state of the plants being cultivated in the plant factory PF without going to the plant factory PF, thereby more effectively realizing efficiency and optimization of plant cultivation.

[0054] (Example) Examples conducted to verify the accuracy of estimation of the growth state of a plant by the plant cultivation management server 100 of the present embodiment described above will be described below. Figures 7 to 9 are explanatory diagrams showing the results of Example 1. Details of Example 1 are as follows. Cultivated plants: spinach Data acquisition period: April 1, 2019 ~ June 30, 2020 January 1, 2021 - December 31, 2021 Data acquisition environment: A semi-open, sunlight-utilizing plant factory Data breakdown: Training data: 4,302 images (including 861 validation data) Test data: 719 images Image information: RGB or HSV frequency distribution Explanatory variables: (Case 1) RGB or HSV frequency distribution only (Case 2) Frequency distribution + number of days since cultivation (Case 3) Frequency distribution + number of days since cultivation + month variable · Objective variable: Average size (plant height) Machine learning methods: XGBoost (Figure 7) LightGBM (Figure 8) Random Forest (Figure 9) Evaluation index: RMSE, MAE, R 2 Hyperparameter optimization method: Optuna

[0055] As shown in Figures 7 to 9, regardless of which machine learning method is used, Case 3, in which the explanatory variables include a month variable, tends to have higher estimation accuracy than Cases 1 and 2, in which the explanatory variables do not include a month variable. Therefore, it can be said that using a month variable as an explanatory variable can improve the estimation accuracy of plant growth conditions.

[0056] 10 and 11 are explanatory diagrams showing the results of Example 2. Details of Example 2 are as follows. Plants cultivated: Tomato Data acquisition period: April 10, 2019 ~ June 10, 2019 July 22, 2019 ~ September 26, 2019 June 25, 2020 ~ September 4, 2020 Data acquisition environment: A semi-open, sunlight-utilizing plant factory Data breakdown: Training data: 10,246 images (including validation data: 2,049 images) Test data: 2866 images Image information: The image itself (RGB data) Explanatory variables and estimation methods: (Case 1) Image only (Case 2) Image only (ensemble) (Case 3) Image + number of days since cultivation (Case 4) Image + number of days since cultivation (ensemble) Target variables: Growth rate relative to the end of harvest and growth rate relative to the start of harvest Machine learning methods: Xception VGG16 MobileNet EfficientNetB0 For ensembles, the single models with the highest accuracy are stacked. Evaluation index: RMSE, MAE, R 2 Hyperparameter optimization method: Optuna

[0057] Figures 10 and 11 show the index values ​​of the model that achieved the highest accuracy for each of the above cases. As shown in Figures 10 and 11, in all of the above cases, the growth degree for the harvest end date and the growth degree for the harvest start date were accurately estimated. Note that ensemble models tend to have higher estimation accuracy than single models. Also, using images + number of days since cultivation as explanatory variables tends to have higher estimation accuracy than using images alone.

[0058] (Variation) The technology disclosed in this specification is not limited to the above-described embodiments, and can be modified into various forms without departing from the spirit thereof, for example, the following modifications are also possible.

[0059] The configuration of the plant cultivation management system 10 in the above embodiment is merely an example and can be modified in various ways. For example, in the above embodiment, the plant cultivation management system 10 includes the photographing device 300, but the plant cultivation management system 10 does not have to include the photographing device 300. Even if the plant cultivation management system 10 does not include the photographing device 300, for example, a farm worker can use a photographing device (e.g., a camera mounted on the terminal device 200) to photograph a plant cultivated in the plant factory PF to generate image data, and transmit the image data to the plant cultivation management server 100, thereby allowing the plant cultivation management server 100 to acquire image data of the plant.

[0060] In the above embodiment, the photographing device 300 transmits the image data generated by photographing to the plant cultivation management server 100 each time a photograph is taken, but the image data may also be transmitted to the plant cultivation management server 100 at other times (for example, when a request is received from the plant cultivation management server 100).

[0061] In the above embodiment, the plant cultivation management server 100 executes the estimation model creation process for creating the estimation model MO, but the estimation model creation process may be executed by a device other than the plant cultivation management server 100, and the model acquisition unit 114 of the plant cultivation management server 100 may acquire the estimation model MO from the other device. Alternatively, the plant cultivation management server 100 may transmit data to the other device, and the plant cultivation management server 100 may acquire a result of estimation made in the other device using the estimation model MO.

[0062] In each of the above embodiments, a part of the configuration realized by hardware may be replaced by software, and conversely, a part of the configuration realized by software may be replaced by hardware. [Explanation of symbols]

[0063] 10: Plant cultivation management system 100: Plant cultivation management server 110: Control unit 111: Plant cultivation management unit 112: Training data acquisition unit 114: Model acquisition unit 115: Target data acquisition unit 116: Estimation processing unit 118: Information transmission unit 120: Memory unit 130: Display unit 140: Operation input unit 150: Interface unit 190: Bus 200: Terminal device 210: Display unit 300: Imaging device

Claims

1. An information processing device, a model acquisition unit that acquires a trained model, the trained model being a model created by performing predetermined machine learning using first information, including image information on an image of a specific type of cultivated plant and image capture time information indicating the time of year when the image was captured, as explanatory variables, and second information indicating the growth state of the plant as a target variable; and an estimation processing unit that estimates a growth state of the plant using the first information about the plant that is an estimation target and the trained model; An information processing device comprising:

2. 2. The information processing device according to claim 1, An information processing device, wherein the photographing time information is information indicating the month in which the photographing was performed.

3. 2. The information processing device according to claim 1, An information processing device, wherein the photographing time information is information indicating the season in which the photographing was performed.

4. 2. The information processing device according to claim 1, The information processing device, wherein the second information is information indicating at least one of the size of the plant, the growth degree of the plant, and the harvest period of the plant as the growth state.

5. An information processing device, a model acquisition unit that acquires a trained model, the trained model being a model created by performing predetermined machine learning using first information including image information relating to an image of a specific type of cultivated plant as an explanatory variable and second information indicating a harvest period of the plant as a target variable; and an estimation processing unit that estimates a harvest period of the plant by using the first information about the plant that is an estimation target and the trained model; An information processing device comprising:

6. 6. The information processing device according to claim 5, An information processing device, wherein the harvest period is defined by a harvest start date and a harvest end date.

7. 7. The information processing device according to claim 1, The information processing device, wherein the first information further includes elapsed time information indicating the elapsed time from the start of cultivation of the plant to the photographing.

8. 7. The information processing device according to claim 1, An information processing apparatus, wherein the image information is information indicating a frequency distribution of each component of a color space in the image.

9. The information processing device according to any one of claims 1 to 6, further comprising: a training data acquisition unit that acquires training data in which a plurality of images generated by photographing the plant during cultivation at a plurality of dates and times and information indicating a growth state of the plant are associated with each other; The model acquisition unit creates the trained model by performing the machine learning using the training data.

10. An information processing device according to any one of claims 1 to 6; a terminal device capable of communicating with the information processing device via a communication network; An information processing system comprising: the information processing device further includes an information transmitting unit that transmits information indicating an estimation result by the estimation processing unit to the terminal device via the communication network. Information processing system.

11. An information processing method, comprising: a step of acquiring a trained model, the trained model being a model created by performing predetermined machine learning using first information, including image information on an image of a specific type of cultivated plant and information on the time of photographing the plant, as an explanatory variable, and second information indicating the growth state of the plant as a target variable; estimating a growth state of the plant using the first information about the plant to be estimated and the trained model; An information processing method comprising:

12. An information processing method, comprising: a step of acquiring a trained model, the trained model being a model created by performing predetermined machine learning using first information including image information relating to an image of a specific type of cultivated plant as an explanatory variable and second information indicating a harvest period of the plant as a target variable; estimating a harvest period of the plant using the first information about the plant to be estimated and the trained model; An information processing method comprising:

13. A computer program comprising: On the computer, a process for acquiring a trained model, the trained model being a model created by performing predetermined machine learning using first information, including image information on an image of a specific type of cultivated plant and image capture time information indicating the time of year when the image was captured, as explanatory variables, and second information indicating the growth state of the plant as a target variable; a process of estimating a growth state of the plant using the first information about the plant to be estimated and the trained model; Execute Computer program.

14. A computer program comprising: On the computer, a process for acquiring a trained model, the trained model being a model created by performing predetermined machine learning using first information including image information relating to an image of a specific type of cultivated plant as an explanatory variable and second information indicating a harvest period of the plant as a target variable; a process of estimating a harvest period of the plant using the first information about the plant to be estimated and the trained model; Execute Computer program.

Images