Estimation device, display device, image generation device, estimation method, display method, image generation method, and recording medium

Abstract

This estimation device comprises: an image analysis means for specifying, from an image obtained by imaging at least a part of a field in which a crop having a harvest object is planted, at least one of the number or amount of harvest objects appearing in the image and a leaf region of the crop in the image, on the basis of the color in the image; and a quantity estimation means for estimating at least one of the number or amount of harvest objects in the field on the basis of at the least one of the number or amount of the harvest objects appearing in the image and the leaf region of the crop in the image.

Description

Estimation device, display device, image generation device, estimation method, display method, image generation method, and recording medium

[0001] The present disclosure relates to an estimation device, a display device, an image generation device, an estimation method, a display method, an image generation method, and a recording medium.

[0002] Techniques for predicting crop yields have been proposed. For example, Patent Literature 1 describes predicting the sugar content of sugar beets using remote sensing data.

[0003] Japanese Patent Publication No. 2010-166851

[0004] When estimating at least the number or amount of harvest targets using an image of a crop having harvest targets, it is preferable to be able to estimate at least the number or amount of harvest targets with high accuracy.

[0005] An example of an object of the present disclosure is to provide an estimation device, a display device, an image generation device, an estimation method, a display method, an image generation method, and a recording medium that can solve the above-mentioned problems.

[0006] According to a first aspect of the present disclosure, an estimation device includes an image analysis means that identifies, from an image taken of at least a portion of a field in which a crop having harvest objects is planted, at least one of the number or quantity of harvest objects appearing in the image and the area of ​​the crop leaves in the image based on the color in the image, and a quantity estimation means that estimates at least one of the number or quantity of harvest objects in the field based on at least one of the number or quantity of harvest objects appearing in the image and the area of ​​the crop leaves in the image.

[0007] According to a second aspect of the present disclosure, a display device includes a display means for displaying an image of a farm field and a guide for adjusting the width of ridges shown in the image of the farm field.

[0008] According to a third aspect of the present disclosure, an image generating device includes a camera and a display means for displaying a real-time image captured by the camera and a guide for adjusting the width of the ridges in the field shown in the image.

[0009] According to a fourth aspect of the present disclosure, the estimation method includes a computer identifying, from an image taken of at least a portion of a field in which a crop having harvest objects is planted, at least one of the number or quantity of harvest objects shown in the image and the area of ​​the crop's leaves in the image based on the colors in the image, and estimating at least one of the number or quantity of the harvest objects in the field based on at least one of the number or quantity of harvest objects shown in the image and the area of ​​the crop's leaves in the image.

[0010] According to a fifth aspect of the present disclosure, a display method includes displaying, by a display device, an image of a farm field and a guide for adjusting the width of ridges shown in the image of the farm field.

[0011] According to a sixth aspect of the present disclosure, an image generation method includes an image generation device having a camera displaying a real-time image being captured by the camera and a guide for adjusting the width of ridges in a field shown in the image.

[0012] According to a seventh aspect of the present disclosure, the recording medium is a recording medium having recorded thereon a program that causes a computer to perform the following operations: identify, from an image of at least a portion of a field in which a crop having harvest objects is planted, at least one of the number or quantity of harvest objects appearing in the image and the area of ​​the crop's leaves in the image, based on the color in the image; and estimate at least one of the number or quantity of harvest objects in the field, based on at least one of the number or quantity of harvest objects appearing in the image and the area of ​​the crop's leaves in the image.

[0013] According to an eighth aspect of the present disclosure, the recording medium is a recording medium having recorded thereon a program that causes a computer controlling a display device to display an image of a field and a guide for adjusting the width of the ridges shown in the image of the field.

[0014] According to a ninth aspect of the present disclosure, the recording medium is a recording medium having recorded thereon a program that causes a computer that controls an image generating device having a camera to display a real-time image being captured by the camera and a guide for adjusting the width of the ridges in the field that are shown in the image.

[0015] According to the present disclosure, when estimating at least the number or quantity of harvest objects using an image of a crop having harvest objects, it is expected that at least the number or quantity of harvest objects can be estimated with relatively high accuracy.

[0016] 1 is a diagram illustrating an example of the configuration of an estimation system according to at least one embodiment; FIG. 2 is a diagram illustrating an example of the configuration of an estimation device according to at least one embodiment; FIG. 3 is a diagram illustrating an example of the configuration of a terminal device according to at least one embodiment; FIG. 4 is a diagram illustrating a first example of a data input screen displayed by a second display unit according to at least one embodiment; FIG. 5 is a diagram illustrating a second example of a data input screen displayed by a second display unit according to at least one embodiment; FIG. 6 is a diagram illustrating a first example of a monitor screen displayed by a second display unit according to at least one embodiment when a camera takes an image for registration; FIG. 7 is a diagram illustrating a second example of a monitor screen displayed by a second display unit according to at least one embodiment when a camera takes an image for registration; FIG. 8 is a diagram illustrating an example of an image registration screen displayed by a second display unit according to at least one embodiment; FIG. 1 is a diagram showing a second example of processing procedures when an estimation system according to at least one embodiment predicts a yield of fruit in a farm field. FIG. 2 is a diagram showing an example of the configuration of a learning device according to at least one embodiment. FIG. 3 is a diagram showing an example of the configuration of an estimation device according to at least one embodiment. FIG. 4 is a diagram showing an example of the configuration of a display device according to at least one embodiment. FIG. 5 is a diagram showing an example of the configuration of an image generation device according to at least one embodiment. FIG. 6 is a diagram showing an example of processing procedures in an estimation method according to at least one embodiment. FIG. 7 is a diagram showing an example of processing procedures in a display method according to at least one embodiment. FIG. 8 is a diagram showing an example of the processing procedures in an image generation method according to at least one embodiment. FIG. 9 is a diagram showing an example of the configuration of a computer according to at least one embodiment.

[0017] The following describes embodiments of the present disclosure, but the following embodiments do not limit the scope of the invention as claimed. Furthermore, not all of the combinations of features described in the embodiments are necessarily essential to the solution of the invention.

[0018] <First embodiment> Fig. 1 is a diagram showing an example of the configuration of an estimation system according to at least one embodiment. In the configuration shown in Fig. 1, the estimation system 1 includes an estimation device 100 and a terminal device 200. The estimation system 1 is a system that estimates at least the number or amount of harvest targets in a field.

[0019] The following describes an example in which the estimation system 1 predicts the total weight of processing tomatoes that can be harvested in a single field. Hereinafter, processing tomatoes will also be referred to simply as "fruit." The total weight of harvestable fruits will also be referred to as the fruit yield, the tomato yield, or simply the harvest yield. However, the objects for which the estimation system 1 estimates quantity are not limited to specific ones, and can include various items that can be harvested from crops (plants grown in a field) and that can be photographed while the crop is planted in the field. The estimation system 1 may also estimate the number of harvest objects in addition to or instead of the weight of the harvest objects.

[0020] The terminal device 200 photographs tomatoes planted in a field and transmits the obtained image data to the estimation device 100. An image photographed by the terminal device 200 of tomatoes planted in a field corresponds to an example of an image photographed of at least a portion of a field in which tomatoes are planted. The terminal device 200 may be configured using a general-purpose device, such as a mobile phone with a camera, such as a smartphone. Alternatively, the terminal device 200 may be configured as a device dedicated to the estimation system 1. The act of the terminal device 200 photographing (the camera of the terminal device 200 photographing) is also referred to as the user of the terminal device 200 photographing.

[0021] The estimation device 100 predicts the total weight of fruit that can be harvested from a field using images captured by the terminal device 200. The estimation device 100 may be configured using a computer such as a personal computer (PC). The estimation device 100 and the terminal device 200 may also be integrated into one device. For example, the functions of the estimation device 100 and the terminal device 200 may be implemented in a portable computer such as a laptop or tablet.

[0022] Fig. 2 is a diagram showing an example of the configuration of the estimation device 100. In the configuration shown in Fig. 2, the estimation device 100 includes a first communication unit 110, a first display unit 120, a first operation input unit 130, a first storage unit 180, and a first control unit 190. The first control unit 190 includes a photography location designation unit 191, a data acquisition unit 192, an image analysis unit 193, and a quantity estimation unit 194.

[0023] The first communication unit 110 communicates with other devices. For example, the first communication unit 110 receives image data transmitted by the terminal device 200. The first display unit 120 has a display screen such as a liquid crystal panel, an organic electroluminescence (EL) panel, or an LED (light emitting diode) panel, and acquires various images. For example, the first display unit 120 may display various data used to predict the yield, such as images captured by the terminal device 200. The first display unit 120 may also display the results of the yield prediction by the estimation device 100.

[0024] The first operation input unit 130 includes input devices such as a keyboard and a mouse, and accepts user operations. For example, the first operation input unit 130 may accept a user operation to instruct the user to start predicting the yield. The display device and / or input device included in the terminal device 200 may be used as a user interface for the estimation device 100. In this case, the estimation device 100 may be configured without the first display unit 120 and / or the first operation input unit 130.

[0025] The first memory unit 180 stores various types of data. For example, the first memory unit 180 stores various types of data used to predict harvest yields, such as images captured by the terminal device 200. The first memory unit 180 is configured using a storage device provided in the estimation device 100. The first control unit 190 controls each unit of the estimation device 100 to perform various processes. The functions of the first control unit 190 are executed, for example, by a CPU (Central Processing Unit) provided in the estimation device 100 reading and executing a program from the first memory unit 180.

[0026] The photographing location instruction unit 191 determines a location where the terminal device 200 should photograph tomatoes based on the growth status data. The growth status data here is data that indicates the growth status of tomatoes in each part of the field. The parts of the field here are also referred to as cells. The growth status data indicates the growth status of tomatoes in each cell. The photographing location instruction unit 191 determines a cell where the terminal device 200 should photograph tomatoes based on the growth status data. The photographing location instruction unit 191 notifies the terminal device 200 of the determined cell where tomatoes should be photographed via the first communication unit 110. The cell where tomatoes should be photographed is also referred to as a photographing target cell. Cells other than the photographing target cell are also referred to as non-photographing target cells.

[0027] The growth status data used by the estimation system 1 is not limited to a specific one, and may be various data indicating the growth status of tomatoes for each cell in the field for which the yield is to be predicted. For example, the estimation system 1 may use, but is not limited to, NDVI (Normalized Difference Vegetation Index) or NDRE (Normalized Difference Red Edge Index) as the growth status data.

[0028] The photographing location designation unit 191 may determine a cell that shows an average growth condition relative to the growth condition of tomatoes in the field (the growth condition of tomatoes in the entire field) as the cell to be photographed. In this case, the estimation device 100 can use the predicted yield calculated for that cell as the average predicted yield per cell in the field. Therefore, the estimation device 100 can calculate the predicted yield for the field by multiplying the predicted yield for that cell by the number of cells (the number of cells).

[0029] The photographing location designation unit 191 may determine, as photographing target cells, a plurality of cells that show an average growth condition of tomatoes in the field. In this case, the average value of the predicted yield values ​​for the plurality of cells can be used as the average value of the predicted yield value per cell in the field. Therefore, the estimation device 100 can calculate the predicted yield value for the field (the predicted yield value for the entire field) by multiplying the average value of the predicted yield values ​​for the plurality of cells by the number of cells.

[0030] Alternatively, the photography location designation unit 191 may determine, as the photography target cells, a plurality of cells having various values ​​of growth status data. In this case, the estimation device 100 is expected to be able to grasp with relatively high accuracy the relationship between the growth status data value and the predicted yield value by using the value of the growth status data for each photography target cell and the predicted yield value calculated for that cell. As a result, the estimation device 100 is expected to be able to estimate with relatively high accuracy the predicted yield value for cells that are not photography targets.

[0031] The estimation device 100 may interpolate or extrapolate the predicted yield value for the cells not being photographed based on the value of the growth status data for each cell being photographed, the predicted yield value calculated for that cell, and the value of the growth status data for the cells not being photographed.

[0032] If the estimation device 100 determines that there is a target cell that has a growth status data value similar to the growth status data value of a non-target cell, the estimated yield value of the non-target cell may be estimated to be the same as the estimated yield value of the target cell. Here, the similarity of the growth status data value of the non-target cell and the growth status data value of the target cell may mean that the difference between the growth status data value of the non-target cell and the growth status data value of the target cell is equal to or less than a predetermined value.

[0033] The data acquisition unit 192 acquires various data used by the estimation device 100 to predict the harvest yield in the field. For example, the data acquisition unit 192 may extract growth status data for each cell in the field from the received data received by the first communication unit 110 from a server device that provides the growth status data, and store the data in the first storage unit 180. Furthermore, the data acquisition unit 192 may extract image data of an image of a tomato and data indicating the number of tomato plants shown in the image from the received data received by the first communication unit 110 from the terminal device 200, and store the data in the first storage unit 180.

[0034] The image analysis unit 193 analyzes images captured by the terminal device 200. The image that is the target of analysis by the image analysis unit 193 is also referred to as a target image. In particular, the image analysis unit 193 identifies the fruit area and the tomato leaf area from the target image based on the colors in the image. The image analysis unit 193 is an example of an image analysis means.

[0035] To identify the fruit region, the image analysis unit 193 performs image recognition processing on the target image to detect the red fruit region and counts the number of detected red fruits or the number of pixels contained in the region. When directly detecting and counting the number of individual fruits, the image analysis unit 193 may use an object detection method such as You Only Look Once (YOLO) or Faster R-CNN. However, the method by which the image analysis unit 193 counts the number of fruits is not limited to a specific method.

[0036] As described above, the image analysis unit 193 may count the number of pixels in an area detected as an image of red fruit. Specifically, the image analysis unit 193 detects pixels in the target image that fall within a predetermined color range for red fruit colors and counts the number of detected pixels. Pixels that fall within a predetermined color range for red fruit colors are also referred to as red pixels. Alternatively, the image analysis unit 193 may directly identify pixels corresponding to red fruit using a semantic segmentation method such as a Fully Convolutional Network (FCN) or Masked Faster R-CNN. However, the method by which the image analysis unit 193 identifies pixels corresponding to red fruit is not limited to a specific method.

[0037] In the same manner as in the case of fruit, the image analysis unit 193 performs image recognition processing on the target image to detect tomato leaf regions and counts the number of detected leaves or the number of pixels included in the region. Pixels included in a color range predetermined as the color of leaves are also referred to as green pixels.

[0038] The quantity estimation unit 194 predicts the yield of fruit in the field based on the number of fruits or the number of pixels, or both, in the target image and the number of leaves or the number of pixels, or both, in the target image, identified by the image analysis unit 193. The quantity estimation unit 194 is an example of a quantity estimation means.

[0039] The quantity estimation unit 194 estimates the number of fruits in the area of ​​the farm field that is captured in the target image, and predicts the fruit harvest yield in the farm field based on the estimation result. When the image analysis unit 193 counts the number of red fruits, the number of red pixels, and the number of green pixels, the quantity estimation unit 194 estimates the number of fruits in the area that is captured in the target image based on equation (1).

[0040]

[0041] * indicates scalar multiplication. ｆｒｕｉｔ＿ａｌｌ N denotes an estimate of the number of fruits in the area of ​​the target image. ｆｒｕｉｔ＿ａｌｌ N includes the number of fruits hidden by leaves and the number of green fruits. ｆｒｕｉｔ＿ｄｅｔｅｃｔｅｄ＿ｒｅｄ indicates the number of red fruits in the target image.

[0042] P ｒｅｄ indicates the number of red pixels in the target image. ｇｒｅｅｎ indicates the number of green pixels in the target image. ｒｅｄ and P ｇｒｅｅｎ However, both may indicate the proportion of pixels. ｒｅｄ P may be a value obtained by dividing the number of red pixels in the target image by the total number of pixels in the target image. ｇｒｅｅｎ may be a value obtained by dividing the number of green pixels among the pixels of the target image by the total number of pixels of the target image.

[0043] a and b are the linear coefficient and constant term in the linear equation. The values ​​of a and b may be determined by learning (machine learning) or by a statistical method. Alternatively, the image analysis unit 193 or the quantity estimation unit 194 may calculate the value of the coefficient a using an equation that uses the planting density. The planting density here refers to the number of tomato plants per unit area in the field or a portion thereof. For example, the image analysis unit 193 or the quantity estimation unit 194 may calculate the value of the coefficient a so that the value of the coefficient a increases as the planting density increases (is higher).

[0044] For example, consider the following assumptions: (Assumption 1) The pixels in the target image's area containing green fruit also fall under the category of green pixels. (Assumption 2) The ratio of the number of tomatoes in the red pixel area to the number of red pixels is proportional to the ratio of the number of tomatoes in the green pixel area to the number of green pixels.

[0045] Assumption 1 corresponds to the fact that the color of green fruit and the color of leaves are similar. Assumption 2 corresponds to the fact that fruit hidden by leaves and green fruit appearing in the target image are present at a density similar to the density of red fruit appearing in the target image. Fruit density here refers to the number of fruits per unit area in the target image.

[0046] Under the above assumptions 1 and 2, the sum of the number of fruits hidden by leaves and the number of green fruits in the target image is calculated as "(N ｆｒｕｉｔ＿ｄｅｔｅｃｔｅｄ＿ｒｅｄ / P ｒｅｄ ) *P ｇｒｅｅｎ " Therefore, N ｆｒｕｉｔ＿ａｌｌ = (N ｆｒｕｉｔ＿ｄｅｔｅｃｔｅｄ＿ｒｅｄ / P ｒｅｄ ) *P ｇｒｅｅｎ +N ｆｒｕｉｔ＿ｄｅｔｅｃｔｅｄ＿ｒｅｄ In this case, a=1 and b=0 can be set in equation (1).

[0047] Equation (1) corresponds to an expansion of Assumption 2 above, which assumes that there is a linear relationship between the number of tomatoes in a red pixel region divided by the number of red pixels and the number of tomatoes in a green pixel region divided by the number of green pixels. Furthermore, this expansion may also be extended to include cases where the relationship between the number of tomatoes in a red pixel region, the number of red pixels, and the number of tomatoes in a green pixel region, and the number of green pixels, is nonlinear. In this case, the quantity estimation unit 194 estimates the number of fruits in the region of the target image based on Equation (2).

[0048]

[0049] f is the number of red fruits in the target image, N ｆｒｕｉｔ＿ｄｅｔｅｃｔｅｄ＿ｒｅｄ and the number of red pixels in the target image P ｒｅｄ and the number of green pixels in the target image P ｇｒｅｅｎ The input is used to calculate the estimated number of fruits in the target image, N ｆｒｕｉｔ＿ａｌｌ The function f may be determined by learning or by a statistical method.

[0050] The terminal device 200 takes images of tomatoes at one or more locations in the farm field, and the quantity estimation unit 194 calculates an estimated value N of the number of fruits in the area shown in the target image for each shooting location (i.e., for each target image). ｆｒｕｉｔ＿ａｌｌ Then, the quantity estimation unit 194 calculates the estimated value N of the number of fruits in the area captured in the target image, calculated for each shooting location. ｆｒｕｉｔ＿ａｌｌ For example, the quantity estimation unit 194 predicts the total weight of fruit in the field at harvest time based on equation (3).

[0051]

[0052] W ｆｒｕｉｔ＿ａｌｌ（ｆｉｅｌｄ） indicates the predicted total weight of fruit at harvest in the field. Here, green fruits are harvested after turning red, and W ｆｒｕｉｔ＿ａｌｌ（ｆｉｅｌｄ） contains the predicted weight of the red fruit when the green fruit turns red at the time of the photograph.ｓａｍｐｌｉｎｇ indicates the number of shooting locations of the target image. Here, it is assumed that the terminal device 200 shoots one target image per shooting location. Therefore, the number of target images is also N ｓａｍｐｌｉｎｇ It shall be shown as follows.

[0053] i is an identification number that identifies the location where the target image was taken, and 1≦i≦N ｓａｍｐｌｉｎｇ The shooting location identified by the identification number i is also referred to as the i-th shooting location. The target image identified by the identification number i is also referred to as the i-th target image. N ｆｒｕｉｔ＿ａｌｌ （ｉ） is the estimated number of fruits in the area of ​​the i-th target image, N ｆｒｕｉｔ＿ａｌｌ Indicates N ｐｌａｎｔｓ （ｉ） indicates the number of tomato plants in the i-th target image. ｐｌａｎｔｓ （ｉ） may be input by the user of the terminal device 200.

[0054] W ｓｔａｎｄａｒｄ indicates the standard weight of a single red fruit. ｓｔａｎｄａｒｄ For example, the actual weight of red fruit may be used as W ｓｔａｎｄａｒｄ Alternatively, the average weight of fruit sampled from fruits harvested in the past may be used as W ｓｔａｎｄａｒｄ The average weight of red fruits obtained in a shaking test may be used as the weight of the fruit.

[0055] D indicates the planting density. ｆｉｅｌｄ indicates the area of ​​the field. ｆｉｅｌｄ may be set in advance in the estimation device 100. In addition, the quantity estimation unit 194 may use the area S of the field as the predicted value of the harvest yield. ｆｉｅｌｄ Alternatively, a predicted value per unit area may be calculated without multiplying the area by the above.

[0056] Equation (3) shows an example in which a cell showing an average growth condition relative to the growth conditions of tomatoes in a field is set as the target cell for photography. For example, as described above, the photography location instruction unit 191 determines a cell showing an average growth condition as the target cell for photography based on the growth condition data, and instructs the terminal device 200 to photograph the target cell for photography. The terminal device 200 photographs the designated target cell for photography. Specifically, the user of the terminal device 200 may refer to the instruction from the photography location instruction unit 191 and use the terminal device 200 to photograph tomatoes in the designated target cell for photography.

[0057] "N" on the right side of equation (3) ｆｒｕｉｔ＿ａｌｌ （ｉ） / N ｐｌａｎｔｓ （ｉ） " indicates the estimated number of fruits per tomato plant at the i-th photographing location. "(1 / N ｓａｍｐｌｉｎｇ )Σ ｉ＝１ Ｎｓａｍｐｌｉｎｇ (N ｆｒｕｉｔ＿ａｌｌ （ｉ） / N ｐｌａｎｔｓ （ｉ） ) indicates the average estimated number of fruits per tomato plant across all photographed locations.

[0058] "(1 / N ｓａｍｐｌｉｎｇ )Σ ｉ＝１ Ｎｓａｍｐｌｉｎｇ (N ｆｒｕｉｔ＿ａｌｌ （ｉ） / N ｐｌａｎｔｓ （ｉ） ) *W ｓｔａｎｄａｒｄ " indicates the average predicted total weight of fruit per tomato plant at harvest time for all photographed locations. "(1 / N ｓａｍｐｌｉｎｇ )Σ ｉ＝１ Ｎｓａｍｐｌｉｎｇ (N ｆｒｕｉｔ＿ａｌｌ （ｉ） / N ｐｌａｎｔｓ （ｉ） ) *W ｓｔａｎｄａｒｄ *D" represents the predicted value of the total weight of fruit per unit area of ​​the field at the time of harvest. The right side of equation (3) represents the predicted value of the total weight of fruit in the field at the time of harvest.

[0059] The quantity estimation unit 194 estimates the total weight of the fruit in the field at the time of harvesting W ｆｒｕｉｔ＿ａｌｌ（ｆｉｅｌｄ）Alternatively, when some fruits in the field are expected not to be harvested, the quantity estimation unit 194 may calculate the predicted harvest value in the field based on formula (4).

[0060]

[0061] W ｙｉｅｌｄ indicates the predicted yield in the field. Y is a real number in the range of 0≦Y≦1, indicating the proportion of fruits in the field that are to be harvested. The proportion Y may be set in advance in the estimation device 100. For example, if harvesting is performed when approximately 95% of the fruits have turned red, green fruits are discarded, and the proportion of red fruits that cannot be shipped is negligible, Y can be set to 0.95.

[0062] Fig. 3 is a diagram showing an example of the configuration of the terminal device 200. In the configuration shown in Fig. 3, the terminal device 200 includes a second communication unit 210, a second display unit 220, a second operation input unit 230, a camera 240, a second storage unit 280, and a second control unit 290. The terminal device 200 corresponds to an example of a display device. The terminal device 200 also corresponds to an example of an image generation device.

[0063] The second communication unit 210 communicates with other devices. For example, the second communication unit 210 receives instructions on where to photograph tomatoes (e.g., designation of the target cell) from the estimation device 100. The second communication unit 210 also transmits to the estimation device 100 various data used by the estimation device 100 to predict the harvest yield, such as the photographed image of the tomatoes and the number of tomato plants shown in the image.

[0064] The second display unit 220 has a display screen such as a liquid crystal panel, an organic EL panel, or an LED panel, and acquires various images. For example, the second display unit 220 displays instructions on where to photograph tomatoes, a monitor screen for photographing tomatoes, and a data input screen for the user to input data such as the number of tomato plants. The second display unit 220 is an example of a display means.

[0065] The second operation input unit 230 is configured to include input devices such as a touch sensor that is provided on the display screen of the second display unit 220 and forms a touch panel, and operation keys, and accepts user operations. For example, the second operation input unit 230 accepts user operations to input various data such as the number of tomato plants. The second operation input unit 230 also accepts user operations such as a shutter operation to capture images using the camera 240 and user operations to trim and register image data to be transmitted to the estimation device 100.

[0066] The camera 240 takes an image in response to a user's shutter operation and outputs image data of the captured image. In particular, the camera 240 takes an image in response to a shutter operation while the camera 240's own imaging range (angle of view) is directed toward the tomatoes in the field, thereby generating image data of the target image used for harvest yield prediction by the estimation device 100.

[0067] The second storage unit 280 stores various types of data. For example, the second storage unit 280 stores various image formats, such as the format of a data input screen displayed by the second display unit 220. The second storage unit 280 also stores image data of images captured by the camera 240. The second storage unit 280 is configured using a storage device included in the terminal device 200.

[0068] The second control unit 290 controls each unit of the terminal device 200 to perform various processes. For example, the second control unit 290 controls the second communication unit 210 to transmit and receive various types of data. The second control unit 290 also controls the second display unit 220 to display various images. The second control unit 290 also detects user operations based on electrical signals output by the second operation input unit 230 in response to user operations. The second control unit 290 also controls the camera 240 to take images. The second control unit 290 also writes data to and reads data from the second storage unit 280. The functions of the second control unit 290 are performed, for example, by a CPU included in the terminal device 200 reading and executing a program from the second storage unit 280.

[0069] Fig. 4 is a diagram showing a first example of a data input screen displayed by the second display unit 220. In the example of Fig. 4, area A11 is an area for accepting input of the average weight (standard weight) per fruit at harvest time, and area A12 is an area for accepting input of the planting density.

[0070] Area A13 is an area that accepts registration of an image (target image) to be used for harvest yield prediction by the estimation device 100. An image that is registered as an image to be used for harvest yield prediction by the estimation device 100 is also referred to as a registered image. The user may select whether to take a registered image using the camera 240 or to register an image already stored in the second storage unit 280. Area A14 is a field that accepts input of the number of tomato plants shown in the image to be registered in area A13. Figure 4 shows an example in which the user directly inputs the number of plants. In the example of Figure 4, the user selects "directly input number of plants" and inputs the number of plants.

[0071] FIG. 5 is a diagram showing a second example of a data input screen displayed by the second display unit 220. The method for inputting the number of plants in FIG. 5 differs from that in FIG. 4. In the example in FIG. 5, the user inputs "Calculate based on the photographed area" and inputs the horizontal size of the plant shown in the image and the distance between the plants. The horizontal size of the plant shown in the image here refers to the length of the area shown in the photograph in the longitudinal direction of the ridge. The terminal device 200 divides the horizontal size of the plant shown in the image by the distance between the plants to calculate and display the number of plants shown in the image. In other respects, the example in FIG. 5 is similar to the example in FIG. 4.

[0072] 5, if the user knows the horizontal size and the distance between plants in the image, the user can provide the information necessary to calculate the number of plants without having to count and input the number of plants in the image. For example, even when counting the number of plants is not easy, such as when tomatoes are growing and the boundaries between the plants are not clear, the user can provide the information necessary to calculate the number of plants.

[0073] Fig. 6 is a diagram showing a first example of a monitor screen displayed by second display unit 220 when camera 240 captures an image for registration. Second display unit 220 displays the screen shown in Fig. 6 when the user selects to capture an image to be registered using camera 240 in the example of Fig. 4 or the example of Fig. 5 .

[0074] 6, area A21 is an area that displays in real time an image being captured by camera 240. Area A22 is an area that displays a message of precautions to take when capturing an image. Area A23 is an area that displays a portion of an image captured by camera 240 that will be used as a registration image.

[0075] Area A24 is an area that accepts a switching operation to display or hide a guide for trimming the registered image. Trimming the registered image here refers to adjusting the image by enlarging / reducing and aligning it so that an appropriate portion of the image becomes the portion to be registered (the portion included in area A23). In the example of Figure 6, the user has selected not to display the guide, and the guide is not displayed.

[0076] Area A25 is a display area for the shutter button. When the shutter button in area A25 is touched, terminal device 200 temporarily stores the image captured by camera 240 in first storage unit 180, and after the user confirms it, stores the image in area A23 in first storage unit 180 as a registered image.

[0077] 7 is a diagram showing a second example of the monitor screen displayed by the second display unit 220 when the camera 240 captures a registration image. The example of FIG. 7 differs from the example of FIG. 6 in that a guide is displayed. In the example of FIG. 7, the user selects the guide display by touching area A24. As a result, two lines, L11 and L12, are displayed in area A23. In other respects, the example of FIG. 7 is similar to the example of FIG. 6.

[0078] Lines L11 and L12 are lines that indicate a guide for adjusting the ridge width. By aligning the monitor image so that both ends of the ridge width (both ends in the short direction of the mounded soil portion) roughly overlap with lines L11 and L12, the user is expected to be able to capture a registered image so that the entire short-side area of ​​the ridge to be captured is included in the registered image, and adjacent ridges are not included in the registered image.

[0079] Here, if part of the short side of the ridge being photographed is cut off from the registered image, or if an adjacent ridge is included in the registered image, the number of plants entered by the user will not match the number of plants actually appearing in the registered image, which may reduce the accuracy of the harvest yield prediction by the estimation device 100.

[0080] In contrast, if the user aligns the monitor image so that both ends of the ridge width roughly overlap with lines L11 and L12, it is expected that the user will be able to capture the registered image so that the entire width of the ridge being photographed is included in the registered image, and adjacent ridges are not included in the registered image. This is expected to enable the estimation device 100 to predict the harvest yield with relatively high accuracy.

[0081] Fig. 8 is a diagram showing an example of an image registration screen displayed by the second display unit 220. The second display unit 220 displays the screen shown in Fig. 8 when the user selects to register an image that is already saved in the second storage unit 280 in the example of Fig. 4 or the example of Fig. 5. The second display unit 220 also displays the screen shown in Fig. 8 when the user takes a photograph (i.e., when the user touches the shutter button) in the example of Fig. 6 or the example of Fig. 7.

[0082] In the example of Fig. 8, area A31 is an area that displays a message about the image registration procedure. Area A32 is an area that displays an image for registration. The second display unit 220 displays in area A32 an image that has been taken as an image for registration, or an image that has been selected as an image for registration from among the images stored in the second storage unit 280. Area A33, like area A24 in the examples of Figs. 6 and 7, is an area that accepts a switching operation to switch between displaying and not displaying a guide for trimming the registration image.

[0083] In the example of Figure 8, the user has selected the guide display, and lines L21 and L22 are displayed in area A32. Lines L21 and L22, like lines L11 and L12 in Figure 7, are lines that indicate a guide for adjusting the ridge width. By aligning the monitor image so that both ends of the ridge width roughly overlap lines L21 and L22, the user is expected to be able to trim the registered image so that the entire width of the ridge being photographed is included in the registered image, and adjacent ridges are not included in the registered image. This is expected to enable the estimation device 100 to predict the yield with relatively high accuracy, as described for the example of Figure 7.

[0084] Area A34 is a display area for buttons for redoing image capture or selection. Fig. 8 shows an example in which the user captures an image for registration using camera 240, and a "retake" button is displayed in area A34. When the user performs a touch operation on the "retake" button, terminal device 200 displays the monitor screens exemplified in Figs. 6 and 7 without registering the image displayed in area A32, and accepts a user operation to capture an image using camera 240.

[0085] When the user selects an image to be registered from the images already stored in the second storage unit 280 and the second display unit 220 displays the screen illustrated in Fig. 8, the second display unit 220 displays a reselect button in area A34. When the user performs a touch operation on the reselect button, the terminal device 200 displays a screen for image selection and accepts the user's operation to select an image without registering the image displayed in area A32.

[0086] Area A35 is a display area for a save button. When a user touches the save button, the terminal device 200 saves the image displayed in area A32 as a registered image in the first storage unit 180. At that time, the first communication unit 110 may transmit the registered image to the estimation device 100. Alternatively, the first communication unit 110 may transmit the registered image to the estimation device 100 at the timing when a user operation is performed to instruct transmission of the registered image.

[0087] 9 is a diagram showing a first example of a processing procedure when the estimation system 1 predicts the yield of fruit in a farm field. In the processing of FIG. 9, the data acquisition unit 192 acquires growth status data (step S11). For example, the data acquisition unit 192 acquires the growth status data from a server device that provides the growth status data via the first communication unit 110. Alternatively, the estimation device 100 may acquire the growth status data in advance (before starting the processing of FIG. 9).

[0088] Next, the photography location instruction unit 191 determines a location where the tomatoes should be photographed by referring to the growth status data (step S12). The photography location instruction unit 191 may determine a cell showing an average growth status relative to the growth status of tomatoes in the field, or a location within that cell, as the location where the tomatoes should be photographed. In this case, the photography location instruction unit 191 may determine one location or multiple locations as the location where the tomatoes should be photographed. Under the control of the first control unit 190, the first communication unit 110 transmits to the terminal device 200 a photography instruction specifying the location determined by the photography location instruction unit 191 and instructing the terminal device 200 to take a photograph (step S13).

[0089] In the terminal device 200 that has received the image capturing instruction, the camera 240 captures an image of tomatoes at the specified location under the control of the second control unit 290 (step S14). For example, a user refers to the image capturing instruction, carries the terminal device 200, moves to the specified location, and captures an image of tomatoes using the camera 240 at the specified location. The second communication unit 210 transmits the captured image and data such as the number of plants captured in the image to the estimation device 100 under the control of the second control unit 290 (step S15). The second communication unit 210 transmits the above-mentioned registered image to the estimation device 100 as a captured image.

[0090] The estimation device 100, which has received the data from the terminal device 200, estimates the number of fruits in the area of ​​the farm field that appears in the captured image for each captured image from the terminal device 200 (step S16). The estimation device 100 treats the captured image from the terminal device 200 as the target image described above, and estimates the number of fruits in the area of ​​the farm field that appears in the captured image.

[0091] Fig. 10 is a diagram showing a first example of a processing procedure by which the estimation device 100 estimates the number of fruits in an area of ​​a farm field that is captured in a captured image. In step S16 of Fig. 9, the estimation device 100 performs the processing of Fig. 10 for each captured image from the terminal device 200. In the processing of Fig. 10, the image analysis unit 193 performs image recognition processing on the target image to detect images of red fruits and count the number of detected red fruits (step S21). The target image here is the captured image from the terminal device 200 that is the target of analysis by the image analysis unit 193. The image analysis unit 193 also detects red pixels and green pixels from the pixels of the target image and counts the number of red pixels and the number of green pixels (step S22).

[0092] After steps S21 and S22, the quantity estimation unit 194 estimates the number of fruits in the area of ​​the farm field that is captured in the target image based on the number of red fruits, the number of red pixels, and the number of green pixels counted by the image analysis unit 193 (step S23). The image analysis unit 193 estimates the number of fruits in the area that is captured in the target image based on, for example, the above formula (1). After step S23, the estimation device 100 ends the processing of FIG. 10.

[0093] The estimation device 100 may determine whether the captured image is suitable for predicting the yield, and if it is determined that the captured image is not suitable, may instruct the user to retake the image. For example, the image analysis unit 193 may calculate the ratio of the number of red pixels calculated in step S22 to the total number of pixels in the target image and compare it with a predetermined threshold. If the image analysis unit 193 determines that the calculated ratio is less than the threshold, the first control unit 190 may control the first display unit 120 to display a message instructing the user to retake the image and the captured image for which it has been determined that the calculated ratio is less than the threshold.

[0094] When the estimation device 100 finishes the process of Fig. 10, the process returns to the process of Fig. 9, where the quantity estimation unit 194 predicts the harvest yield in the field based on the estimated value of the number of fruits for each captured image from the terminal device 200 (step S17 in Fig. 9). The quantity estimation unit 194 calculates the predicted harvest yield in the field based on, for example, the above equations (3) and (4). After step S17, the estimation system 1 finishes the process of Fig. 9.

[0095] 9, the quantity estimation unit 194 may predict the total weight of fruits at harvest time in the area of ​​the farm field that is captured in the captured image. For example, the quantity estimation unit 194 may multiply the calculated estimated value of the number of fruits by the standard weight W of one red fruit. ｓｔａｎｄａｒｄ By multiplying the calculated values ​​by 1 / 2, a predicted value of the total weight of fruit at harvest time in the area of ​​the farm field that is captured in the captured image can be calculated.

[0096] In this case, the quantity estimation unit 194 calculates "N ｆｒｕｉｔ＿ａｌｌ （ｉ） " is replaced with the predicted total weight of the fruit at harvest in the area of ​​the field captured in the photographed image, and "*W ｓｔａｎｄａｒｄ Using the formula without ", the predicted total weight of fruit at harvest in the field, W ｆｒｕｉｔ＿ａｌｌ（ｆｉｅｌｄ） Calculate.

[0097] In step S17 of Fig. 9, the quantity estimation unit 194 may calculate an estimate of the number of fruits in the field. For example, the quantity estimation unit 194 may calculate "*W ｓｔａｎｄａｒｄ The formula without " may be used to calculate an estimate of the number of fruits in the field.

[0098] For example, the varieties of tomatoes vary from field to field, and the standard weight of each red fruit is W ｓｔａｎｄａｒｄ If the weights are different, the quantity estimation unit 194 calculates an estimated value of the number of fruits in the field to obtain the standard weight W per red fruit. ｓｔａｎｄａｒｄ The estimation system 1 can be applied to each field without the need to change the settings for each field.

[0099] A person (e.g., a user of the estimation system 1) who refers to the estimated value of the number of fruits in each field calculated by the quantity estimation unit 194 for each field may want to estimate the standard weight W of each red fruit in the field for which the yield is to be predicted. ｓｔａｎｄａｒｄ By multiplying this by the estimated number of fruits in the field, a predicted yield for the field can be obtained.

[0100] The colors of the fruits may be classified into three or more colors. In this case, the estimation device 100 may treat fruits whose color range at least partially overlaps with that of the leaf color range in the same manner as the green fruits described above, and treat fruits of other colors in the same manner as the red fruits described above.

[0101] For example, if there are red fruits, orange fruits, and green fruits, the image analysis unit 193 may treat the sum of the number of red fruits and the number of orange fruits in the same way as the number of red fruits described above, and may treat the sum of the number of red pixels and the number of orange pixels in the same way as the number of red pixels described above.

[0102] Here, the term "orange" refers to a fruit that falls within a predetermined color range for oranges. The color range for oranges does not overlap with the color range for leaves. The term "orange pixel" refers to a pixel that falls within the color range for oranges.

[0103] As described above, the image analysis unit 193 may count the number of red fruits, the number of green fruits, and the number of leaves, instead of the number of red pixels and the number of green pixels. Fig. 11 is a diagram showing a second example of the processing procedure by which the estimation device 100 estimates the number of fruits in an area of ​​a farm field that is captured in a captured image. Fig. 11 shows an example in which the image analysis unit 193 counts the number of green fruits and the number of leaves, instead of the number of red pixels and the number of green pixels. In step S16 of Fig. 9, the estimation device 100 performs the processing of Fig. 11 instead of the processing of Fig. 10 for each captured image from the terminal device 200.

[0104] 11 , the image analysis unit 193 performs image recognition processing on the target image to detect images of red fruits, green fruits, and leaves, and counts the number of red fruits, the number of green fruits, and the number of leaves (step S31). The target image here is an image captured by the terminal device 200 and is the target of analysis by the image analysis unit 193.

[0105] Then, the quantity estimation unit 194 estimates the number of fruits in the area of ​​the farm field that is captured in the target image based on the number of red fruits, the number of green fruits, and the number of leaves counted by the image analysis unit 193 (step S32). The image analysis unit 193 estimates the number of fruits in the area that is captured in the target image based on, for example, equation (5).

[0106]

[0107] N ｆｒｕｉｔ＿ｄｅｔｅｃｔｅｄ＿ｇｒｅｅｎ indicates the number of green fruits in the target image. ｌｅａｆ＿ｄｅｔｅｃｔｅｄ indicates the number of leaves in the target image. g indicates the number of red fruits in the target image. ｆｒｕｉｔ＿ｄｅｔｅｃｔｅｄ＿ｒｅｄ and the number of green fruits in the target image, N ｆｒｕｉｔ＿ｄｅｔｅｃｔｅｄ＿ｇｒｅｅｎ and the number of leaves in the target image, N ｌｅａｆ＿ｄｅｔｅｃｔｅｄ The input is used to calculate the estimated number of fruits in the target image, N ｆｒｕｉｔ＿ａｌｌThe function g may be determined by learning or a statistical method. After step S32, the estimation device 100 ends the processing in FIG. 11. In this case, the processing proceeds to step S17 in FIG. 9, as in the case where the estimation device 100 ends the processing in FIG. 10.

[0108] The terminal device 200 may photograph tomatoes at any multiple locations in the field. Fig. 12 is a diagram showing a second example of the processing procedure when the estimation system 1 predicts the fruit yield in the field. Fig. 12 shows an example in which the terminal device 200 photographs tomatoes at any multiple locations in the field.

[0109] 12 , the camera 240 of the terminal device 200 photographs tomatoes at each of a plurality of locations in the field (step S41) under the control of the second control unit 290. For example, a user moves around the field carrying the terminal device 200 and photographs tomatoes at a plurality of locations in the field using the camera 240.

[0110] Alternatively, the photography location instruction unit 191 may determine, as the location where the tomatoes should be photographed, a plurality of cells having various values ​​of growth status data or a location within each of those cells. Then, as in step S13 of Fig. 9, the first communication unit 110 may, under the control of the first control unit 190, transmit to the terminal device 200 a photography instruction specifying the location determined by the photography location instruction unit 191 and instructing the terminal device 200 to take a photograph.

[0111] 12, the camera 240 photographs the tomatoes at each of the locations determined by the photographing location instruction unit 191 as locations where the tomatoes should be photographed, under the control of the second control unit 290. Steps S42 and S43 are the same as steps S15 and S16 in FIG.

[0112] The data acquisition unit 192 also acquires growth status data (step S44). The process in which the data acquisition unit 192 acquires growth status data in step S44 is similar to the process in step S11 of Fig. 9. In the process of Fig. 12, the data acquisition unit 192 may acquire the growth status data at any time prior to step S45. Alternatively, the estimation device 100 may acquire the growth status data in advance (before the process of Fig. 12 begins).

[0113] Next, the quantity estimation unit 194 predicts the total weight of fruit in each cell of the field at the time of harvest (step S45). As described above, a cell here is a portion of the field, and the growth status data indicates the growth status of tomatoes for each cell. For example, the quantity estimation unit 194 calculates a predicted value of the total weight of fruit at the time of harvest in each cell where an image was captured, based on equation (6).

[0114]

[0115] Equation (6) can be considered as a transformation of equation (3) from an equation for predicting the total weight of fruit in a field to an equation for predicting the total weight of fruit in an individual cell. ｆｒｕｉｔ＿ａｌｌ（ｃｅｌｌ） （ｉ） indicates the predicted value of the total weight of fruit at harvest in the cell containing the i-th photographing location. The cell containing the i-th photographing location is also referred to as the i-th photographing target cell. S ｃｅｌｌ indicates the area of ​​one cell. If the area of ​​each cell is different, S ｃｅｌｌ Instead, the area of ​​the i-th imaging target cell is used.

[0116] The quantity estimation unit 194 interpolates or extrapolates the predicted total weight of fruit at harvest in the non-photographed cells based on the value of the growth status data for each photographed cell, the predicted total weight of fruit at harvest calculated for that cell, and the value of the growth status data for the non-photographed cells. Cells that include the photographed location correspond to the photographed cells, and cells that do not include the photographed location correspond to the non-photographed cells. The method of interpolation or extrapolation in this case is not limited to a specific one. For example, the quantity estimation unit 194 may perform linear interpolation, but is not limited to this.

[0117] The quantity estimation unit 194 may predict the total weight of fruit at harvest in the non-photographed cells based on the relationship between the value of the growth status data in the photographed cells and the predicted total weight of fruit at harvest, and the value of the growth status data in the non-photographed cells. The interpolation or extrapolation described above can be considered as one method of implementing this prediction. Alternatively, the relationship between the value of the growth status data in the photographed cells and the predicted total weight of fruit at harvest may be determined by learning or a statistical method.

[0118] After step S45, the quantity estimation unit 194 calculates a predicted value of the total weight of fruit at harvest in the field by summing the predicted values ​​of the total weight of fruit calculated for each cell for all cells included in the field (step S46). The quantity estimation unit 194 calculates a predicted value W of the total weight of fruit at harvest in the field based on equation (7). ｆｒｕｉｔ＿ａｌｌ（ｆｉｅｌｄ） Calculate.

[0119]

[0120] N ｃｅｌｌ indicates the number of cells included in the field, and j is an identification number that identifies all cells included in the field, where 1≦j≦N ｃｅｌｌ The cell identified by the identification number j is also called the j-th cell. ｆｒｕｉｔ＿ａｌｌ（ｃｅｌｌ） ［ｊ］ denotes the predicted value of the total weight of fruit at harvest in the jth cell.

[0121] As described above, the quantity estimation unit 194 estimates the predicted value W of the total weight of the fruits at the time of harvest in the farm field. ｆｒｕｉｔ＿ａｌｌ（ｆｉｅｌｄ） may be used as the predicted value of the harvest yield in the field. Alternatively, if some fruits in the field are expected not to be harvested, the quantity estimation unit 194 may calculate the predicted value of the harvest yield in the field based on the above formula (4). After step S46, the estimation system 1 ends the processing of FIG. 12.

[0122] 13 is a diagram illustrating an example of the configuration of a learning device according to at least one embodiment. In the configuration illustrated in FIG. 13, learning device 300 includes third communication unit 310, third display unit 320, third operation input unit 330, third storage unit 380, and third control unit 390. Third control unit 390 includes training data acquisition unit 391 and learning unit 392.

[0123] The learning device 300 performs training of a model used for estimation performed by the estimation device 100. Model training here refers to adjusting the values ​​of learning parameters (parameters to be learned) of the model based on training data. The model to be learned by the learning device 300 is not limited to a specific type. For example, the learning device 300 may perform training of a neural network, or may perform training of a mathematical formula (mathematical model) having learning parameters.

[0124] The learning device 300 may be configured using a computer such as a personal computer. The learning device 300 may be configured integrally with the estimation device 100. For example, the learning device 300 may be configured using the computer that constitutes the estimation device 100. Alternatively, the learning device 300 and the estimation device 100 may be configured as separate devices.

[0125] The third communication unit 310 communicates with other devices. For example, the third communication unit 310 may receive training data from the estimation device 100 or a server device that stores the training data. The third communication unit 310 may also transmit a trained model to the estimation device 100. The third display unit 320 has a display screen such as a liquid crystal panel, an organic EL panel, or an LED panel, and acquires various images. For example, the third display unit 320 may display information related to learning, such as the degree of progress of learning or the values ​​of learning parameters.

[0126] The third operation input unit 330 includes input devices such as a keyboard and a mouse, and accepts user operations. For example, the third operation input unit 330 may accept a user operation to set a value of a meta-parameter related to learning, such as a learning rate. The third memory unit 380 stores various data. For example, the third memory unit 380 may store a model to be learned. The third memory unit 380 is configured using a memory device included in the learning device 300.

[0127] The third control unit 390 controls each unit of the learning device 300 to perform various processes. The functions of the third control unit 390 are performed, for example, by a CPU included in the learning device 300 reading and executing a program from the third storage unit 380. The training data acquisition unit 391 acquires training data. For example, the training data acquisition unit 391 may extract training data from a signal received by the third communication unit 310 from the estimation device 100 or a server device that stores training data, and store the training data in the third storage unit 380.

[0128] For example, when the learning device 300 learns the model shown in equation (1) or the model shown in equation (2), the training data acquisition unit 391 may input the number of red fruits, the number of red pixels, and the number of green pixels in the image into the model, and acquire training data in which the actual measured value of the number of fruits in the area shown in the image is the correct answer (teacher data).

[0129] Furthermore, when the learning device 300 learns the model shown in equation (5), the training data acquisition unit 391 may input the number of red fruits in the image, the number of green fruits in the image, and the number of leaves in the image into the model, and acquire training data in which the actual measured value of the number of fruits in the area shown in the image is the correct answer.

[0130] Furthermore, when the learning device 300 learns a model showing the relationship between the value of growth status data in a certain cell and the predicted value of the total weight of fruit at harvest in that cell, the training data acquisition unit 391 may input the value of the growth status data in the cell to be photographed into the model and acquire training data in which the predicted value of the total weight of fruit at harvest calculated by the estimation device 100 for that cell to be photographed is the correct answer.

[0131] The learning unit 392 learns the model using the training data acquired by the training data acquisition unit 391. The learning method used by the learning unit 392 to learn the model is not limited to a specific method. For example, when the learning unit 392 learns a differentiable model, it may use backpropagation, but is not limited to this.

[0132] As described above, the image analysis unit 193 identifies at least one of the number or quantity of fruits shown in an image of at least a portion of a field where tomatoes are planted, and the area of ​​tomato leaves in the image, based on the colors in the image. The quantity estimation unit 194 estimates at least one of the number or quantity of fruits in the field, based on the number or quantity of fruits shown in the image and the area of ​​tomato leaves in the image.

[0133] In this way, the quantity estimation unit 194 is expected to be able to estimate at least the number or amount of fruits in a farm field based on at least the number or amount of fruits shown in an image and the area of ​​tomato leaves in the image, thereby enabling more accurate estimation than estimation based only on at least the number or amount of fruits shown in an image. For example, it is expected that the quantity estimation unit 194 can estimate at least the number or amount of fruits in a farm field by taking into account fruits hidden by tomato leaves.

[0134] The image analysis unit 193 also identifies pixels within a predetermined color range for tomato leaf colors as the tomato leaf region and counts the number of identified pixels. The image analysis unit 193 is expected to be able to accurately count the number of pixels even in an image in which individual leaves are difficult to detect, such as when tomato leaves are overlapping. The estimation device 100 is expected to be able to estimate at least the number or quantity of fruits in a field with relatively high accuracy.

[0135] The image analysis unit 193 also counts the number of fruits shown in the image. The quantity estimation unit 194 calculates a predicted value for the total harvestable weight of the harvest target in the field. The image analysis unit 193 only needs to count the quantity of fruit, without needing to distinguish whether the fruit is harvestable or not at the time of image capture. In this respect, the estimation device 100 is expected to be able to estimate the total harvestable weight of the harvest target in the field with relatively high accuracy.

[0136] The image analysis unit 193 also counts the number of fruits within each of a plurality of predetermined color ranges for the fruit colors. At least a portion of at least one of the plurality of predetermined color ranges for the fruit colors overlaps with at least a portion of the predetermined color range for the tomatoes. The estimation device 100 can estimate at least the number or quantity of fruits in a field even when there are three or more different fruit colors, such as when there are red, green, and orange fruits.

[0137] Furthermore, the quantity estimation unit 194 estimates, for each image, based on an image taken of at least one location in the field, at least either the number or the quantity of the harvest objects at the location where the image was taken, and estimates, for each image, based on at least either the number or the quantity of fruits at the location where the image was taken and data indicating the growth status of tomatoes in each part of the field, at least either the number or the quantity of fruits in the field.

[0138] The estimation device 100 can estimate at least the number or quantity of fruits in a field, taking into account variations in the growth conditions of tomatoes in different parts of the field. In this respect, the estimation device 100 is expected to be able to estimate the total harvestable weight of harvest targets in the field with relatively high accuracy.

[0139] Furthermore, the images are taken at locations determined based on data showing the growth conditions of tomatoes in each part of the field. The estimation device 100 can collect data taking into account variations in the growth conditions of tomatoes in each part of the field. In this respect, the estimation device 100 is expected to be able to estimate the total harvestable weight of harvest targets in the field with relatively high accuracy.

[0140] The image is an image taken at at least one location determined as a location with an average tomato growth condition based on data indicating the tomato growth condition for each part of the field. The quantity estimation unit 194 estimates at least either the number or the quantity of fruits per tomato plant in the part of the field shown in the image, and estimates at least either the number or the quantity of fruits in the field based on at least either the number or the quantity of fruits per tomato plant in the part of the field shown in the image and data indicating the number of fruit plants in the field.

[0141] The estimation device 100 estimates at least either the number or quantity of fruits per plant based on data from a location with average tomato growth conditions, and is expected to be able to estimate at least either the number or quantity of fruits in a field with relatively high accuracy by a relatively simple calculation of multiplying this estimated value by the number of plants in the field.

[0142] The second display unit 220 also displays an image of the field and a guideline for aligning the width of the ridges shown in the image of the field. By aligning the image so that both ends of the ridge width roughly match the guideline, the terminal device 200 is expected to obtain a registered image in which the entire width of the ridge is included in the registered image, but adjacent ridges are not included in the registered image. The estimation device 100 is expected to be able to estimate at least one of the number and quantity of fruits in the field using this image, enabling relatively high-accuracy estimation.

[0143] Second Embodiment Fig. 14 is a diagram showing an example of the configuration of an estimation device according to at least one embodiment. In the configuration shown in Fig. 14, an estimation device 610 includes an image analysis unit 611 and a quantity estimation unit 612.

[0144] With this configuration, the image analysis unit 611 identifies, from an image captured of at least a portion of a field in which crops having harvest targets are planted, at least one of the number or quantity of harvest targets depicted in the image and the area of ​​crop leaves in the image, based on the colors in the image. The quantity estimation unit 612 estimates at least one of the number or quantity of harvest targets in the field, based on the number or quantity of harvest targets depicted in the image and the area of ​​crop leaves in the image. The image analysis unit 611 is an example of image analysis means. The quantity estimation unit 612 is an example of quantity estimation means.

[0145] In this way, the quantity estimation unit 612 is expected to estimate at least the number or quantity of harvest objects in the field based on at least the number or quantity of harvest objects shown in the image and the area of ​​crop leaves in the image, thereby enabling more accurate estimation than when estimating based solely on at least the number or quantity of harvest objects shown in the image. For example, the quantity estimation unit 612 is expected to be able to estimate at least the number or quantity of harvest objects in the field by taking into account harvest objects hidden by crop leaves. The image analysis unit 611 can be realized, for example, using the functions of the image analysis unit 193 in FIG. 2 , etc. The quantity estimation unit 612 can be realized, for example, using the functions of the quantity estimation unit 194 in FIG. 2 , etc.

[0146] <Third embodiment> Fig. 15 is a diagram showing an example of the configuration of a display device according to at least one embodiment. In the configuration shown in Fig. 15, a display device 620 includes a display unit 621. In this configuration, the display unit 621 displays an image of a farm field and a guide for adjusting the width of ridges shown in the image of the farm field. The display unit 621 corresponds to an example of a display means.

[0147] By using the display device 620, it is expected that an image can be obtained in which the entire width of the ridge is included in the image, but adjacent ridges are not included, by aligning the image so that both ends of the ridge width are roughly aligned with the target. By using this image to estimate at least the number or quantity of fruit in the field, it is expected that the estimation can be performed with relatively high accuracy. The display unit 621 can be realized, for example, using the functions of the second display unit 220 in FIG. 3 .

[0148] <Fourth embodiment> Fig. 16 is a diagram showing an example of the configuration of an image generation device according to at least one embodiment. In the configuration shown in Fig. 16, an image generation device 630 includes a camera 631 and a display unit 632. In this configuration, the display unit 632 displays a real-time image captured by the camera 631 and a guide for adjusting the width of the ridges in the field shown in the image. The display unit 632 corresponds to an example of a display means.

[0149] By using the image generation device 630, it is expected that by aligning the image so that both ends of the ridge width roughly match the target, a captured image can be obtained in which the entire width of the ridge is included in the image, but adjacent ridges are not included in the image. By using this captured image to estimate at least the number or quantity of fruit in the field, it is expected that estimation can be performed with relatively high accuracy. The camera 631 can be realized, for example, using the functions of the camera 240 in FIG. 3 . The display unit 632 can be realized, for example, using the functions of the second display unit 220 in FIG. 3 .

[0150] Fifth Embodiment Fig. 17 is a diagram showing an example of a processing procedure in an estimation method according to at least one embodiment. The estimation method shown in Fig. 17 includes analyzing an image (step S611) and estimating a quantity (step S612).

[0151] In analyzing the image (step S611), the computer identifies at least one of the number and quantity of harvest objects in the image and the area of ​​the crop leaves in the image based on the color of the image from an image of at least a portion of a field where crops having harvest objects are planted. In estimating the quantity (step S612), the computer estimates at least one of the number and quantity of harvest objects in the field based on the number and quantity of harvest objects in the image and the area of ​​the crop leaves in the image.

[0152] 17 , by estimating at least the number or amount of harvest objects in a field based on at least the number or amount of harvest objects shown in an image and the area of ​​crop leaves in the image, it is expected that estimation will be more accurate than estimation based only on at least the number or amount of harvest objects shown in an image. For example, it is expected that the estimation method shown in FIG. 17 will be able to estimate at least the number or amount of harvest objects in a field by taking into account harvest objects hidden by crop leaves.

[0153] Sixth Embodiment Fig. 18 is a diagram showing an example of a processing procedure in a display method according to at least one embodiment. The display method shown in Fig. 18 includes performing display (step S621). In performing display (step S621), a display device displays an image of a field and a guide for adjusting the width of ridges shown in the image of the field.

[0154] 18, by aligning the image so that both ends of the ridge width are roughly aligned with the target, it is expected that an image can be obtained in which the entire width of the ridge is included in the image, but adjacent ridges are not included in the image. By using this image to estimate at least the number or quantity of fruit in the field, it is expected that the estimation can be performed with relatively high accuracy.

[0155] Seventh Embodiment Fig. 19 is a diagram showing an example of a processing procedure in an image generation method according to at least one embodiment. The image generation method shown in Fig. 19 includes performing display (step S631). In performing display (step S631), an image generation device having a camera displays a real-time image captured by the camera and a guide for adjusting the width of the ridges in the field shown in the image.

[0156] 19, by aligning the image so that both ends of the ridge width are roughly aligned with the target, it is expected that a captured image can be obtained in which the entire width of the ridge is included in the image, but adjacent ridges are not included in the image. By using this captured image to estimate at least the number or quantity of fruit in the field, it is expected that the estimation can be performed with relatively high accuracy.

[0157] 20 is a diagram illustrating an example of a computer configuration according to at least one embodiment. In the configuration shown in FIG. 20, a computer 700 includes a CPU 710, a main memory device 720, an auxiliary memory device 730, an interface 740, and a non-volatile recording medium 750.

[0158] One or more of the above-described estimation device 100, terminal device 200, learning device 300, estimation device 610, display device 620, and image generation device 630, or a part thereof, may be implemented in a computer 700. In this case, the operation of each of the above-described processing units is stored in the auxiliary storage device 730 in the form of a program. The CPU 710 reads the program from the auxiliary storage device 730, loads it into the main storage device 720, and executes the above-described processing in accordance with the program. The CPU 710 also allocates storage areas in the main storage device 720 corresponding to each of the above-described storage units in accordance with the program. Communication between each device and other devices is performed by an interface 740 having a communication function and performing communication under the control of the CPU 710. The interface 740 also has a port for a non-volatile storage medium 750, and reads information from the non-volatile storage medium 750 and writes information to the non-volatile storage medium 750.

[0159] When the estimation device 100 is implemented in a computer 700, the operations of the first control unit 190 and each of its units are stored in the form of a program in an auxiliary storage device 730. The CPU 710 reads the program from the auxiliary storage device 730, loads it into the main storage device 720, and executes the above-described processing in accordance with the program.

[0160] Furthermore, the CPU 710 allocates a storage area for the first storage unit 180 in the main storage device 720 in accordance with the program. Communication with other devices by the first communication unit 110 is performed by the interface 740 having a communication function and operating under the control of the CPU 710. Display of images by the first display unit 120 is performed by the interface 740 having a display device and displaying various images under the control of the CPU 710. Reception of user operations by the first operation input unit 130 is performed by the interface 740 having an input device and receiving user operations under the control of the CPU 710.

[0161] When the terminal device 200 is implemented in the computer 700, the operations of the second control unit 290 and each of its units are stored in the form of a program in the auxiliary storage device 730. The CPU 710 reads the program from the auxiliary storage device 730, loads it into the main storage device 720, and executes the above-described processing in accordance with the program.

[0162] Furthermore, the CPU 710 allocates a storage area for the second storage unit 280 in the main storage device 720 in accordance with the program. Communication with other devices by the second communication unit 210 is performed by the interface 740 having a communication function and operating under the control of the CPU 710. Display of images by the second display unit 220 is performed by the interface 740 having a display device and displaying various images under the control of the CPU 710. Reception of user operations by the second operation input unit 230 is performed by the interface 740 having an input device and receiving user operations under the control of the CPU 710.

[0163] When learning device 300 is implemented in computer 700, the operations of third control unit 390 and each of its units are stored in the form of a program in auxiliary storage device 730. CPU 710 reads the program from auxiliary storage device 730, loads it into main storage device 720, and executes the above-described processing in accordance with the program.

[0164] Furthermore, the CPU 710 allocates a storage area for the third storage unit 380 in the main storage device 720 in accordance with the program. Communication with other devices by the third communication unit 310 is performed by the interface 740 having a communication function and operating under the control of the CPU 710. Display of images by the third display unit 320 is performed by the interface 740 having a display device and displaying various images under the control of the CPU 710. Acceptance of user operations by the third operation input unit 330 is performed by the interface 740 having an input device and accepting user operations under the control of the CPU 710.

[0165] When the estimation device 610 is implemented in the computer 700, the operations of the image analysis unit 611 and the quantity estimation unit 612 are stored in the form of a program in the auxiliary storage device 730. The CPU 710 reads the program from the auxiliary storage device 730, loads it into the main storage device 720, and executes the above-described processing in accordance with the program.

[0166] Furthermore, the CPU 710 allocates a storage area in the main storage device 720 for the estimating device 610 to perform processing in accordance with the program. Communication between the estimating device 610 and other devices is performed by the interface 740, which has a communication function and operates under the control of the CPU 710. Interaction between the estimating device 610 and a user is performed by the interface 740, which has an input device and an output device, presenting information to the user via the output device under the control of the CPU 710 and accepting user operations via the input device.

[0167] When the display device 620 is implemented in the computer 700, the operation of the display device 620 is stored in the form of a program in the auxiliary storage device 730. The CPU 710 reads the program from the auxiliary storage device 730, loads it into the main storage device 720, and executes the above-described processing in accordance with the program.

[0168] Furthermore, the CPU 710 allocates a storage area in the main storage device 720 for the display device 620 to perform processing in accordance with the program. Communication between the display device 620 and other devices is achieved by the interface 740 having a communication function and operating under the control of the CPU 710. Display of images by the display unit 621 is achieved by the interface 740 having a display device and displaying various images under the control of the CPU 710. Acceptance of user operations by the display device 620 is achieved by the interface 740 having an input device and accepting user operations under the control of the CPU 710.

[0169] When the image generating device 630 is implemented in the computer 700, the operation of the image generating device 630 is stored in the form of a program in the auxiliary storage device 730. The CPU 710 reads the program from the auxiliary storage device 730, loads it into the main storage device 720, and executes the above-described processing in accordance with the program.

[0170] Furthermore, the CPU 710 allocates a storage area in the main storage device 720 for the image generation device 630 to perform processing in accordance with a program. Communication between the image generation device 630 and other devices is achieved by the interface 740 having a communication function and operating under the control of the CPU 710. Photography by the camera 631 is achieved by the interface 740 being equipped with a camera and performing photography under the control of the CPU 710. Display of images by the display unit 632 is achieved by the interface 740 being equipped with a display device and displaying various images under the control of the CPU 710. Reception of user operations by the image generation device 630 is achieved by the interface 740 being equipped with an input device and receiving user operations under the control of the CPU 710.

[0171] One or more of the above-described programs may be recorded on nonvolatile recording medium 750. In this case, interface 740 may read the programs from nonvolatile recording medium 750. Then, CPU 710 may directly execute the programs read by interface 740, or may temporarily store the programs in main storage device 720 or auxiliary storage device 730 and then execute them.

[0172] Note that a program for executing all or part of the processing performed by the estimation device 100, the terminal device 200, the learning device 300, the estimation device 610, the display device 620, and the image generation device 630 may be recorded on a computer-readable recording medium, and the program recorded on the recording medium may be loaded into a computer system and executed to perform the processing of each unit. Note that the term "computer system" herein includes hardware such as an operating system (OS) and peripheral devices. Furthermore, the term "computer-readable recording medium" refers to portable media such as a flexible disk, a magneto-optical disk, a read-only memory (ROM), a compact disc read-only memory (CD-ROM), and a storage device such as a hard disk built into a computer system. The program may be for implementing part of the aforementioned functions, or may be capable of implementing the aforementioned functions in combination with a program already recorded on the computer system.

[0173] Although the embodiments of the present disclosure have been described in detail above with reference to the drawings, the specific configuration is not limited to these embodiments and includes designs that do not deviate from the gist of the present disclosure. In addition, the above-described embodiments may be combined with other embodiments as appropriate.

[0174] A part or all of the above-described embodiments can be described as, but not limited to, the following supplementary notes.

[0175] (Supplementary Note 1) An estimation device comprising: an image analysis means for identifying, from an image taken of at least a portion of a field in which a crop having harvest objects is planted, at least one of the number or quantity of harvest objects shown in the image and the area of ​​the leaves of the crop based on the color in the image; and a quantity estimation means for estimating at least one of the number or quantity of the harvest objects in the field based on at least one of the number or quantity of the harvest objects shown in the image and the area of ​​the leaves of the crop in the image.

[0176] (Supplementary Note 2) The estimation device according to Supplementary Note 1, wherein the image analysis means identifies pixels that fall within a predetermined color range as the leaf color of the crop, and counts the number of identified pixels.

[0177] (Supplementary Note 3) The estimation device according to Supplementary Note 1 or Supplementary Note 2, wherein the image analysis means counts the number of harvest objects shown in the image, and the quantity estimation means calculates a predicted value of a total harvestable weight of the harvest objects in the field.

[0178] (Supplementary Note 4) The estimation device described in any one of Supplementary Notes 1 to 3, wherein the image analysis means counts the number of harvest objects falling within each of a plurality of color ranges predetermined as the colors of the harvest objects, and at least a portion of at least one of the plurality of color ranges predetermined as the colors of the harvest objects overlaps with at least a portion of the color range predetermined as the color of the leaves.

[0179] (Supplementary Note 5) The estimation device described in any one of Supplementary Notes 1 to 4, wherein the quantity estimation means estimates, for each image based on an image taken of at least one location in the field, at least one of the number or quantity of the harvest objects at the location where the image was taken, and estimates at least one of the number or quantity of the harvest objects in the field based on at least one of the number or quantity of the harvest objects at the location where the image was taken and data indicating the growth status of the crops in each part of the field.

[0180] (Supplementary Note 6) The estimation device according to Supplementary Note 5, wherein the images are images taken at locations determined based on data indicating a growth status of the crop for each portion of the field.

[0181] (Supplementary Note 7) The estimation device described in any one of Supplementary Notes 1 to 4, wherein the image is an image taken at at least one location determined to be an average location for the growth conditions of the crop based on data indicating the growth conditions of the crop for each part of the field, and the quantity estimation means estimates at least either the number or the quantity of the harvest objects per plant of the crop in the part of the field shown in the image, and estimates at least either the number or the quantity of the harvest objects in the field based on at least either the number or the quantity of the harvest objects per plant of the crop in the part of the field shown in the image and data indicating the number of plants of the crop in the field.

[0182] (Supplementary Note 8) A display device comprising: a display means for displaying an image of a farm field and a guide for adjusting the width of ridges shown in the image of the farm field.

[0183] (Supplementary Note 9) An image generating device comprising: a camera; and a display means for displaying a real-time image captured by the camera and a guide for adjusting the width of ridges in a field shown in the image.

[0184] (Supplementary Note 10) An estimation method including: a computer taking an image of at least a portion of a field in which a crop having harvest objects is planted, and identifying, based on the color in the image, at least one of the number or quantity of harvest objects shown in the image and the area of ​​the leaves of the crop in the image; and estimating, based on the number or quantity of harvest objects shown in the image and the area of ​​the leaves of the crop in the image, at least one of the number or quantity of harvest objects in the field.

[0185] (Supplementary Note 11) The estimation method according to Supplementary Note 10, wherein the identifying step includes identifying pixels included in a predetermined color range of the leaf color as the leaf area of ​​the crop, and counting the number of identified pixels.

[0186] (Appendix 12) The estimation method described in Appendix 10 or Appendix 11, wherein the identifying step includes counting the number of harvest objects shown in the image, and the estimating step includes calculating a predicted value of the total harvestable weight of the harvest objects in the field.

[0187] (Appendix 13) The estimation method described in any one of Appendices 10 to 12, wherein the identification includes counting the number of harvest objects that fall within each of multiple color ranges that are predetermined as the colors of the harvest objects, and at least a portion of at least one of the multiple color ranges that are predetermined as the colors of the harvest objects overlaps with at least a portion of the color range that is predetermined as the color of the leaves.

[0188] (Appendix 14) The estimation method according to any one of Appendices 10 to 13, wherein the identifying step comprises estimating, for each image based on an image taken of at least one location within the field, at least one of the number or quantity of the harvest objects at the location where the image was taken, and estimating, for each image based on at least one of the number or quantity of the harvest objects at the location where the image was taken and data indicating the growth status of the crops in each part of the field.

[0189] (Supplementary Note 15) The estimation method according to Supplementary Note 14, wherein the image is an image taken at a location determined based on data indicating a growth state of the crop for each part of the field.

[0190] (Appendix 16) The estimation method according to any one of Appendices 10 to 13, wherein the image is an image taken at at least one location determined as a location where the growth conditions of the crop are average based on data showing the growth conditions of the crop for each part of the field, and the identifying includes estimating at least either the number or the quantity of the harvest objects per plant of the crop in the part of the field shown in the image, and estimating at least either the number or the quantity of the harvest objects in the field based on at least either the number or the quantity of the harvest objects per plant of the crop in the part of the field shown in the image and data showing the number of plants of the crop in the field.

[0191] (Supplementary Note 17) A display method including: a display device displaying an image of a farm field and a guide for adjusting the width of ridges shown in the image of the farm field.

[0192] (Supplementary Note 18) An image generating method including: an image generating device having a camera; displaying a real-time image being captured by the camera and a guide for adjusting the width of ridges in a field shown in the image.

[0193] (Supplementary Note 19) A recording medium having recorded thereon a program that causes a computer to perform the following: from an image of at least a portion of a field in which a crop having harvest objects is planted, based on the color in the image, identify at least one of the number or quantity of harvest objects shown in the image and the area of ​​the leaves of the crop in the image; and estimate at least one of the number or quantity of harvest objects in the field based on the number or quantity of harvest objects shown in the image and the area of ​​the leaves of the crop in the image.

[0194] (Supplementary Note 20) The recording medium according to Supplementary Note 19, wherein the program causes the computer to identify pixels that are within a predetermined color range as the leaf color of the crop as the leaf area of ​​the crop, and count the number of identified pixels.

[0195] (Appendix 21) The recording medium described in Appendix 19 or Appendix 20, wherein, in the step of identifying, the program causes the computer to count the number of harvest objects shown in the image, and in the step of estimating, the program causes the computer to calculate a predicted value for the total harvestable weight of the harvest objects in the field.

[0196] (Appendix 22) In the identifying step, the program causes the computer to count the number of harvest objects falling within each of a plurality of color ranges predetermined as the colors of the harvest objects, and at least a portion of at least one of the plurality of color ranges predetermined as the colors of the harvest objects overlaps with at least a portion of the color range predetermined as the color of the leaves. A recording medium described in any one of Appendices 19 to 21.

[0197] (Appendix 23) The recording medium described in any one of Appendices 19 to 22, wherein the identifying step causes the program to cause the computer to estimate, for each image based on an image of at least one location in the field, at least one of the number or quantity of the harvest objects at the location where the image was taken, and to estimate at least one of the number or quantity of the harvest objects in the field based on the at least one of the number or quantity of the harvest objects at the location where the image was taken and data indicating the growth status of the crops in each part of the field.

[0198] (Supplementary Note 24) The recording medium according to Supplementary Note 23, wherein the image is an image taken at a location determined based on data indicating a growth state of the crop for each part of the field.

[0199] (Appendix 25) The image is an image taken at at least one location determined to be an average location for the growth conditions of the crop based on data indicating the growth conditions of the crop for each part of the field, and the identifying step causes the program to estimate at least either the number or quantity of the harvest objects per plant of the crop in the part of the field shown in the image, and to estimate at least either the number or quantity of the harvest objects in the field based on at least either the number or quantity of the harvest objects per plant of the crop in the part of the field shown in the image and data indicating the number of plants of the crop in the field. A recording medium described in any one of Appendices 19 to 22.

[0200] (Appendix 26) A recording medium storing a program that causes a computer that controls a display device to display an image of a farm field and a guide for adjusting the width of ridges shown in the image of the farm field.

[0201] (Supplementary Note 27) A recording medium having recorded thereon a program that causes a computer that controls an image generating device having a camera to display a real-time image captured by the camera and a guide for adjusting the width of ridges in the field that is shown in the image.

[0202] This application claims priority based on Japanese Patent Application No. 2023-149398, filed on September 14, 2023, the contents of which are incorporated herein by reference.

[0203] The present disclosure may be applied to an estimation device, a display device, an image generation device, an estimation method, a display method, an image generation method, and a recording medium.

[0204] 1 Estimation system 100, 610 Estimation device 110 First communication unit 120 First display unit 130 First operation input unit 180 First memory unit 190 First control unit 191 Shooting location instruction unit 192 Data acquisition unit 193, 611 Image analysis unit 194, 612 Quantity estimation unit 200 Terminal device 210 Second communication unit 220 Second display unit 230 Second operation input unit 240 Camera 280 Second memory unit 290 Second control unit 300 Learning device 310 Third communication unit 320 Third display unit 330 Third operation input unit 380 Third memory unit 390 Third control unit 391 Training data acquisition unit 392 Learning unit 620 Display device 621, 632 Display unit 631 Camera 630 Image generation device

Claims

1. An estimation device comprising: an image analysis means for identifying, from an image taken of at least a portion of a field in which a crop having harvest objects is planted, at least one of the number or amount of harvest objects appearing in the image and the area of ​​the leaves of the crop based on the colors in the image; and a quantity estimation means for estimating at least one of the number or amount of harvest objects in the field based on the number or amount of harvest objects appearing in the image and the area of ​​the leaves of the crop in the image.

2. The estimation device according to claim 1, wherein the image analysis means identifies pixels within a range of colors that is predetermined as the color of the leaves as the region of the leaves of the crop, and counts the number of identified pixels.

3. The estimation device according to claim 1 or claim 2, wherein the image analysis means counts the number of harvest objects shown in the image, and the quantity estimation means calculates a predicted value of the total harvestable weight of the harvest objects in the field.

4. The estimation device described in any one of claims 1 to 3, wherein the image analysis means counts the number of harvest objects falling within each of a plurality of color ranges predetermined as the colors of the harvest objects, and at least a portion of at least one of the plurality of color ranges predetermined as the colors of the harvest objects overlaps with at least a portion of the color range predetermined as the color of the leaves.

5. An estimation device as claimed in any one of claims 1 to 4, wherein the quantity estimation means estimates, for each image based on an image taken of at least one location in the field, at least one of the number or amount of the harvest objects at the location where the image was taken, and estimates at least one of the number or amount of the harvest objects in the field based on at least one of the number or amount of the harvest objects at the location where the image was taken for each image and data indicating the growth status of the crops in each part of the field.

6. The estimation device according to claim 5, wherein the images are taken at locations determined based on data showing the growth conditions of the crops in each part of the field.

7. An estimation device as described in any one of claims 1 to 4, wherein the image is an image taken at at least one location determined to be an average location for the growth condition of the crop based on data indicating the growth condition of the crop for each part of the field, and the quantity estimation means estimates at least one of the number or amount of the harvest objects per plant of the crop in the part of the field shown in the image, and estimates at least one of the number or amount of the harvest objects in the field based on at least one of the number or amount of the harvest objects per plant of the crop in the part of the field shown in the image and data indicating the number of plants of the crop in the field.

8. A display device having a display means for displaying an image of a farm field and a guide for adjusting the width of ridges shown in the image of the farm field.

9. An image generating device comprising: a camera; and a display means for displaying a real-time image captured by the camera and a guide for adjusting the width of ridges in a farm field shown in the image.

10. An estimation method comprising: a computer identifying, from an image taken of at least a portion of a field in which a crop having harvest objects is planted, at least one of the number or amount of harvest objects appearing in the image and the area of ​​the leaves of the crop in the image based on the colors in the image; and estimating at least one of the number or amount of harvest objects in the field based on the number or amount of harvest objects appearing in the image and the area of ​​the leaves of the crop in the image.

11. The estimation method according to claim 10, wherein the step of identifying comprises identifying pixels that are within a range of colors that is predetermined as the color of the leaves as the leaf area of ​​the crop, and counting the number of identified pixels.

12. The estimation method according to claim 10 or 11, wherein the identifying step comprises counting the number of harvest objects shown in the image, and the estimating step comprises calculating a predicted total harvestable weight of the harvest objects in the field.

13. The estimation method of any one of claims 10 to 12, wherein the identification comprises counting the number of harvest objects falling within each of a plurality of color ranges that are predefined as the colors of the harvest objects, and at least a portion of at least one of the plurality of color ranges that are predefined as the colors of the harvest objects overlaps with at least a portion of the color range that is predefined as the color of the leaves.

14. An estimation method according to any one of claims 10 to 13, wherein the identifying step comprises estimating, for each image based on an image taken of at least one location in the field, at least one of the number or amount of the harvest objects at the location where the image was taken, and estimating, for each image based on the number or amount of the harvest objects at the location where the image was taken and data indicating the growth status of the crops in each part of the field, at least one of the number or amount of the harvest objects in the field.

15. The estimation method according to claim 14, wherein the images are taken at locations determined based on data showing the growth conditions of the crops in each part of the field.

16. A display method including: a display device displaying an image of a farm field and a guide for adjusting the width of ridges shown in the image of the farm field.

17. An image generating method including: an image generating device having a camera, which displays a real-time image captured by said camera and a guide for adjusting the width of ridges in a field shown in said image.

18. A recording medium having recorded thereon a program that causes a computer to: identify, from an image taken of at least a portion of a field in which a crop having harvest objects is planted, at least one of the number or amount of harvest objects appearing in the image and the area of ​​the leaves of the crop based on the colors in the image; and estimate at least one of the number or amount of harvest objects in the field based on the number or amount of harvest objects appearing in the image and the area of ​​the leaves of the crop in the image.

19. A recording medium having recorded thereon a program for causing a computer that controls a display device to display an image of a farm field and a guide for adjusting the width of ridges shown in the image of the farm field.

20. A recording medium having recorded thereon a program for causing a computer that controls an image generating device having a camera to display a real-time image captured by said camera and a guide for adjusting the width of ridges in a farm field that is shown in said image.

Images