Harvesting system, control method of harvesting system, and program

Abstract

To properly determine whether target crops can be harvested or not.SOLUTION: A plant harvesting system Sys includes: a storage apparatus 3 which stores criteria information DH indicating criteria for determining whether a fruit SF can be harvested or not; a user reference information acquisition unit 21 which acquires user input growth information DS2 which indicates criteria of a user related to the determination as to whether the fruit SF can be harvested; a captured image information acquisition unit 22 which acquires user captured image information DG2 indicating a user captured image GG2 of the fruit SF; and a criteria correction unit 23 which corrects the criteria indicated by the criteria information DH on the basis of the user captured image GG2 indicated by the user captured image information DG2 and the criteria of the user indicated by the user input growth information DS2.SELECTED DRAWING: Figure 9

Description

[Technical Field]

[0001] The present invention relates to a harvesting system, a control method for a harvesting system, and a program. [Background technology]

[0002] In recent years, techniques have become known for determining whether or not fruits and vegetables, such as vegetables and fruits (hereinafter sometimes referred to as "harvest targets"), are suitable for harvesting. For example, Patent Document 1 discloses a technique for determining whether or not a harvest target is suitable for harvesting, based on the color tone of the harvest target shown in an image of the harvest target obtained by imaging the harvest target. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Application Laid-Open No. 2012-088100 Summary of the Invention [Problem to be solved by the invention]

[0004] However, the color tone of a harvest target in a captured image of the harvest target may change depending on the brightness of the surrounding environment of the harvest target, etc. Therefore, with conventional techniques, there is a possibility that the influence of the surrounding environment of the harvest target may make it impossible to appropriately determine whether the harvest target can be harvested.

[0005] The present invention has been made in consideration of the above-mentioned circumstances, and one of the problems to be solved is to more appropriately determine whether or not a harvest target can be harvested, compared to conventional techniques. [Means for solving the problem]

[0006] In order to solve the above problems, the harvesting system of the present invention is characterized by comprising a memory unit that stores judgment criteria information indicating the criteria for determining whether or not a harvest object can be harvested; a criteria information acquisition unit that acquires user criteria information that indicates the user's judgment criteria for whether or not the harvest object can be harvested; an image information acquisition unit that acquires captured image information that indicates an image of the harvest object; and a correction unit that corrects the judgment criteria indicated by the judgment criteria information based on the captured image indicated by the captured image information and the judgment criteria indicated by the user criteria information.

[0007] In addition, the control method for a harvesting system of the present invention is a control method for a harvesting system that has a memory unit that stores judgment criteria information that indicates the criteria for determining whether or not a harvest object can be harvested, and is characterized in that it acquires user criteria information that indicates the user's judgment criteria for whether or not the harvest object can be harvested, acquires captured image information that indicates an image of the harvest object, and corrects the judgment criteria indicated by the judgment criteria information based on the captured image indicated by the captured image information and the judgment criteria indicated by the user criteria information.

[0008] In addition, the program of the present invention is characterized in that it causes a processor to function as a criteria information acquisition unit that acquires user criteria information indicating the user's criteria for whether or not a harvest object can be harvested, an image information acquisition unit that acquires captured image information indicating an image of the harvest object, and a correction unit that corrects the criteria for whether or not the harvest object can be harvested, indicated by the criteria information stored in a memory unit, based on the captured image indicated by the captured image information and the criteria indicated by the user criteria information. [Effects of the Invention]

[0009] According to the present invention, it is possible to more appropriately determine whether or not harvesting is possible for harvest targets, compared to conventional techniques. [Brief explanation of the drawings]

[0010] [Figure 1] FIG. 1 is a block diagram showing an example of the configuration of a plant harvesting system Sys according to an embodiment of the present invention. [Figure 2]FIG. 2 is an explanatory diagram showing an example of the internal environment of an agricultural greenhouse 8. [Figure 3] FIG. 2 is a block diagram showing an example of the configuration of an automatic harvesting device 5. [Figure 4] FIG. 2 is a block diagram showing an example of the configuration of a user terminal device 9. [Figure 5] 1 is a block diagram showing an example of the configuration of a plant harvest management server 1. FIG. [Figure 6] FIG. 10 is a diagram showing an example of the data configuration of determination criterion information DH. [Figure 7] 10 is a flowchart showing an example of a harvest assessment process. [Figure 8] 10 is a flowchart illustrating an example of a determination criterion correction process. [Figure 9] 10 is a flowchart illustrating an example of a determination criterion correction process. [Figure 10] FIG. 10 is an explanatory diagram showing an example of a user criteria information input screen G1. [Figure 11] 10 is a flowchart showing an example of a determination criterion correction process according to Modification 1 of the present invention. [Figure 12] 10 is a flowchart showing an example of a determination criterion correction process according to Modification 1 of the present invention. [Figure 13] FIG. 10 is an explanatory diagram showing an example of a criterion comparison screen G2 according to Modification 1 of the present invention. [Figure 14] FIG. 10 is a block diagram showing an example of the configuration of a plant harvest managing server 1B according to a second modification of the present invention. [Figure 15] 10 is a flowchart showing an example of a criterion selection process according to Modification 2 of the present invention. DETAILED DESCRIPTION OF THE INVENTION

[0011] Hereinafter, embodiments for carrying out the present invention will be described with reference to the drawings. In each figure, the dimensions and scales of each part are appropriately different from the actual ones. Further, the embodiments described below are preferred specific examples of the present invention, and thus various technically preferable limitations are imposed. However, the scope of the present invention is not limited to these embodiments unless there is a description to particularly limit the present invention in the following description.

[0012] <A. Embodiment> Hereinafter, embodiments of the present invention will be described.

[0013] <A.1. Overview of the Plant Harvesting System Sys> Hereinafter, the overview of the plant harvesting system Sys will be described while referring to FIGS. 1 to 6.

[0014] <A.1.1. Overall Overview of the Plant Harvesting System Sys> FIG. 1 is a block diagram showing an example of the outline of the configuration of the plant harvesting system Sys.

[0015] As illustrated in FIG. 1, the plant harvesting system Sys includes a plant harvesting management server 1, an automatic harvesting device 5, and a user terminal device 9. In the present embodiment, the plant harvesting system Sys is an example of a "harvesting system".

[0016] The automatic harvesting device 5 harvests harvest targets such as fruits and vegetables of plants cultivated by the plant harvesting system Sys. The user terminal device 9 sets information regarding the harvesting criteria when the automatic harvesting device 5 harvests the harvest targets of plants cultivated by the plant harvesting system Sys. The plant harvesting management server 1 can communicate with the automatic harvesting device 5 and the user terminal device 9 via the network NW. The plant harvesting management server 1 controls the automatic harvesting device 5. The plant harvesting management server 1 acquires various information from the user terminal device 9 and also causes the user terminal device 9 to display various information.

[0017] In this embodiment, the automatic harvesting device 5 is placed in an agricultural greenhouse 8 shown in Fig. 2. In this embodiment, the plant harvesting system Sys manages the harvesting of harvest targets of plants cultivated in the agricultural greenhouse 8. That is, in this embodiment, as an example, it is assumed that the environment for cultivating plants to be harvested by the plant harvesting system Sys (hereinafter referred to as the "plant cultivation environment") is the environment inside the agricultural greenhouse 8 (i.e., the indoor environment). However, the present invention is not limited to this embodiment. For example, the plant cultivation environment may be a farmland provided outdoors (i.e., an outdoor environment). In this case, the automatic harvesting device 5 is placed in the farmland, which is the plant cultivation environment.

[0018] The agricultural greenhouse 8 is a facility for cultivating plants, such as a plastic greenhouse or a glass greenhouse. Part or all of the exterior walls, such as the side walls and roof, of the agricultural greenhouse 8 are made of a light-transmitting material, such as plastic, vinyl, or glass. The vinyl may be, for example, an agricultural polyolefin film (also called an agricultural PO film) or a film made of fluororesin.

[0019] In addition, in this embodiment, as an example, it is assumed that the plants cultivated in the agricultural greenhouse 8 are strawberries. That is, in this embodiment, as an example, it is assumed that the objects to be harvested by the plant harvesting system Sys are strawberry fruits. However, the present invention is not limited to such an embodiment. The object to be harvested by the plant harvesting system Sys may be a fruit other than strawberries, such as an apple or a melon, an edible part of a vegetable, such as an eggplant part or a corn part, or an ornamental part of an ornamental plant, such as a rose flower or a carnation flower.

[0020] In the present embodiment, as an example, it is assumed that the strawberry variety cultivated in the agricultural greenhouse 8 is "Amaou" (registered trademark). However, the present invention is not limited to such an aspect. The strawberry variety cultivated in the agricultural greenhouse 8 may be "Tochiotome" (registered trademark) or other varieties.

[0021] <A.1.2. Cultivation and Harvest of Strawberries in Agricultural Greenhouse 8> Hereinafter, referring to FIG. 2, an overview of the cultivation and harvest of strawberries in the agricultural greenhouse 8 will be described.

[0022] FIG. 2 is an explanatory diagram for explaining an example of the cultivation and harvest of strawberries in the agricultural greenhouse 8.

[0023] As illustrated in FIG. 2, in the present embodiment, inside the agricultural greenhouse 8, strawberries S are cultivated on the plant cultivation shelf PL. The strawberry S has fruits SF (an example of "harvest target object"), leaves SL, and a stem SK. Hereinafter, among the fruits SF of the strawberry S, the fruits SF that have matured to a degree that can be harvested are referred to as mature fruits SFA. Also, hereinafter, among the fruits SF of the strawberry S, the fruits SF that have not matured to a degree that can be harvested are referred to as immature fruits SFB.

[0024] In the present embodiment, as an example, the plant harvesting system Sys determines whether the strawberry S is a mature fruit SFA or an immature fruit SFB. And in the present embodiment, when there is a strawberry S determined to be a mature fruit SFA, the plant harvesting system Sys harvests the strawberry S by the automatic harvesting device 5.

[0025] <A.1.3. Overview of Automatic Harvesting Device 5> Hereinafter, referring to FIGS. 2 and 3, an overview of the automatic harvesting device 5 will be described.

[0026] FIG. 3 is a functional block diagram showing an example of the outline of the configuration of the automatic harvesting device 5.

[0027] As illustrated in Figures 2 and 3, the automatic harvesting device 5 includes a control device 51, a memory device 52, a traveling device 53, a manipulator 54, an imaging device 55, a detection device 56, and a communication device 57.

[0028] The memory device 52 is configured to include, for example, one or both of a volatile memory such as a RAM (Random Access Memory) that functions as a working area for the control device 51, and a non-volatile memory such as an EEPROM (Electrically Erasable Programmable Read-Only Memory) that stores various information such as the control program of the automatic harvesting device 5.

[0029] The control device 51 is configured to include, for example, a processor, and controls each part of the automatic harvesting device 5. The processor provided in the control device 51 is configured to include, for example, one or more central processing units (CPUs). However, the processor provided in the control device 51 may be configured to include hardware such as a graphics processing unit (GPU), a digital signal processor (DSP), or a field programmable gate array (FPGA) in addition to the one or more CPUs, or in place of some or all of the one or more CPUs. The processor provided in the control device 51 executes a control program for the automatic harvesting device 5 stored in the storage device 52, and controls each part of the automatic harvesting device 5 by operating in accordance with the control program.

[0030] The traveling device 53 includes a moving mechanism including a plurality of tires or leg mechanisms, and a drive mechanism such as a motor for driving the moving mechanism. The traveling device 53 changes the position and posture of the automatic harvesting device 5 by driving the moving mechanism with the drive mechanism under the control of the control device 51.

[0031] The manipulator 54 includes an articulated arm-type robot and an end effector attached to the tip of the arm-type robot. The manipulator 54 operates the arm-type robot and the end effector under the control of the control device 51 to harvest the fruits SF of the strawberries S by cutting the fruits SF from the stems SK. In other words, under the control of the control device 51 based on commands from the plant harvest management server 1, the automatic harvesting device 5 moves to the vicinity of the strawberries S using the traveling device 53 and harvests the fruits SF of the strawberries S using the manipulator 54.

[0032] The imaging device 55, under the control of the control device 51, captures an image of an area including strawberries S grown on the plant cultivation shelf PL. When the imaging device 55 captures an image of an area including strawberries S, it generates device captured image information DG3 indicating a device captured image GG3 obtained as a result of the imaging. The imaging device 55 may be attached to the tip of the manipulator 54.

[0033] The detection device 56 is equipped with an obstacle detection sensor such as an infrared sensor and an ultrasonic sensor. When the obstacle sensor detects the presence of an obstacle in the direction of movement of the automatic harvesting device 5, the control device 51 stops the movement of the automatic harvesting device 5. The detection device 56 may also include a position detection sensor, such as a GPS (Global Positioning System) receiver, an acceleration sensor, and an angular velocity sensor, for detecting the position of the automatic harvesting device 5. The control device 51 may move along a predetermined route inside the agricultural greenhouse 8 based on the detection results of the position detection sensor.

[0034] In addition, in this embodiment, it is assumed that the detection device 56 includes an obstacle detection sensor and a position detection sensor. However, the present invention is not limited to such an aspect. The detection device 56 may include only one of the obstacle detection sensor and the position detection sensor, or may include a sensor other than the obstacle detection sensor and the position detection sensor. Further, in this embodiment, it is assumed that the automatic harvesting device 5 includes the detection device 56. However, the present invention is not limited to such an aspect. The automatic harvesting device 5 may be configured without including the detection device 56.

[0035] The communication device 57 is hardware for communicating with an external device such as the plant harvesting management server 1 existing outside the automatic harvesting device 5 via, for example, the network NW.

[0036] <A.1.4. Overview of the User Terminal Device 9> Hereinafter, the overview of the user terminal device 9 will be described while referring to FIG. 4.

[0037] FIG. 4 is a functional block diagram showing an example of the overview of the configuration of the user terminal device 9.

[0038] As illustrated in FIG. 4, the user terminal device 9 includes a control device 91, a storage device 92, a display device 93, an input device 94, an imaging device 95, and a communication device 96.

[0039] The storage device 92 includes, for example, one or both of a volatile memory such as a RAM that functions as a work area of the control device 91 and a non-volatile memory such as an EEPROM that stores various information such as a control program of the user terminal device 9.

[0040] The control device 91 is configured to include, for example, a processor, and controls each unit of the user terminal device 9. The processor provided in the control device 91 is configured to include, for example, one or more CPUs. However, the processor provided in the control device 91 may be configured to include hardware such as a GPU, DSP, or FPGA in addition to the one or more CPUs, or in place of some or all of the one or more CPUs. The processor provided in the control device 91 executes a control program for the user terminal device 9 stored in the storage device 92, and controls each part of the user terminal device 9 by operating in accordance with the control program.

[0041] The imaging device 95 captures various images under the operation of the user of the user terminal device 9. Specifically, in this embodiment, the imaging device 95 captures an image of an area including strawberries S grown on the plant cultivation shelf PL under the operation of the user of the user terminal device 9. Then, when the imaging device 95 captures an image of the area including the strawberries S, it generates user-captured image information DG2 indicating a user-captured image GG2 obtained as a result of the imaging. In this embodiment, the user-captured image information DG2 is an example of "captured image information," and the user-captured image GG2 is an example of "captured image."

[0042] The input device 94 accepts input of various information from the user of the user terminal device 9. Specifically, in this embodiment, the user of the user terminal device 9 inputs, from the input device 94, a user-input growth degree AS2, which is a value indicating the degree of growth of the strawberry S imaged by the imaging device 95 and is a value indicating the degree of growth of the strawberry S as determined by the user of the user terminal device 9. Then, the input device 94 generates user-input growth degree information DS2 indicating the user-input growth degree AS2 input from the input device 94. In this embodiment, the user input growth level information DS2 is an example of "user standard information", and the user input growth level AS2 is an example of "growth level indicated by the user standard information" and "user judgment standard".

[0043] The display device 93 displays various information related to the strawberries S being cultivated in the greenhouse 8 for agriculture. Specifically, in the present embodiment, the display device 93 displays a user captured image GG2 including an image of the strawberries S captured by the imaging device 95 and a user input growth length AS2 input from the input device 94.

[0044] The communication device 96 is hardware for communicating with external devices such as the plant harvest management server 1 existing outside the user terminal device 9 via the network NW.

[0045] <A.1.5. Overview of the Plant Harvest Management Server 1> Hereinafter, with reference to FIG. 5, the overview of the plant harvest management server 1 will be described.

[0046] FIG. 5 is a functional block diagram showing an example of the outline of the configuration of the plant harvest management server 1.

[0047] As illustrated in FIG. 5, the plant harvest management server 1 includes a control device 2, a storage device 3, and a communication device 4.

[0048] [[ID=...]] The communication device 4 is hardware for communicating with external devices such as the automatic harvest device 5 and the user terminal device 9 existing outside the plant harvest management server 1 via the network NW.

[0049] The storage device 3 is configured to include, for example, one or both of a volatile memory that functions as a work area of the control device 2 and a non-volatile memory such as an EEPROM that stores various information. As illustrated in FIG. 5, the storage device 3 stores determination criterion information DH and a control program PG of the plant harvest management server 1. Note that the determination criterion information DH will be described later.

[0050] The control device 2 is configured to include, for example, a processor, and controls each part of the plant harvest management server 1. The processor provided in the control device 2 is configured to include, for example, one or more CPUs. However, the processor provided in the control device 2 may be configured to include hardware such as a GPU, DSP, or FPGA in addition to the one or more CPUs, or in place of some or all of the one or more CPUs. The processor provided in the control device 2 executes the control program PG of the plant harvest management server 1 stored in the memory device 3 and operates in accordance with the control program PG, thereby operating as a user criteria information acquisition unit 21, a captured image information acquisition unit 22, a judgment criteria correction unit 23, a harvest device control unit 24, a display control unit 25, and a harvest judgment unit 26, and controlling each part of the plant harvest management server 1.

[0051] The user reference information acquisition unit 21 (an example of a “reference information acquisition unit”) acquires the user input growth degree information DS2 generated by the user terminal device 9. The captured image information acquisition unit 22 (an example of an “image information acquisition unit”) acquires user captured image information DG2 generated by the user terminal device 9. The judgment criterion correction unit 23 (an example of a "correction unit") corrects the judgment criterion information DH stored in the storage device 3 based on the user input growth degree information DS2 acquired by the user criterion information acquisition unit 21 and the user captured image information DG2 acquired by the captured image information acquisition unit 22. Note that, hereinafter, the process of correcting the judgment criterion information DH may be referred to as the "judgment criterion correction process."

[0052] The harvester control unit 24 controls the automatic harvester 5. Specifically, the harvester control unit 24 controls the traveling device 53 via the control device 51 by supplying to the control device 51 a control command to control the traveling device 53 so that the automatic harvester 5 moves or stops. The harvester control unit 24 also controls the manipulator 54 via the control device 51 by supplying to the control device 51 a control command to control the manipulator 54 so that the manipulator 54 harvests the fruits SF of the strawberries S. The harvester control unit 24 also controls the imaging device 55 via the control device 51 by supplying to the control device 51 a control command to control the imaging device 55 so that the imaging device 55 captures an image of an area including the strawberries S. In this embodiment, the harvesting device control unit 24 controls the traveling device 53, the manipulator 54, and the imaging device 55 via the control device 51, but the present invention is not limited to this. The harvesting device control unit 24 may directly control some or all of the traveling device 53, the manipulator 54, and the imaging device 55 without going through the control device 51.

[0053] The display control unit 25 controls the display device 93 provided in the user terminal device 9 via the control device 91 so that various images are displayed on the display device 93.

[0054] The harvest determination unit 26 determines, based on determination criteria information DH stored in the storage device 3, whether the strawberry S included in the device captured image GG3 captured by the imaging device 55 is a mature fruit SFA to be harvested by the manipulator 54, or an immature fruit SFB to be harvested by the manipulator 54. If the harvest determination unit 26 determines that the strawberry S included in the device captured image GG3 is a mature fruit SFA, the harvesting device control unit 24 causes the manipulator 54 to harvest the strawberry S. If the harvest determination unit 26 determines that the strawberry S included in the device captured image GG3 is an immature fruit SFB, the harvesting device control unit 24 stops the manipulator 54 from harvesting the strawberry S. Hereinafter, the process of determining whether the strawberry S included in the device captured image GG3 is a mature fruit SFA will be referred to as the "harvest determination process".

[0055] <A.1.6. Outline of the determination criterion information DH> Hereinafter, while referring to FIG. 6, the outline of the determination criterion information DH will be described. Here, the determination criterion information DH is information indicating the determination criteria for determining whether the strawberry S included in the device captured image GG3 captured by the imaging device 55 is a mature fruit SFA that is a harvest target by the manipulator 54 or an immature fruit SFB that is not a harvest target by the manipulator 54.

[0056] FIG. 6 is a diagram showing an example of the data configuration of the determination criterion information DH.

[0057] As illustrated in FIG. 6, the determination criterion information DH has a plurality of records that correspond one-to-one with the growth degrees of multiple stages of the strawberry S. Each record of the determination criterion information DH has a determination criterion ID and individual determination criterion information DK.

[0058] In this embodiment, as an example, it is assumed that the plant harvest management server 1 manages the growth degree of the strawberry S in "10 stages of growth degree" in increments of "10%" from the growth degree of "10%" to the growth degree of "100%". However, the present invention is not limited to such a mode. The plant harvest management server 1 only needs to manage the growth degree of the strawberry S in "two or more stages of growth degree". For example, the plant harvest management server 1 may manage the growth degree of the strawberry S in "5 stages of growth degree" in increments of "10%" from "60%" to "100%".

[0059] Furthermore, in this embodiment, it is assumed that the growth level at which a strawberry S can be harvested is the harvestable growth level AS0, and when the growth level of the strawberry S is equal to or greater than the harvestable growth level AS0, the harvest assessment unit 26 assesses the strawberry S as a mature fruit SFA, and when the growth level of the strawberry S is less than the harvestable growth level AS0, the harvest assessment unit 26 assesses the strawberry S as an immature fruit SFB. In this embodiment, as an example, it is assumed that the harvestable growth level AS0 is a fixed value of "80%". However, the present invention is not limited to this aspect. The growth level required for a strawberry S to become a mature fruit SFA may be set, for example, by a user of the user terminal device 9.

[0060] The judgment criterion ID is information for uniquely identifying each growth stage among the multiple stages (10 stages in this embodiment) of growth stage of the strawberries S managed by the plant harvest management server 1.

[0061] The individual determination criterion information DK (an example of "individual criterion information") includes server-managed growth level information DS1, reference image information DG1, and reference image color tone information DT1.

[0062] The server-managed growth level information DS1 is information indicating a server-managed growth level AS1, which is the growth level of the strawberry S corresponding to the judgment criterion ID. The reference image information DG1 is information indicating the reference image GG1, which is a standard image of a strawberry S having a growth degree corresponding to the judgment criterion ID. The reference image color tone information DT1 is information indicating the reference image color tone AT1, which is the color tone of the reference image GG1 of the strawberry S having the growth degree corresponding to the judgment criterion ID.

[0063] In this embodiment, as an example, it is assumed that the reference image color tone AT1 of the strawberry S is expressed as the average value of the color tones of the area corresponding to the strawberry S in the image in which the strawberry S is depicted. However, the present invention is not limited to this aspect. For example, the reference image color tone AT1 of the strawberry S may be the color tone of one or more feature points located in the area corresponding to the strawberry S in the image in which the strawberry S is depicted.

[0064] In this embodiment, as an example, it is assumed that the reference image color tone AT1 of the strawberry S is expressed as three-dimensional information in the HSV color space. However, the present invention is not limited to this aspect. For example, the reference image color tone AT1 of the strawberry S may be expressed in a format other than the HSV color space, such as the Lab color space or the RGB (red-green-blue) color space. Furthermore, for example, the reference image color tone AT1 of the strawberry S may be expressed by a one-dimensional parameter, such as the degree of redness in the color tone of the area corresponding to the strawberry S in the image in which the strawberry S is represented.

[0065] The reference image color tone information DT1 includes reference image hue information DT11, reference image saturation information DT12, and reference image lightness information DT13.

[0066] The reference image hue information DT11 indicates the hue of the reference image GG1 of the strawberry S having the growth degree corresponding to the judgment criterion ID, i.e., the reference image hue AT11, which is a value corresponding to the color type of the reference image GG1. In this embodiment, as an example, it is assumed that the reference image hue AT11 indicates a value in the range from "0%" to "100%." The reference image saturation information DT12 indicates a reference image saturation AT12, which is a value corresponding to the color saturation of the reference image GG1 of strawberry S having a growth degree corresponding to the judgment criterion ID. In this embodiment, as an example, it is assumed that the reference image saturation AT12 indicates a value in the range from "0%" to "100%." The reference image brightness information DT13 indicates a reference image brightness AT13, which is a value corresponding to the brightness (Value) of the color of the reference image GG1 of the strawberry S having the growth degree corresponding to the judgment criterion ID. In this embodiment, as an example, it is assumed that the reference image brightness AT13 indicates a value in the range from "0%" to "100%." That is, the reference image color tone AT1 indicated by the reference image color tone information DT1 includes three values: a reference image hue AT11, a reference image saturation AT12, and a reference image lightness AT13.

[0067] In addition, in this embodiment, based on the user captured image information DG2 acquired by the imaging image information acquisition unit 22, the control device 2 generates user captured image color tone information DT2 indicating a user captured image color tone AT2, which is the color tone of the user captured image GG2 indicated by the user captured image information DG2. The user captured image color tone information DT2 includes user captured image hue information DT21 indicating a user captured image hue AT21, which is the hue of the user captured image GG2, user captured image saturation information DT22 indicating a user captured image saturation AT22, which is the saturation of the user captured image GG2, and user captured image lightness information DT23 indicating a user captured image lightness AT23, which is the lightness of the user captured image GG2. In this embodiment, it is assumed that each of the user captured image hue AT21, the user captured image saturation AT22, and the user captured image lightness AT23 indicates a value in the range from "0%" to "100%" in the same manner as the reference image color tone AT1.

[0068] Also, in this embodiment, based on the device captured image information DG3 acquired by the harvesting device control unit 24, the control device 2 generates device captured image color tone information DT3 indicating a device captured image color tone AT3, which is the color tone of the device captured image GG3 indicated by the device captured image information DG3. The device captured image color tone information DT3 includes device captured image hue information DT31 indicating a device captured image hue AT31, which is the hue of the device captured image GG3, device captured image saturation information DT32 indicating a device captured image saturation AT32, which is the saturation of the device captured image GG3, and device captured image lightness information DT33 indicating a device captured image lightness AT33, which is the lightness of the device captured image GG3. In this embodiment, it is assumed that each of the device captured image hue AT31, the device captured image saturation AT32, and the device captured image lightness AT33 indicates a value in the range from "0%" to "100%" in the same manner as the reference image color tone AT1.

[0069] <A.2. Operations of the Plant Harvest Management Server 1> The following describes the outline of the operations of the plant harvest management server 1.

[0070] <A.2.1. Harvest Judgment Process> FIG. 7 is a diagram showing an example of an outline of the operation of the plant harvest managing server 1 when the harvest assessment process is executed. Here, the harvest determination process is a process of determining whether or not the strawberry S included in the device-captured image GG3 captured by the imaging device 55 is a mature fruit SFA, as described above.

[0071] As illustrated in FIG. 7, when the harvest assessment process is started, the harvester control unit 24 acquires device captured image information DG3 from the automatic harvester 5 (S11).

[0072] Next, the harvest judgment unit 26 generates device captured image color tone information DT3 indicating device captured image color tone AT3 based on the device captured image information DG3 acquired by the harvest device control unit 24 in step S11 (S13).

[0073] Next, the harvest assessment unit 26 generates color tone difference information DVR indicating the color tone difference VR for each of the multiple individual assessment criteria information DK included in the assessment criteria information DH stored in the memory device 3, based on the reference image color tone information DT1 included in the individual assessment criteria information DK and the device-captured image color tone information DT3 generated in step S13 (S15).

[0074] Specifically, in step S15, the harvest assessment unit 26 calculates, as the hue difference VR, a value indicating the degree of difference between the reference image hue AT1 indicated by the reference image hue information DT1 and the device-captured image hue AT3 indicated by the device-captured image hue information DT3. More specifically, in step S15, the harvest assessment unit 26 calculates, as the hue difference VR, the distance between the coordinate in the HSV color space corresponding to the reference image hue AT1 indicated by the reference image hue information DT1 and the coordinate in the HSV color space corresponding to the device-captured image hue AT3 indicated by the device-captured image hue information DT3. That is, in step S15, the harvest assessment unit 26 calculates the hue difference VR based on the following equation (1). The value VRS appearing in equation (1) is expressed by the following equation (2). VR = {VRS} 1 / 2 … (1) VRS = (AT11 - AT31) 2 +(AT12 - AT32) 2 +(AT12 - AT32) 2 … (2)

[0075] Next, the harvest determination unit 26 selects the individual determination criterion information DK in the determination criterion ID having the reference image color tone information DT1 with the minimum color tone difference VR calculated in step S15 from among the plurality of individual determination criterion information DK included in the determination criterion information DH (S17).

[0076] Next, the harvest determination unit 26 specifies the server - managed growth length AS1 indicated by the server - managed growth length information DS1 included in the individual determination criterion information DK selected in step S17 (S19).

[0077] Next, the harvest determination unit 26 determines whether or not the server - managed growth length AS1 specified in step S19 is greater than or equal to the harvest - possible growth length AS0 (S21).

[0078] If the result of the determination in step S21 is negative, that is, if the server - managed growth length AS1 is greater than or equal to the harvest - possible growth length AS0, the harvest determination unit 26 determines that the strawberry S included in the device - captured image GG3 indicated by the device - captured image information DG3 acquired in step S11 is a mature fruit SFA (S23), and ends the harvest determination process. On the other hand, if the result of the determination in step S21 is negative, that is, if the server - managed growth length AS1 is less than the harvest - possible growth length AS0, the harvest determination unit 26 determines that the strawberry S included in the device - captured image GG3 indicated by the device - captured image information DG3 acquired in step S11 is an immature fruit SFB (S25), and ends the harvest determination process.

[0079] As described above, in the harvest determination process, when it is determined that the strawberry S is a mature fruit SFA, the harvest device control unit 24 causes the manipulator 54 to harvest the strawberry S.

[0080] <A.2.2. Determination Criterion Correction Process> 8 and 9 are diagrams showing an example of an outline of the operation of the plant harvest management server 1 when the judgment criterion correction process is executed. The user can start the judgment criterion correction process by pressing the "judgment criterion correction process" button displayed on the display device 93 of the user terminal device 9. Here, the judgment criteria correction process is, as described above, a process of correcting the judgment criteria information DH stored in the storage device 3 based on the user input growth information DS2 acquired by the user criteria information acquisition unit 21 and the user captured image information DG2 acquired by the captured image information acquisition unit 22.

[0081] As illustrated in FIG. 8, when the criterion correction process is started, the display control unit 25 supplies the display information HG1 for displaying the user criterion information input screen G1 to the user terminal device 9 (S101).

[0082] FIG. 10 shows an example of a user reference information input screen G1 displayed on the user terminal device 9 based on the display information HG1 supplied from the plant harvest management server 1.

[0083] As shown in FIG. 10, the user reference information input screen G1 includes a user captured image display area Ar11, a user input growth degree input area Ar12, and a confirm button BT1.

[0084] The user-captured image display area Ar11 is an area where a user-captured image GG2 obtained as a result of capturing an image of an area including strawberries S using the imaging device 95 is displayed when the user of the user terminal device 9 captures the image of the area including strawberries S. In the user-captured image GG2 displayed in the user-captured image display area Ar11, the area where the strawberries S exist may be highlighted by, for example, a dashed line Ar111. The user-input growth degree input area Ar12 is an area where the user of the user terminal device 9 uses the input device 94 to input the user-input growth degree AS2.

[0085] The decision button BT1 is a button for supplying user-captured image information DG2 and user-input growth degree information DS2 from the user terminal device 9 to the plant harvest managing server 1. Specifically, when the user of the user terminal device 9 presses the decision button BT1, user-captured image information DG2 indicating the user-captured image GG2 displayed in the user-captured image display area Ar11 is supplied from the user terminal device 9 to the plant harvest managing server 1, and user-input growth degree information DS2 indicating the user-input growth degree AS2 input from the user-input growth degree input area Ar12 is supplied from the user terminal device 9 to the plant harvest managing server 1.

[0086] Returning to the explanation of FIG. As illustrated in FIG. 8, when the enter button BT1 is pressed on the user terminal device 9, the captured image information acquisition unit 22 acquires the user captured image information DG2 supplied from the user terminal device 9 (S103). Furthermore, when the enter button BT1 is pressed on the user terminal device 9, the user reference information acquiring unit 21 acquires the user input growth degree information DS2 supplied from the user terminal device 9 (S105).

[0087] Next, the control device 2 determines whether or not there are any abnormalities in the user-captured image GG2 indicated by the user-captured image information DG2 acquired in step S103 and the user input growth degree AS2 indicated by the user input growth degree information DS2 acquired in step S105 (S107). Specifically, in step S107, for example, if the user-captured image GG2 includes an image of strawberries S, the control device 2 determines that the user-captured image GG2 is normal, and if the user-captured image GG2 does not include an image of strawberries S, the control device 2 determines that an abnormality exists in the user-captured image GG2. Also, in step S107, for example, if the user-input growth rate AS2 is an appropriate value (for example, a value in 10% increments from 10% to 100%), the control device 2 determines that the user-input growth rate AS2 is normal, and if the user-input growth rate AS2 includes a value other than the appropriate value, the control device 2 determines that an abnormality exists in the user-input growth rate AS2.

[0088] If the result of the determination in step S107 is negative, that is, if an abnormality exists in the user-captured image GG2 or the user-input growth degree AS2, the control device 2 returns the process to step S103. On the other hand, if the result of the judgment in step S107 is positive, that is, if the user-captured image GG2 and the user input growth rate AS2 are normal, the judgment criterion correction unit 23 selects, from the multiple individual judgment criterion information DK included in the judgment criterion information DH, individual judgment criterion information DK for the judgment criterion ID including server management growth rate information DS1 indicating server management growth rate AS1 that is approximate to the user input growth rate AS2 indicated by the user input growth rate information DS2 acquired in step S105 (S109).

[0089] Next, as illustrated in FIG. 9, the determination criterion corrector 23 generates user-captured image color tone information DT2 indicating user-captured image color tone AT2 based on the user-captured image information DG2 acquired in step S103 (S111).

[0090] Next, the judgment criterion correction unit 23 generates color tone correction amount information DdT indicating the color tone correction amount dT based on the reference image color tone information DT1 included in the individual judgment criterion information DK selected in step S109 and the user-captured image color tone information DT2 generated in step S111 (S113).

[0091] Specifically, in step S113, the criterion corrector 23 calculates a color tone correction amount dT as a correction value for correcting the reference image color tone AT1 indicated by the reference image color tone information DT1 included in the individual criterion information DK selected in step S109 to the user-captured image color tone AT2 indicated by the user-captured image color tone information DT2 generated in step S111, thereby generating color tone correction amount information DdT indicating the calculated color tone correction amount dT. More specifically, in step S113, the criterion corrector 23 generates the color tone correction amount dT, for example, as shown in the following equation (3). Note that in this embodiment, the color tone correction amount dT is a three-dimensional vector in the HSV color space. Below, each component of the color tone correction amount dT may also be expressed by the following equation (4). dT = AT2-AT1 = (AT21-AT11, AT22-AT12, AT23-AT13) … (3) dT = (dT1, dT2, dT3) … (4)

[0092] Next, the judgment criterion correction unit 23 corrects the reference image information DG1 and the reference image color tone information DT1 for each of the multiple individual judgment criterion information DK corresponding to the multiple judgment criterion IDs included in the judgment criterion information DH based on the color tone correction amount information DdT generated in step S113 (S115), and terminates the judgment criterion correction process.

[0093] Specifically, in step S115, the judgment criterion correction unit 23 corrects the reference image color tone AT1 indicated by the reference image color tone information DT1 held by each of the multiple individual judgment criterion information DK included in the judgment criterion information DH based on the color tone correction amount dT generated in step S113. More specifically, in step S115, the judgment criterion correction unit 23 corrects the reference image color tone AT1, for example, based on the following equation (5). Note that in equation (5), the reference image color tone AT1x is the corrected reference image color tone AT1. Also, in equation (5), the reference image color tone AT1 is the reference image color tone AT1 indicated by the reference image color tone information DT1 held by each of the multiple individual judgment criterion information DK included in the judgment criterion information DH. AT1x = AT1 + dT = (AT11 + dT1, AT12 + dT2, AT13 + dT3) … (5) Note that the values of the color tone correction amounts dT (dT1, dT2, dT3) may be multiplied by coefficients for each determination criterion ID.

[0094] Further, in step S115, the determination criterion correction unit 23 corrects the reference image GG1 indicated by the reference image information DG1 included in each of the plurality of individual determination criterion information DK included in the determination criterion information DH based on the corrected reference image color tone AT1 (reference image color tone AT1x). Specifically, in step S115, the determination criterion correction unit 23 corrects the reference image GG1 so that the color tone of the reference image GG1 becomes the corrected reference image color tone AT1x.

[0095] <A.3. Conclusion of the embodiment> As described above, the plant harvesting system Sys according to the embodiment corrects the determination criterion information DH indicating the determination criterion for whether or not to harvest the fruit SF by using the user imaging image information DG2 and the user input growth degree information DS2 sent from the user terminal device 9. Therefore, according to the present embodiment, for example, the determination criterion information DH corrected in consideration of the location and sunlight of the agricultural house 8, the type and illuminance of the lighting provided in the agricultural house 8, and the time zone and weather when harvesting the fruit SF in the agricultural house 8, etc., it is possible to determine whether or not to harvest the fruit SF. That is, according to the present embodiment, even when there is a difference between the surrounding environment of the fruit SF assumed by the generator of the determination criterion information DH when the determination criterion information DH is generated and the environment in the agricultural house 8 where the fruit SF is actually cultivated, and the environment in the agricultural house 8 at the timing when the fruit SF is harvested, it is possible to correct the determination criterion indicated by the determination criterion information DH to a determination criterion corresponding to the actual environment in the agricultural house 8. Therefore, compared with the mode in which such correction is not performed, it is possible to appropriately determine whether or not to harvest the fruit SF.

[0096] Further, according to the present embodiment, in order to correct the determination criterion information DH based on the user imaging image information DG2 and the user input growth degree information DS2 sent from the user terminal device 9, compared with the mode without performing such correction, it is possible to set the determination criterion for whether the fruit SF can be harvested as a criterion according to the needs of the user of the user terminal device 9. Therefore, according to the present embodiment, for example, it is possible to more flexibly determine whether the fruit SF can be harvested based on the determination criterion according to the needs of the user of the user terminal device 9, such as the shipping status of the fruit SF.

[0097] <B. Variation Example> Each of the above embodiments can be variously modified. Specific modification modes are exemplified below. Two or more modes arbitrarily selected from the following examples can be appropriately combined within a range where they do not conflict with each other. In the variation examples exemplified below, for elements whose actions and functions are equivalent to those of the embodiment, the reference numerals referred to in the above description are used, and the detailed description of each is appropriately omitted.

[0098] <B.1. Variation Example 1> In the above-described embodiment, the control device 2 corrects the reference image information DG1 indicating the reference image GG1 and the reference image color tone information DT1 indicating the reference image color tone AT1 among the determination criterion information DH based on the user imaging image information DG2 and the user input growth degree information DS2 sent from the user terminal device 9, and has been described by way of example. However, the present invention is not limited to such a mode. For example, the control device 2 may correct the server management growth degree information DS1 indicating the server management growth degree AS1 among the determination criterion information DH based on the user imaging image information DG2 and the user input growth degree information DS2 sent from the user terminal device 9.

[0099] FIGS. 11 and 12 are diagrams showing an example of an outline of the operation of the plant harvest management server 1 when the determination criterion correction process according to this variation example is executed.

[0100] As illustrated in FIG. 11, when the determination criterion correction process is started, the control device 2 executes the processes of steps S101 to S107 described above.

[0101] If the result of the judgment in step S107 is positive, that is, if the user-captured image GG2 and the user-input growth rate AS2 are normal, the judgment criterion correction unit 23 selects, from the multiple individual judgment criterion information DK included in the judgment criterion information DH, individual judgment criterion information DK for the judgment criterion ID including reference image information DG1 indicating a reference image GG1 that is similar to the user-captured image GG2 indicated by the user-captured image information DG2 acquired in step S103 (S201).

[0102] Specifically, in step S201, the criterion correction unit 23 generates, for example, a user-captured image color tone AT2 from the user-captured image GG2, and identifies a reference image color tone AT1 that is similar to the generated user-captured image color tone AT2, thereby selecting individual criterion information DK indicating a reference image GG1 that is similar to the user-captured image GG2. However, the present invention is not limited to this embodiment. In step S201, the criterion correction unit 23 may select, for example, individual criterion information DK indicating a reference image GG1 that is similar to the user-captured image GG2, using a learning model for recognizing an image displaying fruit SF.

[0103] Next, the display control unit 25 supplies the display information HG2 for displaying the criterion comparison screen G2 to the user terminal device 9 (S203).

[0104] FIG. 13 shows an example of a criterion comparison screen G2 displayed on the user terminal device 9 based on the display information HG2 supplied from the plant harvest management server 1.

[0105] As illustrated in Figure 13, the judgment criteria comparison screen G2 includes a user-captured image display area Ar21, a user-input growth level display area Ar22, a pre-correction judgment criteria display area Ar23, a post-correction judgment criteria display area Ar24, an update button BT21, and a reset button BT22.

[0106] The user-captured image display area Ar21 is an area where the user-captured image GG2 indicated by the user-captured image information DG2 acquired in step S103 is displayed. Note that in the user-captured image GG2 displayed in the user-captured image display area Ar21, the area where the strawberries S exist may be highlighted by, for example, a dashed line Ar211. The user input growth degree display area Ar22 is an area where the user input growth degree AS2 indicated by the user input growth degree information DS2 acquired in step S105 is displayed.

[0107] The pre-correction criterion display area Ar23 is an area for displaying a plurality of reference images GG1 indicated by a plurality of reference image information DG1 held by a plurality of individual criterion information DK included in the criterion information DH stored in the storage device 3, and a plurality of server management growth degrees AS1 indicated by a plurality of server management growth degree information DS1 held by the plurality of individual criterion information DK. For example, Fig. 13 illustrates a case where the reference image GG1 corresponding to the server management growth degree AS1 of "90%" is displayed in a display area Ar231 of the pre-correction criterion display area Ar23, and the fact that the server management growth degree AS1 is "90%" is displayed in a display area Ar232 corresponding to the display area Ar231 of the pre-correction criterion display area Ar23.

[0108] The corrected criterion display area Ar24 has a display area Ar241 and a display area Ar242. The display area Ar241 is an area where the user-captured image GG2 indicated by the user-captured image information DG2 acquired in step S103 is displayed. The display area Ar241 is provided at a position corresponding to the display area Ar231 where the reference image GG1 similar to the user-captured image GG2 displayed in the display area Ar241 is displayed. Display area Ar242 is an area where the user input growth degree AS2 indicated by the user input growth degree information DS2 acquired in step S105 is displayed. Display area Ar242 is provided at a position corresponding to display area Ar232. For example, Fig. 13 illustrates a case where the server management growth degree AS1 is displayed in display area Ar232 as "90%" and the user input growth degree AS2 is displayed as "80%" in display area Ar242.

[0109] The update button BT21 and the reset button BT22 are buttons for supplying judgment criterion change permission information DP from the user terminal device 9 to the plant harvest management server 1. The judgment criterion change permission information DP is information indicating whether or not it is permitted to change the judgment criterion shown in the pre-correction judgment criterion display area Ar23 to the judgment criterion shown in the corrected judgment criterion display area Ar24. When a user of the user terminal device 9 presses the update button BT21, judgment criterion change permission information DP indicating the value "1" representing that the judgment criterion will be changed is supplied from the user terminal device 9 to the plant harvest management server 1. When a user of the user terminal device 9 presses the reset button BT22, judgment criterion change permission information DP indicating the value "0" representing that the judgment criterion will be reset is supplied from the user terminal device 9 to the plant harvest management server 1.

[0110] Returning to the explanation of FIG. As illustrated in FIG. 11, when the update button BT21 or the reset button BT22 is pressed on the user terminal device 9, the criterion corrector 23 acquires the criterion change permission information DP supplied from the user terminal device 9 (S205).

[0111] Next, the criterion corrector 23 determines whether the criterion change permission information DP acquired in step S205 indicates the value "1" indicating that the criterion is to be changed (S207).

[0112] If the result of the determination in step S207 is negative, that is, if the determination criterion change permission information DP indicates the value "0" indicating that the determination criterion is to be reset, the determination criterion corrector 23 returns the process to step S103. On the other hand, if the result of the determination in step S207 is positive, that is, if the determination criterion change permission information DP indicates the value "1" indicating that the determination criterion is to be changed, the determination criterion corrector 23 advances the process to step S209.

[0113] As illustrated in Figure 12, the judgment criterion correction unit 23 generates growth correction amount information DdS indicating the growth correction amount dS based on the server management growth information DS1 included in the individual judgment criterion information DK selected in step S201 and the user input growth information DS2 acquired in step S105 (S209).

[0114] Specifically, in step S209, the criterion correction unit 23 calculates a growth degree correction amount dS as a correction value for correcting the server-managed growth degree AS1 indicated by the server-managed growth degree information DS1 included in the individual criterion information DK selected in step S201 to the user-input growth degree AS2 indicated by the user-input growth degree information DS2 acquired in step S105, thereby generating growth degree correction amount information DdS indicating the calculated growth degree correction amount dS. More specifically, in step S209, the criterion correction unit 23 calculates the growth degree correction amount dS by, for example, subtracting the server-managed growth degree AS1 indicated by the server-managed growth degree information DS1 included in the individual criterion information DK selected in step S201 from the user-input growth degree AS2 indicated by the user-input growth degree information DS2 acquired in step S105.

[0115] Next, the judgment criterion correction unit 23 corrects the server management growth level information DS1 for each of the multiple individual judgment criterion information DK included in the judgment criterion information DH based on the growth level correction amount information DdS generated in step S209 (S211), and terminates the judgment criterion correction process.

[0116] Specifically, in step S211, the determination criterion correction unit 23 corrects the server management growth degree AS1 indicated by the server management growth degree information DS1 included in each of the plurality of individual determination criterion information DK included in the determination criterion information DH based on the growth degree correction amount dS generated in step S209. More specifically, in step S211, the determination criterion correction unit 23, for example, adds the growth degree correction amount dS generated in step S209 to the server management growth degree AS1 indicated by the server management growth degree information DS1 included in each of the plurality of individual determination criterion information DK included in the determination criterion information DH, thereby correcting the server management growth degree AS1.

[0117] As described above, the plant harvesting system Sys according to this modification example corrects the determination criterion information DH indicating the determination criterion for whether the fruit SF can be harvested by using the user imaging image information DG2 and the user input growth degree information DS2 sent from the user terminal device 9. Therefore, according to this modification example, it is possible to appropriately determine whether the fruit SF can be harvested as compared with the mode without performing such correction. Further, according to this modification example, it is possible to set the determination criterion for whether the fruit SF can be harvested as a criterion according to the needs of the user of the user terminal device 9 as compared with the mode without performing such correction.

[0118] <B.2. Modification Example 2> In the above-described embodiment and modification example 1, an example has been described in which the control device 2 sets the determination criterion information DH suitable for harvesting the fruit SF by correcting the determination criterion information DH stored in the storage device 3. However, the present invention is not limited to such a mode. For example, the control device 2 may set the determination criterion information DH suitable for harvesting the fruit SF by selecting one determination criterion information DH from among the plurality of determination criterion information DH stored in the storage device 3.

[0119] FIG. 14 is a functional block diagram showing an example of the outline of the configuration of the plant harvesting management server 1B according to this modification example.

[0120] As illustrated in Figure 14, the plant harvest management server 1B differs from the plant harvest management server 1 illustrated in Figure 5 in that it has a control device 2B instead of the control device 2 and a memory device 3B instead of the memory device 3.

[0121] The control device 2B differs from the control device 2 in that it includes a criterion selection unit 23B instead of the criterion correction unit 23. The storage device 3B differs from the storage device 3 in that it stores set criterion information DW instead of judgment criterion information DH, and in that it stores a control program PG-B instead of the control program PG.

[0122] The collection reference information DW includes multiple pieces of judgment reference information DH. Here, each of the multiple pieces of judgment reference information DH indicates a judgment criterion for determining whether or not the fruit SF can be harvested in each of multiple agricultural greenhouses having different environments. For example, the collection reference information DW may include 27 pieces of judgment reference information DH corresponding to all combinations (27 combinations) of the following: three types of weather around the agricultural greenhouse ("sunny," "cloudy," and "rainy"); three types of lighting installed inside the agricultural greenhouse ("red light," "blue light," and "white light"); and three types of time periods for harvesting the fruit SF in the agricultural greenhouse ("daytime," "evening," and "nighttime."

[0123] Control device 2B is configured to include a processor. The processor provided in control device 2B executes control program PG-B of plant harvest management server 1B stored in storage device 3B, and operates in accordance with control program PG-B to operate as user criteria information acquisition unit 21, captured image information acquisition unit 22, judgment criteria selection unit 23B, harvesting device control unit 24, display control unit 25, and harvest judgment unit 26, and controls each unit of plant harvest management server 1B.

[0124] The judgment criterion selection unit 23B (an example of a "selection unit") selects one piece of judgment criterion information DH from the plurality of pieces of judgment criterion information DH included in the set reference information DW based on the user-captured image information DG2 and the user-input growth degree information DS2. Hereinafter, the process of selecting one piece of judgment criterion information DH from the plurality of pieces of judgment criterion information DH included in the set reference information DW will be referred to as the "criterion selection process."

[0125] FIG. 15 is a diagram showing an example of an outline of the operation of the plant harvest managing server 1B when the determination criterion selection process is executed.

[0126] As shown in FIG. 15, when the determination criterion selection process is started, the control device 2B executes the processes of steps S101 to S107 described above.

[0127] If the result of the judgment in step S107 is positive, that is, if the user-captured image GG2 and the user-input growth rate AS2 are normal, the judgment criterion selection unit 23B selects one judgment criterion information DH from the multiple judgment criterion information DH included in the set criterion information DW (S301) and terminates the judgment criterion selection process.

[0128] Specifically, in step S301, the judgment criterion selection unit 23B selects judgment criterion information DH from the multiple judgment criterion information DH included in the collective criterion information DW, which includes individual judgment criterion information DK that has reference image information DG1 indicating a reference image GG1 that is similar to the user-captured image GG2 indicated by the user-captured image information DG2 acquired in step S103, and has server-managed growth level information DS1 indicating a server-managed growth level AS1 that is similar to the user-input growth level AS2 indicated by the user-input growth level information DS2 acquired in step S105.

[0129] More specifically, in step S301, the determination criterion selection unit 23B first selects one or more individual determination criterion information DK corresponding to one or more determination criterion IDs having server management growth degree information DS1 indicating a server management growth degree AS1 approximated to the user input growth degree AS2 indicated by the user input growth degree information DS2 acquired in step S105 from among a plurality of individual determination criterion information DK included in the collective criterion information DW. Next, in step S301, the determination criterion selection unit 23B selects one individual determination criterion information DK having criterion image information DG1 indicating a criterion image GG1 with the highest degree of approximation to the user captured image GG2 indicated by the user captured image information DG2 acquired in step S103 from among the selected one or more individual determination criterion information DK. Next, in step S301, the determination criterion selection unit 23B determines the determination criterion information DH to be used in the harvest determination process as one determination criterion information DH to which the selected one individual determination criterion information DK belongs.

[0130] As described above, the plant harvest system according to this modification example selects the determination criterion information DH indicating the determination criterion for whether the fruit SF can be harvested based on the user captured image information DG2 and the user input growth degree information DS2 sent from the user terminal device 9. Therefore, according to this modification example, it is possible to appropriately determine whether the fruit SF can be harvested as compared with the mode in which the determination criterion information DH is predetermined. Further, according to this modification example, it is possible to set the determination criterion for whether the fruit SF can be harvested as a criterion according to the needs of the user of the user terminal device 9 as compared with the mode in which the determination criterion information DH is predetermined.

[0131] <B.3. Modification Example 3> In the above-described modification example 2, the mode in which the control device 2B selects one determination criterion information DH from among a plurality of determination criterion information DH included in the collective criterion information DW has been exemplified and described, but the present invention is not limited to such a mode. One determination criterion information DH may be selected by the user of the user terminal device 9 from among a plurality of determination criterion information DH included in the collective criterion information DW.

[0132] For example, based on information regarding the surrounding environment of the fruit SF corresponding to each of a plurality of determination criterion information DH included in the collective criterion information DW, such as the weather around the agricultural greenhouse, the type of lighting provided inside the agricultural greenhouse, and the time zone when harvesting the fruit SF in the agricultural greenhouse, etc., one determination criterion information DH may be selected by the user of the user terminal device 9 from among the plurality of determination criterion information DH included in the collective criterion information DW. In this case, a plurality of aspects of the surrounding environment of the fruit SF corresponding to the plurality of determination criterion information DH included in the collective criterion information DW may be displayed on the display device 93 of the user terminal device 9. Then, the user of the user terminal device 9 may select one determination criterion information DH from among the plurality of determination criterion information DH included in the collective criterion information DW by selecting the aspect that is closest to the actual surrounding environment of the agricultural greenhouse 8 from among the plurality of aspects of the surrounding environment of the fruit SF displayed on the display device 93.

[0133] Also, for example, the display device 93 may display a combination of a reference image GG1 and a server-managed growth rate AS1 indicated by each of a plurality of individual determination criterion information DK each possessed by each of the plurality of determination criterion information DH included in the collective criterion information DW. In this case, the user of the user terminal device 9 may select one determination criterion information DH corresponding to the one combination by selecting one combination from among the plurality of combinations of the reference image GG1 and the server-managed growth rate AS1 shown on the display device 93, where the individual determination criterion information DK belonging to the one combination is selected.

[0134] <B.4. Modified Example 4> In the above-described embodiments and Modification Example 1, the control device 2 may store, in the storage device 3, information related to the surrounding environment of the agricultural house 8 when the determination criterion correction process is executed, in association with the corrected determination criterion information DH. In this case, the user of the user terminal device 9 may input information related to the current surrounding environment of the agricultural house 8 from the user terminal device 9 and supply the information to the plant harvesting management server 1, thereby selecting the determination criterion information DH corresponding to the current surrounding environment of the agricultural house 8. Thereby, according to this modification example, it becomes possible to perform the harvest determination process using the determination criterion information DH corresponding to the current surrounding environment of the agricultural house 8.

[0135] <B.5. Modification Example 5> In the above-described embodiments and Modification Examples 1 to 4, the plant harvesting system Sys may display, on the display device 93, the reference image GG1 indicated by each of the plurality of individual determination criterion information DK included in the determination criterion information DH stored in the storage device 3 or the storage device 3B and the server management growth degree AS1. In this case, the plant harvesting system Sys may determine whether or not to apply the determination criterion information DH displayed on the display device 93 as the determination criterion information DH used in the harvest determination process, based on the determination result of the user of the user terminal device 9.

[0136] <B.6. Modification Example 6> In the above-described embodiments and Modification Examples 1 to 5, the plant harvesting system Sys may display, on the display device 93, the reference image GG1 indicated by each of the plurality of individual determination criterion information DK included in the determination criterion information DH stored in the storage device 3 or the storage device 3B and the server management growth degree AS1. Then, the plant harvesting system Sys may correct the color tone of the reference image GG1 displayed on the display device 93 based on the operation of the input device 94 by the user of the user terminal device 9.

[0137] <B.7. Modification Example 7> In the above-described embodiments and Modifications 1 to 6, the mode in which the control device 2 or the control device 2B is provided in the plant harvesting management server 1 or the plant harvesting management server 1B has been described by way of example. However, the present invention is not limited to such a mode. A part of the functions provided in the plant harvesting management server 1 or the plant harvesting management server 1B may be provided in the automatic harvesting device 5 or the user terminal device 9.

[0138] <C. Supplementary Note> From the descriptions of the above-described embodiments and modifications, the following modes can be grasped.

[0139] <Supplementary Note 1> The plant harvesting system Sys includes a storage device 3 that stores determination criterion information DH indicating the determination criteria for whether a fruit SF can be harvested, a user criterion information acquisition unit 21 that acquires user input growth degree information DS2 indicating the determination criteria of the user regarding whether the fruit SF can be harvested, an imaging image information acquisition unit 22 that acquires user imaging image information DG2 indicating a user imaging image GG2 obtained by imaging the fruit SF, and a determination criterion correction unit 23 that corrects the determination criteria indicated by the determination criterion information DH based on the user imaging image GG2 indicated by the user imaging image information DG2 and the determination criteria of the user indicated by the user input growth degree information DS2.

[0140] According to the plant harvesting system Sys described in Supplementary Note 1, since the determination criterion information DH stored in the storage device 3 is corrected based on the user imaging image information DG2 and the user input growth degree information DS2, compared with a mode in which the determination criterion information DH is not corrected, it is possible to more appropriately determine whether the fruit SF can be harvested based on the determination criterion information DH adapted to the surrounding environment of the fruit SF.

[0141] <Supplementary Note 2> In the plant harvesting system Sys described in Supplementary Note 1, the judgment criterion information DH includes a plurality of individual judgment criterion information DK corresponding to changes in the growth degree of the fruit SF, each of the plurality of individual judgment criterion information DK includes a server-managed growth degree AS1 which is the growth degree of the fruit SF corresponding to the individual judgment criterion information DK, and a reference image GG1 which shows the fruit SF at the server-managed growth degree AS1, and the user-input growth degree information DS2 indicates a user-input growth degree AS2 which is the growth degree of the fruit SF, and the judgment criterion correction unit 23 selects a user-input growth degree AS2 which is the growth degree of the fruit SF from the plurality of individual judgment criterion information DK included in the judgment criterion information DH. The method may also be characterized in that individual judgment criterion information DK indicating a server-managed growth degree AS1 that is approximate to the user input growth degree AS2 indicated by the user input growth degree information DS2 acquired by the user criteria information acquisition unit 21 is selected, a color correction amount dT related to the difference in color tone between the reference image GG1 indicated by the selected individual judgment criterion information DK and the user-captured image GG2 indicated by the user-captured image information DG2 acquired by the captured image information acquisition unit 22 is calculated, and each of the multiple reference images GG1 indicated by the judgment criterion information DH stored in the storage device 3 is corrected based on the calculated color correction amount dT.

[0142] According to the plant harvesting system Sys described in Appendix 2, the reference image GG1 stored in the memory device 3 is corrected based on the user-captured image information DG2 and the user-input growth information DS2, so that it is possible to more appropriately determine whether the fruit SF can be harvested using judgment criteria information DH that is adapted to the surrounding environment of the fruit SF, compared to a configuration in which the reference image GG1 is not corrected.

[0143] <Appendix 3> In the plant harvesting system Sys described in Supplementary Note 1, the judgment criterion information DH includes a plurality of individual judgment criterion information DK corresponding to changes in the growth degree of the fruit SF, each of the plurality of individual judgment criterion information DK includes a server-managed growth degree AS1 which is the growth degree of the fruit SF corresponding to the individual judgment criterion information DK, and a reference image GG1 which shows the fruit SF at the server-managed growth degree AS1, the user-input growth degree information DS2 indicates a user-input growth degree AS2 which is the growth degree of the fruit SF, and the judgment criterion correction unit 23 selects a reference image GG1 from the plurality of individual judgment criterion information DK included in the judgment criterion information DH. The method may also be characterized in that individual judgment criterion information DK indicating a reference image GG1 that is similar to the user-captured image GG2 indicated by the user-captured image information DG2 acquired by the image information acquisition unit 22 is selected, the difference between the server-managed growth degree AS1 indicated by the selected individual judgment criterion information DK and the user-input growth degree AS2 indicated by the user-input growth degree information DS2 acquired by the user standard information acquisition unit 21 is calculated as a growth correction amount dS, and each of the multiple server-managed growth degrees AS1 indicated by the judgment criterion information DH stored in the storage device 3 is corrected based on the calculated growth correction amount dS.

[0144] According to the plant harvesting system Sys described in Appendix 3, the server-managed growth rate AS1 stored in the memory device 3 is corrected based on the user-captured image information DG2 and the user-input growth rate information DS2, so that it is possible to more appropriately determine whether the fruit SF can be harvested using judgment criteria information DH that is adapted to the surrounding environment of the fruit SF, compared to a configuration in which the server-managed growth rate AS1 is not corrected.

[0145] <Appendix 4> The plant harvesting system Sys described in Appendix 3 may be characterized in that, when the captured image information acquisition unit 22 acquires user-captured image information DG2, a judgment criteria comparison screen G2 including a plurality of reference images GG1 corresponding to a plurality of individual judgment criteria information DK included in the judgment criteria information DH and a user-captured image GG2 indicated by the user-captured image information DG2 acquired by the captured image information acquisition unit 22 is displayed on the user terminal device 9, and the display control unit 25 enables the plurality of individual judgment criteria information DK to be changed.

[0146] According to the plant harvesting system Sys described in Appendix 4, a judgment criteria comparison screen G2 including a reference image GG1 and a user-captured image GG2 is displayed on the user terminal device 9, which enables the user of the user terminal device 9 to appropriately correct the judgment criteria indicated by the judgment criteria information DH, compared to a mode in which the screen is not displayed.

[0147] <Appendix 5> The plant harvesting system Sys comprises a storage device 3B that stores a plurality of pieces of judgment criterion information DH indicating criteria for judging whether or not the fruit SF can be harvested, a user criterion information acquisition unit 21 that acquires user-input growth degree information DS2 indicating a user's judgment criterion for whether or not the fruit SF can be harvested, a captured image information acquisition unit 22 that acquires user-captured image information DG2 indicating a user-captured image GG2 of an image of the fruit SF, and a judgment criterion selection unit 23B that selects one piece of judgment criterion information DH to be used for judging whether or not the fruit SF can be harvested from the plurality of pieces of judgment criterion information DH based on the user-captured image GG2 indicated by the user-captured image information DG2 and the user's judgment criterion indicated by the user-input growth degree information DS2, and each of the plurality of pieces of judgment criterion information DH includes a plurality of individual pieces of judgment criterion information DK corresponding to changes in the growth degree of the fruit SF, and Each of the individual judgment criteria information DK includes a server-managed growth degree AS1, which is the growth degree of the fruit SF corresponding to the individual judgment criteria information DK, and a reference image GG1 showing the fruit SF at the server-managed growth degree AS1, and the user-input growth degree information DS2 indicates a user-input growth degree AS2, which is the growth degree of the fruit SF, and the judgment criteria selection unit 23B selects, from the multiple judgment criteria information DH, the judgment criteria information DH to which the individual judgment criteria information DK belongs, which has a server-managed growth degree AS1 that is approximate to the user-input growth degree AS2 indicated by the user-input growth degree information DS2 acquired by the user criteria information acquisition unit 21 and a reference image GG1 that is approximate to the user-captured image GG2 indicated by the user-captured image information DG2 acquired by the captured image information acquisition unit 22, as one judgment criteria information DH to be used to determine whether the fruit SF can be harvested.

[0148] According to the plant harvesting system Sys described in Appendix 5, one piece of judgment criteria information DH to be used for determining whether the fruit SF can be harvested is selected from multiple pieces of judgment criteria information DH based on the user-captured image information DG2 and the user-input growth level information DS2.Therefore, compared to a mode in which the judgment whether the fruit SF can be harvested is made using predetermined judgment criteria information DH, it is possible to more appropriately determine whether the fruit SF can be harvested using judgment criteria information DH that is suited to the surrounding environment of the fruit SF. [Explanation of symbols]

[0149] 1...plant harvest management server, 2...control device, 3...storage device, 4...communication device, 5...automatic harvesting device, 8...agricultural greenhouse, 9...user terminal device, 21...user standard information acquisition unit, 22...captured image information acquisition unit, 23...judgment standard correction unit, 24...harvesting device control unit, 25...display control unit, 26...harvesting judgment unit, 51...control device, 52...storage device, 53...traveling device, 54...manipulator, 55...imaging device, 56...detection device, 57...communication device, 91...control device, 92...storage device, 93...display device, 94...input device, 95...imaging device, 96...communication device, S...strawberry, SF...fruit

Claims

1. a storage unit that stores judgment criteria information indicating criteria for judging whether harvesting of harvest target items is possible; a criteria information acquisition unit that acquires user criteria information indicating a user's criteria for determining whether or not the harvest target item can be harvested; an image information acquisition unit that acquires captured image information indicating a captured image of the harvest object; Based on the captured image indicated by the captured image information and the determination criterion indicated by the user criterion information, a correction unit that corrects the judgment criterion indicated by the judgment criterion information; Equipped with A harvesting system characterized by:

2. The judgment criteria information is a plurality of individual reference information corresponding to changes in the growth degree of the harvest object; Each of the plurality of individual reference information includes: a growth level of the harvest object corresponding to the individual reference information, and a reference image showing the harvest object at that growth level; The user criteria information is indicating the growth level of the harvest object; The correction unit Among the plurality of individual criteria information included in the determination criteria information, selecting individual reference information indicating a growth degree similar to the growth degree indicated by the user reference information acquired by the reference information acquisition unit; calculating a color correction amount relating to a difference in color tone between the reference image indicated by the selected individual reference information and the captured image indicated by the captured image information acquired by the image information acquisition unit; correcting each of the plurality of reference images indicated by the determination reference information stored in the storage unit based on the calculated amount of color tone correction; 2. A harvesting system according to claim 1, characterized in that

3. The judgment criteria information is a plurality of individual reference information corresponding to changes in the growth degree of the harvest object; Each of the plurality of individual reference information includes: a growth level of the harvest object corresponding to the individual reference information, and a reference image showing the harvest object at that growth level; The user criteria information is indicating the growth level of the harvest object; The correction unit Among the plurality of individual criteria information included in the determination criteria information, selecting individual reference information indicating a reference image that is similar to the captured image indicated by the captured image information acquired by the image information acquisition unit; calculating a difference between the growth degree indicated by the selected individual reference information and the growth degree indicated by the user reference information acquired by the reference information acquisition unit as a growth degree correction amount; correcting each of the plurality of growth degrees indicated by the determination criterion information stored in the storage unit based on the calculated growth degree correction amount; 2. A harvesting system according to claim 1, characterized in that

4. A harvesting system that determines whether or not to harvest an object to be harvested based on device captured image information indicating an image of the object to be harvested and judgment criteria information indicating judgment criteria for whether or not the object to be harvested, which is stored in advance in a storage unit, a display control unit that, when an image information acquisition unit acquires user-captured image information, causes a display image including a plurality of reference images corresponding to a plurality of individual reference information included in the determination reference information and a user-captured image indicated by the user-captured image information acquired by the image information acquisition unit to be displayed on a user's terminal device, and that makes the plurality of individual reference information changeable; A harvesting system characterized by:

5. a storage unit that stores a plurality of pieces of judgment criteria information indicating criteria for judging whether or not harvesting of harvest target items is possible; a criteria information acquisition unit that acquires user criteria information indicating a user's criteria for determining whether or not the harvest target item can be harvested; an image information acquisition unit that acquires captured image information indicating a captured image of the harvest object; Based on the captured image indicated by the captured image information and the determination criterion indicated by the user criterion information, a selection unit that selects one piece of judgment criterion information to be used for determining whether or not the harvest target item can be harvested from the plurality of pieces of judgment criterion information; Equipped with Each of the plurality of pieces of determination criteria information is a plurality of individual reference information corresponding to changes in the growth degree of the harvest object; Each of the plurality of individual reference information includes: a growth level of the harvest object corresponding to the individual reference information, and a reference image showing the harvest object at that growth level; The user criteria information is indicating the growth level of the harvest object; The selection unit From the plurality of pieces of judgment criteria information, a growth degree that is close to the growth degree indicated by the user reference information acquired by the reference information acquisition unit; a reference image that is similar to the captured image indicated by the captured image information acquired by the image information acquisition unit; The judgment criteria information to which the individual criteria information belongs, Selecting the one determination criterion information; A harvesting system characterized by:

6. A control method for a harvesting system having a memory unit that stores judgment criteria information indicating criteria for judging whether or not a harvest target object can be harvested, Acquire captured image information indicating a captured image of the harvest object; correcting the judgment criteria indicated by the judgment criteria information based on the captured image indicated by the captured image information and judgment criteria indicated by user criteria information related to the possibility of harvesting the harvest target item, which has been acquired in advance; 1. A method for controlling a harvesting system, comprising:

7. The processor, a criteria information acquisition unit that acquires user criteria information indicating a user's criteria for determining whether or not the harvest target item can be harvested; an image information acquisition unit that acquires captured image information indicating a captured image of the harvest object; Based on the captured image indicated by the captured image information and the determination criterion indicated by the user criterion information, a correction unit that corrects the criteria for determining whether or not the harvest target object can be harvested, which is indicated by the criteria information stored in the storage unit; and and make it function, A program characterized by:

Images