Assistance device, assistance method, and program

Abstract

To allow easy comprehension of the degree of growth of the entire crops and understanding of accuracy of the comprehension of the degree of growth of the crops.SOLUTION: This assistance device comprises: an acquisition unit that acquires an image generated by capturing a crop; a growth degree calculation unit that calculates a nutrient growth degree indicating a degree of nutrient growth of the crop in a division region in the image; a reliability calculation unit that calculates reliability indicating certainty of a nutrient growth degree in the division region; and an output unit that outputs the nutrient growth degree and the reliability in association with the division region.SELECTED DRAWING: Figure 1

Description

[Technical field]

[0001] The present invention relates to an assistance device, an assistance method, and a program. [Background technology]

[0002] For example, Patent Document 1 describes a technique relating to an information processing device for grasping the reliability of agricultural crop surveys, a control method for the information processing device, and a program. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] JP 2020-144609 A Summary of the Invention [Problem to be solved by the invention]

[0004] Conventionally, the degree of growth of crops in a field (farm) has been grasped from an image taken of the field (farm) using a camera or the like. However, even if the degree of growth of the crops in the front side in the depth direction of the image can be grasped, there are cases where the degree of growth of the crops in the back side in the depth direction of the image cannot be grasped. For example, when the crops are photographed obliquely or from the side, the leaves of the crops overlap at the front and back, and it may be impossible to grasp the degree of growth of the crops in the back side in the depth direction of the image. In some cases, the degree of growth of crops in a field is grasped using satellite data or images taken from the sky by a drone, but not everyone can easily handle satellite data or drones. In addition, if the degree of growth of the entire crop cannot be accurately grasped, the efficiency of work in the field is reduced.

[0005] The present invention has been made in consideration of the above-mentioned situation, and has an object to provide a technology that makes it possible to easily grasp the growth degree of the entire crop and to grasp how accurately the growth degree of the crop is grasped. [Means for solving the problem]

[0006] A support device according to one aspect of the present invention is a support device including: an acquisition unit that acquires an image generated by photographing a crop; a growth degree calculation unit that calculates a nutrient growth degree indicating a degree of nutrient growth of the crop in a divided area in the image; a reliability calculation unit that calculates a reliability indicating the certainty of the nutrient growth degree in the divided area; and an output unit that outputs the nutrient growth degree and the reliability linked to the divided area. With the support device, a worker can easily grasp the growth degree of the entire crop. In addition, the worker can grasp how accurately the growth degree of the entire crop is grasped. That is, the worker can grasp the part of the growth degree of the entire crop that is correctly grasped and the part that is roughly grasped. For example, when a worker determines a place to work on the crop based on the image, the worker can appropriately grasp the place to work on the crop. Therefore, the worker can appropriately grasp the place in the field where he / she will intervene or work with priority, and the work efficiency and yield in the field are improved.

[0007] The support device may include an area dividing section that divides a predetermined area in the image into a plurality of divided areas, the growth degree calculation section calculates the vegetative growth degree for each of the plurality of divided areas, the reliability calculation section calculates the reliability for each of the plurality of divided areas, and the output section outputs the vegetative growth degree and the reliability linked to the plurality of divided areas. In the support device, the growth degree calculation section may calculate the vegetative growth degree based on at least one of the length, area, and volume of the crop in the image and color information of the image. In the support device, the reliability calculation unit may calculate the reliability based on distance information in a depth direction of the image. In the support device, the reliability calculation unit may calculate the reliability based on position information of the image having three-dimensional information. In the support device, the acquisition unit may acquire a shooting angle when the image is shot, and the reliability calculation unit may calculate the reliability according to the shooting angle.

[0008] The support device may further include a display unit that displays the vegetative growth degree and the reliability associated with the divided area together with the image, so that when a worker determines a location for performing work on a crop based on the image displayed on the display unit, the worker can appropriately grasp the location for performing work on the crop.

[0009] A plurality of predetermined regions in the image may be divided into a plurality of the divided regions, and the support device may include an evaluation value calculation unit that calculates an evaluation value based on the vegetative growth degree and the reliability for each of the divided regions, and calculates an overall evaluation value for each of the predetermined regions by adding up the evaluation values ​​calculated for each of the divided regions, and the output unit may output the overall evaluation value linked to each of the predetermined regions. By checking the overall evaluation value for each of the predetermined regions, the worker can properly grasp the location in the field where intervention should be focused or work should be performed with priority. For example, the worker can use the overall evaluation value as a criterion for determining which area to focus on when photographing the area where crops are cultivated in the field next time. For example, by performing an overall evaluation for each ridge in the field, the worker can determine which ridge to start work on.

[0010] The plurality of predetermined regions in the image may be divided into a plurality of the segmented regions, the support device may include an evaluation value calculation unit that calculates evaluation values ​​for all of the segmented regions included in the plurality of the predetermined regions based on the vegetative growth degree and the reliability and calculates a comprehensive evaluation value by adding up the calculated evaluation values, and the output unit may output the comprehensive evaluation value linked to the plurality of the predetermined regions. An operator can check the comprehensive evaluation value for the entire field.

[0011] The support device may include a storage unit that stores the vegetative growth degree and the reliability in association with the plurality of the divided areas, and an update unit that updates the vegetative growth degree and the reliability of the divided areas stored in the storage unit based on the vegetative growth degree and the reliability of the divided areas in the image when the crop was photographed, for the divided areas that overlap between the plurality of the divided areas in the image when the crop was photographed and the plurality of the divided areas stored in the storage unit, when the reliability of the divided area in the image when the crop was photographed satisfies a predetermined standard. By updating the vegetative growth degree and the reliability of the divided areas stored in the storage unit, the vegetative growth degree and the reliability of the divided areas stored in the storage unit become the latest values. This allows the operator to more accurately grasp the areas in the field where he or she should focus on intervention or prioritize work.

[0012] An assistance method according to one aspect of the present invention is a method comprising the steps of: acquiring an image generated by photographing a crop; calculating a vegetative growth degree indicating the degree of vegetative growth of the crop in a divided area within the image; calculating a reliability indicating the likelihood of the vegetative growth degree in the divided area; and outputting the vegetative growth degree and the reliability in association with the divided area.

[0013] A program according to one aspect of the present invention includes a step of causing a computer to perform the steps of: acquiring an image generated by photographing a crop; calculating a vegetative growth degree indicating a degree of vegetative growth of the crop in a divided area in the image; calculating a reliability indicating a likelihood of the vegetative growth degree in the divided area; and calculating the vegetative growth degree and the reliability for the divided area. The program is for executing the steps of linking and outputting the data. Effect of the Invention

[0014] According to the present invention, it is possible to provide a technique that makes it possible to easily grasp the growth level of an entire crop and to grasp how accurately the growth level of the crop is being grasped. [Brief description of the drawings]

[0015] [Figure 1] FIG. 1 is a block diagram showing the configuration of the cultivation support system. [Diagram 2] FIG. 2 is a flowchart showing an example of the operation of the cultivation support system according to the first embodiment. [Diagram 3] FIG. 3 is a diagram showing an example of an image of a cultivation area. [Figure 4] 4A and 4B are explanatory diagrams of a process for extracting a predetermined area from an image of a cultivation area. [Diagram 5] FIG. 5 is an explanatory diagram of a process for dividing a predetermined area into a plurality of partitioned areas. [Figure 6] FIG. 6 is a diagram showing the LAI of each segmented region. [Figure 7] FIG. 7 is an explanatory diagram of the LAI and the reliability. [Figure 8] FIG. 8 is a block diagram showing the configuration of the cultivation support system. [Figure 9] FIG. 9 is a flowchart showing an example of the operation of the cultivation support system according to the second embodiment. [Figure 10] FIG. 10 is a flowchart showing an example of the update process. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0016] Hereinafter, application examples and each embodiment will be described with reference to the drawings. The application examples and each embodiment described below are aspects of the present application and do not limit the scope of the rights of the present application.

[0017] <Application Examples> The present invention can be applied, for example, to a cultivation support system (cultivation support device) 1 shown in FIG. 1. FIG. 1 is a block diagram showing the configuration of the cultivation support system 1. The cultivation support system 1 is a system that supports workers who work in a farm field, which is an example of a workplace for agriculture and the like. The workers perform cultivation work on crops (plants) in the farm field. Types of cultivation work on crops include, but are not limited to, leaf removal, flower removal, bud removal, training, leaf removal, bud removal, top removal, watering, top dressing, pesticide spraying, and harvesting.

[0018] First Embodiment A cultivation support system 1 according to the first embodiment will be described. As shown in FIG. 1, the cultivation support system 1 includes a photographing unit 10, a control unit 20, a display unit 30, and a storage unit 40. The photographing unit 10 is a photographing device such as a camera that photographs an object and generates an image (image data) of the photographed object. The photographing unit 10 may be a handheld camera of an operator. The photographing unit 10 may be a camera included in a mobile phone, smartphone, tablet terminal, or the like of an operator. The photographing unit 10 may be a camera installed in a farm field. The camera installed in a farm field may be a fixed camera whose photographing position is fixed, or may be a movable camera whose photographing position is changeable. The control unit 20 is a processing device that acquires image data from the photographing unit 10 and executes various processes. The control unit 20 is configured to include an acquisition unit 110, an area division unit 120, a vegetative growth degree calculation unit 130, a reliability calculation unit 140, and an output unit 150.

[0019] The control unit 20 includes a CPU (Central Processing Unit) and a RAM (Random Access Memory The storage unit 40 includes a memory 40a, a ROM (Read Only Memory), etc., and controls each part of the control unit 20 and performs various processes. The display unit 30 is a display device such as a display that displays various images and data. The display unit 30 may also have an input device such as a touch panel. The storage unit 40 includes The storage unit 40 stores programs executed by the control unit 20 and various data used in the processes executed by the control unit 20. For example, the storage unit 40 may be a hard disk drive (HDD) or a storage system (SS). The storage unit 40 may be a removable auxiliary storage device such as a solid state drive (SD). The present invention may be realized by a storage medium.

[0020] Not all of the components of the cultivation support system 1 shown in FIG. 1 are essential, and components of the cultivation support system 1 may be added or deleted as appropriate. For example, the cultivation support system 1 may have a communication interface used for communication, an input device such as a mouse or a touch panel, and the like. At least some of the functions of the cultivation support system 1 may be realized by a computer on the cloud. The photographing unit 10, the control unit 20, the display unit 30, and the storage unit 40 may be configured as an integrated unit, or may be configured as separate units. At least two of the photographing unit 10, the control unit 20, the display unit 30, and the storage unit 40 may be configured as an integrated unit.

[0021] (Example of the cultivation support system in operation) FIG. 2 is a flowchart showing an example of the operation of the cultivation support system 1 according to the first embodiment. An example of the operation of the cultivation support system 1 according to the first embodiment will be described with reference to the flowchart of FIG. 2. In step S11, the photographing unit 10 photographs a predetermined area of ​​the field and generates image data including color information (RGB value of each pixel) and distance information (depth value of each pixel). The depth value indicates, for example, the distance between the photographing unit 10 and the photographed object, and is distance information in the depth direction of the image. The predetermined area of ​​the field is, for example, an area where crops are cultivated in the field. A stereo camera or an RGBD camera may be used as the photographing unit 10. FIG. 3 is a diagram showing an example of an image (photographed image) photographed of an area (cultivation area) in the field where crops are cultivated. In the image 200 shown in FIG. 3, it is possible to grasp the degree of growth of the crops on the front side in the depth direction, but it is not easy to grasp the degree of growth of the crops toward the back side in the depth direction.

[0022] In step S12, the acquisition unit 110 acquires image data from the imaging unit 10. In step S13, the region dividing unit 120 extracts a predetermined region (specific area) from the photographed image based on color information included in the image data. The predetermined region is a region where crops are planted from the front to the back, such as a ridge, a rice field, or a field, but may be other regions. The predetermined region extracted from the photographed image may be one of a plurality of predetermined regions arranged at regular intervals, or one of a plurality of predetermined regions arranged randomly. The region dividing unit 120 may extract a predetermined region from the photographed image by pattern matching. The region dividing unit 120 may extract a plurality of predetermined regions from the photographed image. Figures 4A and 4B are explanatory diagrams of a process of extracting a predetermined region from an image of a cultivation area. Figure 4A shows a state before an extraction process is performed on an image 200 of a cultivation area. Figure 4B shows a state after an extraction process is performed on an image 200 of a cultivation area, in which a predetermined region 201 is extracted from the image 200.

[0023] The captured image may be displayed on the display unit 30, and the worker may use an input device to input the boundary of a predetermined area in the captured image displayed on the display unit 30. The area dividing unit 120 may extract a predetermined area from the captured image based on the boundary of the predetermined area in the captured image displayed on the display unit 30. Alternatively, the worker may print the captured image on paper, write the boundary of the predetermined area in the captured image printed on the paper, and then scan the captured image printed on the paper with a scanning device. The area dividing unit 120 may extract a predetermined area from the captured image based on the boundary of the predetermined area in the captured image scanned by the scanning device.

[0024] In step S14, the region dividing unit 120 divides the predetermined region into a plurality of divided regions based on distance information (distance data) included in the image data. Alternatively, each of the plurality of predetermined regions in the captured image may be divided into a plurality of partitioned regions, or one of the plurality of predetermined regions in the captured image may be divided into a plurality of partitioned regions. For example, the region dividing unit 120 may divide the predetermined region into a plurality of partitioned regions by defining a group of pixels in which the depth value of each pixel included in the image data is included within a certain range as one region (area). The region dividing unit 120 may determine one predetermined region in the captured image as one partitioned region. FIG. 5 is an explanatory diagram of a process of dividing a predetermined region into a plurality of partitioned regions. In FIG. 5, the predetermined region 201 is divided into partitioned regions 201A, 201B, and 201C. The partitioned region 201A is a region formed by a group of pixels in which the depth value of each pixel included in the image data is equal to or greater than 0 and less than 0.3. The partitioned region 201B is a region formed by a group of pixels in which the depth value of each pixel included in the image data is equal to or greater than 0.3 and less than 0.6. The partitioned region 201C is a region made up of a group of pixels, each of which has a depth value of 0.6 or more and 1.0 or less, contained in the image data.

[0025] The captured image may be displayed on the display unit 30, and the operator may use an input device to input the width and length of a predetermined area in the captured image displayed on the display unit 30. The area dividing unit 120 may divide the predetermined area into a plurality of segmented areas based on the width and length of the predetermined area. Tags or QR codes (registered trademark) may be placed in the field. The area dividing unit 120 may calculate the width and length of the predetermined area based on the tag or QR code (registered trademark) in the captured image, and divide the predetermined area into a plurality of segmented areas based on the width and length of the predetermined area.

[0026] The photographed image may be displayed on the display unit 30, and the worker may input the boundaries of the plurality of partition areas for the photographed image displayed on the display unit 30 using an input device. The area dividing unit 120 may divide the predetermined area into the plurality of partition areas based on the boundaries of the plurality of partition areas input. The worker may also print the photographed image on paper, write the boundaries of the plurality of partition areas for the photographed image printed on the paper, and then scan the photographed image printed on the paper using a scanning device. The area dividing unit 120 may divide the predetermined area into the plurality of partition areas based on the boundaries of the plurality of partition areas in the photographed image scanned by the scanning device. The predetermined area in the photographed image scanned by the scanning device may be divided into the plurality of partition areas. In this case, the process of step S14 is omitted.

[0027] In step S15, the vegetative growth degree calculation unit 130 calculates a vegetative growth degree indicating the degree of vegetative growth of the crop in the captured image for each of the multiple divided areas based on color information included in the image data. When one specific area in the captured image is determined as one divided area, the vegetative growth degree calculation unit 130 may calculate the vegetative growth degree for one divided area based on color information included in the image data. The vegetative growth degree may include, for example, a Leaf Area Index (LAI) indicating the ratio between the total area of ​​the cultivation area and the total area of ​​the leaves in the cultivation area. In addition, The vegetative growth degree may include other indices than the LAI, such as the stem diameter at a specific location, the size of leaves or fruits, etc. The vegetative growth degree calculation unit 130 may calculate the LAI of each of the multiple segmented regions by dividing the number of green pixels in each segmented region by the total number of pixels in each segmented region. FIG. 6 is a diagram showing the LAI of each segmented region. In FIG. 6, the LAI of segmented region 201A is 0.5, the LAI of segmented region 201B is 0.9, and the LAI of segmented region 201C is 0.8.

[0028] In step S16, the reliability calculation unit 140 calculates a reliability indicating the likelihood of the vegetative growth of the crop in the captured image for each of the multiple divided areas. When one specified area in the captured image is determined to be one divided area, the reliability calculation unit 140 may calculate the reliability for one divided area. The reliability is an index indicating the likelihood of the vegetative growth being calculated from the captured image. For example, when the crop is strawberry and the vegetative growth includes LAI, the reliability is high when the cultivation area is photographed from above, and the reliability is low when the cultivation area is photographed diagonally or from the side. In addition, when the cultivation area is photographed from a long distance, the leaves of the crop overlap at the front and back, and the reliability is low when the cultivation area is photographed from a long distance. When the vegetative growth rate is the thickness of the stalk of a crop, the closer the distance from the photographing position to the crop position, the smaller the measurement error and the higher the reliability, whereas the farther the distance from the photographing position to the crop position, the larger the measurement error and the lower the reliability.

[0029] The reliability calculation section 140 may calculate the reliability based on the following formula 1. (Equation 1) Confidence = 1 - (average depth value of pixels in each segmented area / distance D) The distance D is the length of each partitioned region in the depth direction, and in the case of a ridge, for example, it is the length of the side of the ridge in the depth direction.

[0030] FIG. 7 is an explanatory diagram of LAI and reliability. On the left side of FIG. 7, a predetermined area 201 extracted from an image of a cultivation area photographed from an oblique direction is shown. FIG. 7 also shows an image diagram of the divided areas 201A, 202B, and 202C viewed from directly above. The image diagram of the divided areas 201A, 202B, and 202C viewed from directly above shows the state of crops in each of the divided areas 201A, 202B, and 202C. The LAI of the divided area 201A is 0.5. If the average value of the depth values ​​of the pixels in the divided area 201A is 0.1 and the distance D is 1.0, the reliability of the divided area 201A is 1-(0.1 / 1)=0.9. If the average value of the depth values ​​of the pixels in the divided area 201B is 0.5 and the distance D is 1.0, the reliability of the divided area 201B is 1-(0.5 / 1)=0.5. If the average depth value of the pixels in the partitioned region 201C is 0.9 and the distance D is 1.0, the reliability of the partitioned region 201C is 1-(0.9 / 1)=0.1.

[0031] The LAI of the divided area 201A is 0.5, and the reliability of the divided area 201A is 0.9. The reliability of the divided area 201A is high because the photographing object is close to the photographing point. The image of the cultivation area photographed from the side or oblique direction at a position close to the photographing object includes a portion of the cultivation area where no crops are planted, so the discrepancy between the photographed image and the image is small. The LAI of the divided area 201B is 0.9, and the reliability of the divided area 201B is 0.5. The LAI of the divided area 201C is 0.8, and the reliability of the divided area 201C is 0.1. The reliability of the divided area 201C is low because the photographing object is far from the photographing point. The image of the cultivation area photographed from the side or oblique direction at a position far from the photographing object may not include a portion of the cultivation area where no crops are planted. In such a case, the deviation between the photographed image and the image becomes larger. Thus, the higher the reliability, the smaller the deviation between the photographed image and the image, and the lower the reliability, the larger the deviation between the photographed image and the image. Therefore, the higher the reliability, the more accurately the LAI represents the actual crop condition, and the lower the reliability, the more likely it is that the LAI does not represent the actual crop condition.

[0032] The method of calculating the reliability is not limited to Equation 1. For example, the reliability of each partitioned area may be calculated by multiplying the average depth value of the pixels in each partitioned area by a preset coefficient.

[0033] In step S17, the output unit 150 outputs the nutritional growth degree and the reliability in association with the multiple segmented regions. When one predetermined region in the photographed image is determined as one segmented region, the output unit 150 may output the nutritional growth degree and the reliability in association with one segmented region. The display unit 30 may display the nutritional growth degree and the reliability in association with the multiple segmented regions. When one predetermined region in the photographed image is determined as one segmented region, the display unit 30 may display the nutritional growth degree and the reliability in association with one segmented region. The display unit 30 may display the nutritional growth degree and the reliability associated with the multiple segmented regions together with the photographed image. For example, the display unit 30 may display the photographed image and display the nutritional growth degree and the reliability associated with the multiple segmented regions next to the photographed image. For example, the display unit 30 may display the photographed image and display the nutritional growth degree and the reliability associated with the multiple segmented regions in a superimposed manner on the photographed image. For example, the display unit 30 displays a map of a farm field, and divides the map of the farm field into a plurality of divided regions. The categorized predetermined area may be superimposed on the vegetative growth degree and the reliability associated with the plurality of categorized areas. When one predetermined area in the photographed image is determined as one categorized area, the display unit 30 may display the vegetative growth degree and the reliability associated with the one categorized area together with the photographed image. The display unit 30 may display the vegetative growth degree as a number, a color, or a color saturation. The display unit 30 may display the reliability as a number, a color, or a color saturation. Alternatively, when the display unit 30 displays a map of a farm field, the reliability may be displayed as the thickness of the line indicating the boundary between the plurality of categorized areas by changing the thickness of the line for each categorized area.

[0034] The output unit 150 outputs the nutritional growth degree and the reliability in association with a plurality of divided regions, and the display unit 30 displays the nutritional growth degree and the reliability in association with a plurality of divided regions. This allows the worker to easily grasp the growth degree of the entire crop. In addition, the worker can grasp how accurately the growth degree of the entire crop can be grasped. For example, when determining the location to work on the crop based on the captured image, the worker can properly grasp the location to work on the crop. Therefore, the worker can properly grasp the location in the field where the focus of intervention or the priority of work should be given, and the work efficiency and yield in the field are improved. In addition, according to the cultivation support system 1, anyone can easily collect data, and the system can be used by many users and the accumulation of data can contribute to the spread of smart agriculture.

[0035] The output unit 150 may link the vegetative growth degree and the reliability to a plurality of segmented regions by the region identification ID and store them in the storage unit 40. The region identification ID is identification information for identifying a predetermined region and a plurality of segmented regions. The output unit 150 may link the vegetative growth degree and the reliability to a predetermined region and a plurality of segmented regions by the region identification ID and store them in the storage unit 40. The output unit 150 may link the vegetative growth degree and the reliability to a photographed image, a predetermined region, and a plurality of segmented regions by the region identification ID and store them in the storage unit 40. The output unit 150 may link the vegetative growth degree and the reliability to a map of the field, a predetermined region, and a plurality of segmented regions by the region identification ID and store them in the storage unit 40.

[0036] The control unit 20 may calculate an evaluation value based on the nutritional growth degree and the reliability for each of the multiple partitioned areas, and calculate an overall evaluation value (total score) for each of the multiple specified areas by adding up the evaluation values ​​calculated for each of the multiple partitioned areas. The control unit 20 is an example of an evaluation value calculation unit. The nutritional growth degree calculation unit 130, the reliability calculation unit 140, or the output unit 150 may calculate the overall evaluation value. The output unit 150 may output the overall evaluation value linked to each of the multiple specified areas. The control unit 20 may calculate the overall evaluation value based on the following formula 2. The control unit 20 may multiply the reliability of each partitioned area by a specific coefficient. (Formula 2) Overall evaluation value = nutritional growth of area A × reliability of area A + nutritional growth of area B × reliability of area B + + nutritional growth of area Z × reliability of area Z

[0037] The display unit 30 may display a comprehensive evaluation value linked to each of the multiple predetermined areas. The display unit 30 may display the comprehensive evaluation value linked to each of the multiple predetermined areas together with the captured image. For example, the display unit 30 may display the captured image and display the comprehensive evaluation value linked to each of the multiple predetermined areas next to the captured image. For example, the display unit 30 may display the captured image and display the comprehensive evaluation value linked to each of the multiple predetermined areas superimposed on the captured image. For example, the display unit 30 may display a map of a farm field and display the multiple predetermined areas and the comprehensive evaluation value linked to each of the multiple predetermined areas superimposed on the map of the farm field.

[0038] The output unit 150 outputs the overall evaluation value linked to each of the multiple predetermined regions, and the display unit 30 displays the overall evaluation value linked to each of the multiple predetermined regions. For example, if the crop is strawberry, the predetermined region is a ridge, and if the crop is rice, the predetermined region is a rice field or a predetermined range. By checking the overall evaluation value for each of the multiple predetermined areas, the worker can properly grasp the areas in the field where he / she should focus on intervention or prioritize work. For example, the worker can use the overall evaluation value as a criterion for deciding which cultivation area to focus on when photographing the cultivation area next time. For example, the worker can improve the measurement accuracy of the vegetative growth degree by taking additional photographs of areas with low reliability using a method that increases reliability and updating the vegetative growth degree. Examples of methods that increase reliability include taking the next photograph closer to the location than the previous photograph, and taking the next photograph at a higher resolution than the previous photograph.

[0039] The control unit 20 may calculate evaluation values ​​based on the vegetative growth degree and the reliability for all the divided areas included in the plurality of predetermined areas, and calculate an overall evaluation value by adding up the calculated evaluation values. This allows the operator to check the overall evaluation value of the entire field. The vegetative growth degree calculation unit 130, the reliability calculation unit 140, or the output unit 150 may calculate the overall evaluation value. The output unit 150 may output the overall evaluation value linked to the plurality of predetermined areas. The control unit 20 may multiply the reliability of each divided area by a specific coefficient.

[0040] The display unit 30 may display an overall evaluation value linked to the multiple predetermined areas. The display unit 30 may display the overall evaluation value linked to the multiple predetermined areas together with the captured image. For example, the display unit 30 may display the captured image and display the overall evaluation value linked to the multiple predetermined areas next to the captured image. For example, the display unit 30 may display the captured image and display the overall evaluation value linked to the multiple predetermined areas superimposed on the captured image. For example, the display unit 30 may display a map of a farm field and display the multiple predetermined areas and the overall evaluation value linked to the multiple predetermined areas superimposed on the map of the farm field.

[0041] In the above, the photographing unit 10 photographs a predetermined area of ​​a farm field and generates image data including color information and distance information. However, the present invention is not limited to this processing example. The photographing unit 10 may photograph a predetermined area of ​​a farm field and generate image data including color information and position information (coordinate X, coordinate Y, coordinate Z). That is, the photographing unit 10 may generate a photographed image (three-dimensional image) having color information and three-dimensional information (coordinate X, coordinate Y, coordinate Z). The coordinate X, coordinate Y, and coordinate Z indicate the position of a point in a Cartesian coordinate system with a certain point as the origin. The coordinate X, coordinate Y, and coordinate Z are stored in a point cloud format or a mesh format. The region dividing unit 120 may divide the predetermined area into a plurality of divided areas based on the position information (position data) included in the image data. For example, the predetermined area may be divided into a plurality of divided areas by defining a group of pixels in which the coordinate X, coordinate Y, and coordinate Z of each pixel included in the image data are included in a certain range as one area. Furthermore, the region dividing unit 120 may convert position information included in the image data into distance information, and use the converted distance information as an alternative value for the depth value. The reliability calculation unit 140 may calculate the reliability based on the position information included in the image data. For example, the reliability calculation unit 140 may use the distance information converted from the position information by the region dividing unit 120 as an alternative value for the depth value, and calculate the reliability based on Equation 1.

[0042] (vegetative growth rate) In the above, the process in which the vegetative growth degree calculation unit 130 calculates the vegetative growth degree based on the color information included in the image data has been described, but the process is not limited to this example. The vegetative growth degree calculation unit 130 may calculate the vegetative growth degree for each of a plurality of divided regions based on at least one of the length, area, and volume of the crop in the image and the color information included in the image data. The length of the crop may be, for example, the thickness of the stem at a specific position of the crop. In the case of cherry tomatoes, there is a correlation between the stem diameter at a specific position and the size of the leaves that will grow in the future. In addition, the length of the crop may be, for example, the leaf length or leaf width of a specific leaf of the crop. In the case of strawberries, there is a correlation between the length or width of a specific leaf and photosynthetic products.

[0043] The area of ​​the crop is, for example, the size of the leaf area in a certain region. The size of the leaf area in a certain region may be expressed as the LAI. There is a correlation between the size of the leaf area and the photosynthetic product. The vegetative growth degree calculation unit 130 may measure the degree of withering of the leaves. For example, the vegetative growth degree calculation unit 130 may calculate the degree of withering of the crop in each of the multiple divided regions by dividing the number of at least one of yellow and brown pixels in each divided region by the total number of pixels in each divided region. The vegetative growth degree calculation unit 130 may reduce the LAI of each divided region according to the degree of withering of the crop in each divided region. In addition, the vegetative growth degree calculation unit 130 may calculate the vegetative growth degree including the LAI and the degree of withering of the crop for each divided region. The area of ​​the crop is, for example, the total volume of the stalk in a certain space.

[0044] (Reliability) In the above, the process of calculating the reliability based on Equation 1 has been described, but the process is not limited to this. The reliability calculation unit 140 may calculate the reliability by increasing or decreasing the value calculated based on Equation 1 according to the angle (imaging angle) between the line connecting the imaging device (for example, the imaging unit 10) and the imaging target and the horizontal plane, the direction of gravity, the ground, etc. For example, when a crop such as strawberry is photographed from directly above the stalk, the leaves overlap less, but when the stalk is photographed from the side or diagonally, the leaves overlap more. The imaging target includes a specific crop, a crop in a divided area, and the entire imaging range of the imaging device. The imaging angle in the case of a divided area may be the angle with respect to the center or representative point of the divided area, or the average value of the angles with respect to all points in the divided area. The imaging angle in the case of the entire imaging range may be the angle with respect to the center or representative point of the entire imaging range, or the average value of the angles with respect to all points in the entire imaging range, or may simply be the angle between the imaging direction of the imaging device and the horizontal plane, the direction of gravity, the ground, etc. An acceleration sensor or gyro sensor included in the imaging unit 10 may measure the imaging angle. The acquisition unit 110 may acquire the shooting angle when the captured image was captured from the imaging unit 10. The reliability calculation unit 140 can calculate the reliability according to the shooting angle by increasing or decreasing the value calculated based on Equation 1 according to the shooting angle. If the shooting angle satisfies a predetermined criterion, the reliability calculation unit 140 may increase the value calculated based on Equation 1. If the shooting angle does not satisfy the predetermined criterion, the reliability calculation unit 140 may decrease the value calculated based on Equation 1.

[0045] The reliability calculation unit 140 may calculate the reliability by increasing or decreasing the value calculated based on Equation 1 according to the image resolution of the imaging device (for example, the imaging unit 10). The acquisition unit 110 may acquire the image resolution from the imaging unit 10. The reliability calculation unit 140 can calculate the reliability according to the image resolution of the imaging device by increasing or decreasing the value calculated based on Equation 1 according to the image resolution of the imaging device.

[0046] When the captured image is a 3D model image, the reliability calculation unit 140 may calculate the reliability based on the density of the point cloud included in the captured image. The 3D model image is an image in which a three-dimensional model (3D model) of a captured object is created by capturing the captured object multiple times at different angles. The more captured images used to create the 3D model image, the clearer the 3D model image can be created. In addition, the longer the scanning time in creating the 3D model image, the clearer the 3D model image can be created. When the captured image is a 3D model image, the reliability calculation unit 140 may calculate the reliability based on at least one of the number of captured images used to create the 3D model image and the scanning time in creating the 3D model image.

[0047] <Second embodiment> A cultivation support system 1 according to the second embodiment will be described. In the following description, the same configurations and processes as those in the first embodiment will be denoted by the same reference numerals, and detailed description will be omitted. FIG. 8 is a block diagram showing the configuration of the cultivation support system 1. As shown in FIG. 8, the cultivation support system 1 includes an imaging unit 10, a control unit 20, a display unit 30, and a storage unit 40. The control unit 20 is configured to include an acquisition unit 110 , an area division unit 120 , a nutritional growth degree calculation unit 130 , a reliability calculation unit 140 , an output unit 150 , and an update unit 160 .

[0048] The vegetative growth degree and reliability may be linked to a predetermined area and a plurality of segmented areas by the area identification ID and stored in the storage unit 40. The vegetative growth degree and reliability may be linked to a predetermined area and a plurality of segmented areas by the area identification ID and stored in the storage unit 40. The vegetative growth degree and reliability may be linked to a photographed image, a predetermined area, and a plurality of segmented areas by the area identification ID and stored in the storage unit 40. The vegetative growth degree and reliability may be linked to a field map, a predetermined area, and a plurality of segmented areas by the area identification ID and stored in the storage unit 40.

[0049] (Example of the cultivation support system in operation) Fig. 9 is a flowchart showing an example of the operation of the cultivation support system 1 according to the second embodiment. An example of the operation of the cultivation support system 1 according to the second embodiment will be described with reference to the flowchart in Fig. 9. Steps S21 to S26 are similar to S11 to S16 in the flowchart in Fig. 2, and therefore description thereof will be omitted. In step S27, the update unit 160 acquires the vegetative growth degree and the reliability degree, and performs an update process.

[0050] FIG. 10 is a flowchart showing an example of the update process. An example of the update process will be described with reference to the flowchart in FIG. 10. In step S31, the update unit 160 determines whether or not the predetermined area in the captured image at the time of current shooting is the same as the predetermined area stored in the storage unit 40. If the predetermined area in the captured image at the time of current shooting is the same as the predetermined area stored in the storage unit 40 (step S31; YES), the process proceeds to step S32. If the predetermined area in the captured image at the time of current shooting is not the same as the predetermined area stored in the storage unit 40 (step S31; NO), the process proceeds to step S28.

[0051] The worker may input information (area identification information) for identifying the predetermined area using an input device when or after photographing the cultivation area. A tag or a QR code (registered trademark) may be placed in the field. The update unit 160 may automatically recognize the predetermined area. For example, a tag or a QR code (registered trademark) may be placed in the field, and the update unit 160 may calculate the width and length of the predetermined area based on the tag or the QR code (registered trademark) in the photographed image. The update unit 160 may identify the predetermined area based on the width and length of the predetermined area and output the area identification information. The update unit 160 may identify the predetermined area by pattern matching peripheral information (e.g., characteristics of a greenhouse) in the photographed image, and output the area identification information. The area identification information is stored in the storage unit 40. The update unit 160 may determine whether or not the predetermined area in the photographed image at the time of the current photographing is the same as the predetermined area stored in the storage unit 40, based on the area identification information at the time of the current photographing and the area identification information stored in the storage unit 40.

[0052] In step S32, the update unit 160 determines whether or not there are any overlapping segmented areas (overlapping areas) between the multiple segmented areas in the captured image at the current time of shooting and the multiple segmented areas stored in the storage unit 40. If there are any overlapping segmented areas (step S32; YES), the process proceeds to step S33. If there are no overlapping segmented areas (step S32; NO), the process proceeds to step S28.

[0053] In step S33, the update unit 160 judges whether or not the reliability of the overlapping segmented areas in the captured image at the time of current shooting satisfies a predetermined criterion. If the reliability of the overlapping segmented areas in the captured image at the time of current shooting is higher than the reliability of the segmented areas stored in the storage unit 40, the update unit 160 judges that the reliability of the overlapping segmented areas in the captured image at the time of current shooting satisfies the predetermined criterion. On the other hand, if the reliability of the overlapping segmented areas in the captured image at the time of current shooting is higher than the reliability of the segmented areas stored in the storage unit 40, the update unit 160 judges that the reliability of the overlapping segmented areas in the captured image at the time of current shooting satisfies the predetermined criterion. If it is lower than the reliability of the segmented area, the update unit 160 determines that the reliability of the segmented area in the captured image at the current shooting does not satisfy a predetermined criterion. If the reliability of the segmented area in the captured image at the current shooting satisfies the predetermined criterion for the overlapping segmented areas (step S33; YES), the process proceeds to step S34. If the reliability of the segmented area in the captured image at the current shooting does not satisfy the predetermined criterion for the overlapping segmented areas (step S33; NO), the process proceeds to step S28.

[0054] In step S34, the update unit 160 updates the vegetative growth degree and reliability of the overlapping divided areas stored in the memory unit 40 based on the vegetative growth degree and reliability of the divided areas in the captured image at the current time of shooting. After the processing of step S34 is performed, the processing proceeds to step S28. By updating the vegetative growth degree and reliability of the divided areas stored in the memory unit 40, the vegetative growth degree and reliability of the divided areas stored in the memory unit 40 become the latest values. This allows the worker to more appropriately grasp the areas in the field where intervention should be focused or work should be prioritized.

[0055] In step S28, the output unit 150 outputs the vegetative growth degree and the reliability degree in association with the plurality of segmented regions. Step S28 is similar to S17 in the flowchart of FIG. 2, and therefore a detailed description thereof will be omitted.

[0056] The update unit 160 may perform a process of lowering the reliability of the divided area stored in the storage unit 40 when the elapsed time from the time stored in the storage unit 40 exceeds a predetermined time. As time passes, the length, area, and volume of the crop change due to growth, etc. The update unit 160 may lower the reliability of the divided area stored in the storage unit 40 by a larger amount as the elapsed time from the time stored in the storage unit 40 becomes longer. This allows the worker to more appropriately grasp the areas in the field where intervention should be focused or work should be performed with priority.

[0057] The present invention may be understood as a cultivation support device, an information processing device, or an information processing system having at least a part of each of the configurations, each means, and each function described above. The present invention may be understood as a cultivation support method or an information processing method including at least a part of each of the processes described above. The present invention may be understood as a method in which a computer executes each of the processes described above. The present invention may be understood as a program for causing a computer to execute each of the processes described above, and the program may be provided to the computer via a network or from a computer-readable recording medium that non-temporarily holds data.

[0058] <Computer-readable recording medium> A program for causing an information processing device or other machine or device (hereinafter, computer, etc.) to realize any of the above functions can be recorded on a computer-readable recording medium. The function can be provided by having the computer, etc. read and execute the program from the recording medium. A computer-readable recording medium refers to a recording medium that stores information such as data and programs through electrical, magnetic, optical, mechanical, or chemical action and can be read by a computer, etc.

[0059] <Appendix 1> An acquisition unit (110) that acquires an image generated by photographing a crop; a growth degree calculation unit (130) that calculates a vegetative growth degree indicating a degree of vegetative growth of the crop in the divided region within the image; a reliability calculation unit (140) that calculates a reliability indicating the likelihood of the vegetative growth degree in the divided region; an output unit (150) that outputs the vegetative growth degree and the reliability degree in association with the divided area; An assistance device (1) comprising: <Appendix 2> A region dividing unit (120) that divides a predetermined region in the image into a plurality of divided regions, The growth degree calculation unit (130) calculates the vegetative growth degree for each of the plurality of divided regions, The reliability calculation unit (140) calculates the reliability for each of the plurality of partitioned regions, The output unit (150) outputs the vegetative growth degree and the reliability degree in association with the plurality of divided areas. Assistance device (1) as described in appendix 1. <Appendix 3> the growth degree calculation unit (130) calculates the vegetative growth degree based on at least one of the length, area, and volume of the crop in the image and color information of the image; Assistance device (1) as described in appendix 1. <Appendix 4> The reliability calculation unit (140) calculates the reliability based on distance information in a depth direction of the image. 4. An assistance device (1) according to any one of appendices 1 to 3. <Appendix 5> The reliability calculation unit (140) calculates the reliability based on position information of the image having three-dimensional information. 4. An assistance device (1) according to any one of appendices 1 to 3. <Appendix 6> The acquisition unit acquires a shooting angle when the image was captured, The reliability calculation unit calculates the reliability according to the imaging angle. 6. An assistance device (1) according to any one of appendices 1 to 5. <Appendix 7> A display unit (30) is provided for displaying the vegetative growth degree and the reliability associated with the divided region together with the image. 7. An assistance device (1) according to any one of appendices 1 to 6. <Appendix 8> A plurality of predetermined regions in the image are divided into a plurality of the partition regions, an evaluation value calculation unit (20) that calculates an evaluation value for each of the plurality of partitioned regions based on the vegetative growth degree and the reliability, and calculates a comprehensive evaluation value for each of the plurality of predetermined regions by adding up the evaluation values ​​calculated for each of the plurality of partitioned regions; The output unit (150) outputs the comprehensive evaluation value in association with each of the plurality of predetermined regions. 8. An assistance device (1) according to any one of appendices 1 to 7. <Appendix 9> A plurality of predetermined regions in the image are divided into a plurality of the partition regions, an evaluation value calculation unit (20) that calculates an evaluation value based on the vegetative growth degree and the reliability for all of the partitioned areas included in the plurality of predetermined areas, and calculates a comprehensive evaluation value by adding up the calculated evaluation values; The output unit (150) outputs the comprehensive evaluation value in association with a plurality of the predetermined regions. 8. The assistance device of claim 1 . <Appendix 10> a storage unit (40) that stores the vegetative growth degree and the reliability degree in association with a plurality of the divided areas; an update unit (160) that updates, for a segmented area that overlaps between a plurality of the segmented areas in the image when the crop was photographed and a plurality of the segmented areas stored in the storage unit (40), the vegetative growth degree and the reliability of the segmented area stored in the storage unit (40) based on the vegetative growth degree and the reliability of the segmented area in the image when the crop was photographed, if the reliability of the segmented area in the image when the crop was photographed satisfies a predetermined criterion; 10. An assistance device (1) according to any one of appendices 1 to 9, comprising: <Appendix 11> A step of acquiring an image generated by photographing a crop (S12, S22); A step of calculating a vegetative growth degree indicating a degree of vegetative growth of the crop in the divided region in the image (S15, S25); A step of calculating a reliability indicating the likelihood of the vegetative growth in the divided area (S16, S26); outputting the vegetative growth rate and the reliability rate in association with the divided area (S17, S28); A method of support that provides: <Appendix 12> On the computer, A step of acquiring an image generated by photographing a crop (S12, S22); A step of calculating a vegetative growth degree indicating a degree of vegetative growth of the crop in the divided region in the image (S15, S25); A step of calculating a reliability indicating the likelihood of the vegetative growth in the plurality of regions (S16, S26); outputting the vegetative growth rate and the reliability rate in association with the divided area (S17, S28); A program for executing the above. [Explanation of symbols]

[0060] 1. Cultivation support system 10. Photography Department 20 Control section 30 Display section 40 Storage section 110 Acquisition Department 120 Area division 130 Nutritional Growth Calculation Department 140 Reliability calculation unit 150 Output section 160 Update Department

Claims

1. An acquisition unit that acquires an image generated by photographing a crop; a growth degree calculation unit that calculates a vegetative growth degree indicating a degree of vegetative growth of the crop in the divided region within the image; a reliability calculation unit that calculates a reliability indicating the likelihood of the vegetative growth degree in the divided region; an output unit that outputs the vegetative growth degree and the reliability degree in association with the divided area; A support device comprising:

2. a region dividing unit that divides a predetermined region in the image into a plurality of the divided regions; the growth degree calculation unit calculates the vegetative growth degree for each of the plurality of partitioned regions; The reliability calculation unit calculates the reliability for each of the plurality of partitioned regions, The output unit outputs the vegetative growth degree and the reliability degree in association with the plurality of the divided regions. The support device according to claim 1 .

3. the growth degree calculation unit calculates the vegetative growth degree based on at least one of the length, area, and volume of the crop in the image and color information of the image; The support device according to claim 1 .

4. The reliability calculation unit calculates the reliability based on distance information in a depth direction of the image. The support device according to claim 1 .

5. The reliability calculation unit calculates the reliability based on position information of the image having three-dimensional information. The support device according to claim 1 .

6. The acquisition unit acquires a shooting angle when the image was captured, The reliability calculation unit calculates the reliability according to the imaging angle. The support device according to claim 1 .

7. a display unit that displays the vegetative growth degree and the reliability associated with the segmented region together with the image; The support device according to claim 1 .

8. A plurality of predetermined regions in the image are divided into a plurality of the partition regions, an evaluation value calculation unit that calculates an evaluation value for each of the plurality of partitioned regions based on the vegetative growth degree and the reliability, and calculates a comprehensive evaluation value for each of the plurality of predetermined regions by adding up the evaluation values ​​calculated for each of the plurality of partitioned regions; The output unit outputs the comprehensive evaluation value in association with each of the plurality of predetermined regions. The support device according to claim 1 .

9. A plurality of predetermined regions in the image are divided into a plurality of the partition regions, an evaluation value calculation unit that calculates an evaluation value based on the vegetative growth degree and the reliability for all of the partitioned areas included in the plurality of predetermined areas, and calculates a comprehensive evaluation value by adding up the calculated evaluation values; The output unit outputs the comprehensive evaluation value in association with a plurality of the predetermined regions. The support device according to claim 1 .

10. a storage unit that stores the vegetative growth degree and the reliability degree in association with a plurality of the divided regions; an update unit that updates, for a segmented area that overlaps between a plurality of the segmented areas in the image when the crop was photographed and a plurality of the segmented areas stored in the storage unit, the vegetative growth degree and the reliability of the segmented area stored in the storage unit based on the vegetative growth degree and the reliability of the segmented area in the image when the crop was photographed, if the reliability of the segmented area in the image when the crop was photographed satisfies a predetermined criterion; 10. The assistance device according to claim 1 , further comprising:

11. acquiring an image generated by photographing the crop; calculating a vegetative growth degree indicating a degree of vegetative growth of the crop in the divided region within the image; calculating a reliability indicating the likelihood of the vegetative growth in the divided region; outputting the vegetative growth degree and the reliability degree in association with the segmented area; A method of support that provides:

12. On the computer, acquiring an image generated by photographing the crop; calculating a vegetative growth degree indicating a degree of vegetative growth of the crop in the divided region within the image; calculating a reliability indicating the likelihood of the vegetative growth in the divided region; outputting the vegetative growth degree and the reliability degree in association with the segmented area; A program for executing the above.

Images