Assistance device, assistance method, and program

Abstract

The present invention allows easy comprehension of the degree of growth of whole crops and understanding of accuracy of the comprehension of the degree of growth of the crops. This assistance device comprises: an acquisition unit that acquires an image generated by capturing a crop; a growth degree calculation unit that calculates a nutrient growth degree indicating the degree of nutrient growth of the crop in a division region in the image; a reliability calculation unit that calculates reliability indicating the certainty of the nutrient growth degree in the division region; and an output unit that outputs the nutrient growth degree and the reliability in association with the division region.

Description

Support device, support method, and program

[0001] The present invention relates to an assistance device, an assistance method, and a program.

[0002] For example, Patent Document 1 describes a technology relating to an information processing device, a control method for the information processing device, and a program for understanding the reliability of agricultural crop surveys.

[0003] Japanese Patent Application Laid-Open No. 2020-144609

[0004] Conventionally, the growth level of crops in a field (farm) has been determined from images of the field taken using a camera or the like. However, even if it is possible to determine the growth level of the crops in the foreground in the depth direction of the image, it may not be possible to determine the growth level of the crops in the background in the depth direction of the image. For example, when crops are photographed from an angle or from the side, the leaves of the crops may overlap in the front and back, making it impossible to determine the growth level of the crops in the background in the depth direction of the image. The growth level of crops in a field may be determined using satellite data or images taken from the air by a drone, but satellite data and drones are not necessarily easy for everyone to use. Furthermore, if the overall growth level of the crops cannot be accurately determined, work efficiency in the field will be reduced.

[0005] The present invention has been made in consideration of the above-mentioned situation, and its purpose is to provide a technology that makes it possible to easily grasp the growth level of the entire crop and to grasp how accurately the growth level of the crop is grasped.

[0006] According to one aspect of the present invention, an assistance device includes an acquisition unit that acquires an image generated by photographing a crop; a growth degree calculation unit that calculates a vegetative growth degree indicating the degree of vegetative growth of the crop in a segmented region within the image; a reliability calculation unit that calculates a reliability indicating the likelihood of the vegetative growth degree in the segmented region; and an output unit that outputs the vegetative growth degree and the reliability associated with the segmented region. The assistance device allows a worker to easily grasp the growth degree of the entire crop. Furthermore, the worker can grasp how accurately the growth degree of the entire crop is grasped. That is, the worker can grasp which parts of the growth degree of the entire crop are correctly grasped and which parts are roughly grasped. For example, when determining where to perform work on the crop based on the image, the worker can appropriately grasp the areas where the work will be performed on the crop. Therefore, the worker can appropriately grasp the areas in the field where he or she needs to focus or prioritize work, improving work efficiency and yield in the field.

[0007] The support device may include an area dividing unit that divides a predetermined area in the image into a plurality of divided areas, wherein the growth degree calculation unit calculates the vegetative growth degree for each of the plurality of divided areas, the reliability calculation unit calculates the reliability for each of the plurality of divided areas, and the output unit outputs the vegetative growth degree and the reliability associated with the plurality of divided areas. In the support device, the growth degree calculation unit may calculate the vegetative growth degree based on at least one of the length, area, and volume of the crop in the image and color information of the image. In the support device, the reliability calculation unit may calculate the reliability based on distance information in a depth direction of the image. In the support device, the reliability calculation unit may calculate the reliability based on position information of the image having three-dimensional information. In the support device, the acquisition unit may acquire a shooting angle when the image was captured, and the reliability calculation unit may calculate the reliability according to the shooting angle.

[0008] The support device may further include a display unit that displays the vegetative growth rate and the reliability associated with the segmented area together with the image, allowing the worker to properly understand the location of the crop work when deciding where to work on the crop based on the image displayed on the display unit.

[0009] The plurality of predetermined areas in the image may be divided into a plurality of segmented areas, and the support device may include an evaluation value calculation unit that calculates an evaluation value for each of the segmented areas based on the vegetative growth rate and the reliability, and calculates an overall evaluation value for each of the predetermined areas by adding up the evaluation values ​​calculated for each of the segmented areas. The output unit may output the overall evaluation value associated with each of the predetermined areas. By checking the overall evaluation value for each of the predetermined areas, a worker can appropriately identify areas in the field where intervention should be focused or work should be prioritized. For example, a worker can use the overall evaluation value as a basis for determining which areas to focus on when photographing an area where crops are grown in the field next time. For example, by performing an overall evaluation for each ridge in the field, a worker can determine which ridge to start work on.

[0010] The plurality of predetermined areas in the image may be divided into a plurality of the segmented areas, and the support device may include an evaluation value calculation unit that calculates evaluation values ​​based on the vegetative growth degree and the reliability for all of the segmented areas included in the plurality of predetermined areas and calculates a comprehensive evaluation value by adding up the calculated evaluation values, and the output unit may output the comprehensive evaluation value linked to the plurality of predetermined areas. An operator can check the comprehensive evaluation value for the entire field.

[0011] The support device may include a storage unit that stores the vegetative growth degree and the reliability associated with each of the plurality of segmented areas, and an update unit that updates the vegetative growth degree and the reliability of a segmented area stored in the storage unit based on the vegetative growth degree and the reliability of the segmented area in the image when the crop was photographed, for a segmented area that overlaps between the plurality of segmented areas in the image when the crop was photographed and the plurality of segmented areas stored in the storage unit, if the reliability of the segmented area in the image when the crop was photographed satisfies a predetermined standard. By updating the vegetative growth degree and the reliability of the segmented area stored in the storage unit, the vegetative growth degree and the reliability of the segmented area stored in the storage unit are updated to the latest values. This allows the operator to more accurately identify areas in the field where intervention is focused or work is prioritized.

[0012] An assistance method according to one aspect of the present invention includes the steps of: acquiring an image generated by photographing a crop; calculating a vegetative growth rate indicating the degree of vegetative growth of the crop in a divided area within the image; calculating a reliability indicating the likelihood of the vegetative growth rate in the divided area; and outputting the vegetative growth rate and the reliability in association with the divided area.

[0013] A program according to one aspect of the present invention is a program for causing a computer to execute the steps of: acquiring an image generated by photographing a crop; calculating a vegetative growth rate indicating the degree of vegetative growth of the crop in a divided area within the image; calculating a reliability indicating the likelihood of the vegetative growth rate in the divided area; and outputting the vegetative growth rate and the reliability in association with the divided area.

[0014] According to the present invention, it is possible to provide a technology that makes it possible to easily grasp the growth level of the entire crop and to grasp how accurately the growth level of the crop can be grasped.

[0015] FIG. 1 is a block diagram showing the configuration of a cultivation support system. FIG. 2 is a flowchart showing an example of the operation of the cultivation support system according to the first embodiment. FIG. 3 is a diagram showing an example of an image of a cultivation area. FIGS. 4A and 4B are explanatory diagrams of a process of extracting a predetermined area from an image of a cultivation area. FIG. 5 is an explanatory diagram of a process of dividing a predetermined area into a plurality of segmented areas. FIG. 6 is a diagram showing the LAI of each segmented area. FIG. 7 is an explanatory diagram of the LAI and reliability. FIG. 8 is a block diagram showing the configuration of the cultivation support system. FIG. 9 is a flowchart showing an example of the operation of the cultivation support system according to the second embodiment. FIG. 10 is a flowchart showing an example of an update process.

[0016] The application examples and embodiments will be described below with reference to the drawings. The application examples and embodiments described below are merely aspects of the present application and do not limit the scope of the rights of the present application.

[0017] <Application Example> The present invention can be applied, for example, to a cultivation support system (cultivation support device) 1 shown in Fig. 1. Fig. 1 is a block diagram showing the configuration of the cultivation support system 1. The cultivation support system 1 is a system that supports workers who work in a farm field, which is an example of a workplace in agriculture, etc. The workers perform cultivation work on crops (plants) in the farm field. Types of cultivation work for crops include, but are not limited to, leaf removal, flower removal, bud removal, training, leaf removal, bud removal, topping, watering, top dressing, pesticide spraying, and harvesting.

[0018] First Embodiment A cultivation support system 1 according to a first embodiment will be described. As shown in FIG. 1 , the cultivation support system 1 includes a photographing unit 10, a control unit 20, a display unit 30, and a storage unit 40. The photographing unit 10 is a photographing device such as a camera that photographs an object and generates an image (image data) of the photographed object. The photographing unit 10 may be a handheld camera held by an operator. The photographing unit 10 may also be a camera included in the operator's mobile phone, smartphone, tablet, or the like. The photographing unit 10 may also be a camera installed in a farm field. The camera installed in the farm field may be a fixed camera with a fixed photographing position or a movable camera with a changeable photographing position. The control unit 20 is a processing device that acquires image data from the photographing unit 10 and executes various processes. The control unit 20 is configured to include an acquisition unit 110, a region dividing unit 120, a vegetative growth degree calculation unit 130, a reliability calculation unit 140, and an output unit 150.

[0019] The control unit 20 includes a CPU (Central Processing Unit), RAM (Random Access Memory), ROM (Read Only Memory), etc., and controls each unit of the control unit 20 and performs various processes. The display unit 30 is a display device such as a display that displays various images and data. The display unit 30 may also have an input device such as a touch panel. The storage unit 40 stores programs executed by the control unit 20 and various data used in the processes executed by the control unit 20. For example, the storage unit 40 may be an auxiliary storage device such as an HDD (Hard Disk Drive) or an SSD (Solid State Drive). The storage unit 40 may also be realized by a removable storage medium.

[0020] Not all of the components of the cultivation support system 1 shown in FIG. 1 are essential, and components of the cultivation support system 1 may be added or deleted as appropriate. For example, the cultivation support system 1 may have a communication interface used for communication, an input device such as a mouse or a touch panel, and the like. Furthermore, at least some of the functions of the cultivation support system 1 may be realized by a cloud-based computer. The photographing unit 10, the control unit 20, the display unit 30, and the storage unit 40 may be configured as an integrated unit, or may be configured as separate units. At least two of the photographing unit 10, the control unit 20, the display unit 30, and the storage unit 40 may be configured as an integrated unit.

[0021] (Example of Operation of Cultivation Support System) FIG. 2 is a flowchart showing an example of the operation of the cultivation support system 1 according to the first embodiment. An example of the operation of the cultivation support system 1 according to the first embodiment will be described with reference to the flowchart of FIG. 2. In step S11, the image capture unit 10 captures an image of a predetermined area of ​​the field and generates image data including color information (RGB values ​​of each pixel) and distance information (depth values ​​of each pixel). The depth value indicates, for example, the distance between the image capture unit 10 and the object being captured, and is distance information in the depth direction of the image. The predetermined area of ​​the field is, for example, an area where crops are cultivated in the field. A stereo camera or an RGBD camera may be used as the image capture unit 10. FIG. 3 is a diagram showing an example of an image (captured image) captured of an area (cultivation area) in the field where crops are cultivated. In the image 200 shown in FIG. 3, it is possible to grasp the degree of growth of the crops in the foreground in the depth direction, but it is difficult to grasp the degree of growth of the crops as one moves further back in the depth direction.

[0022] In step S12, the acquisition unit 110 acquires image data from the photographing unit 10. In step S13, the region dividing unit 120 extracts a predetermined region (specific area) from the photographed image based on color information included in the image data. The predetermined region is an area where crops are planted from the front to the back, such as a ridge, rice paddy, or field, but may be any other area. The predetermined region extracted from the photographed image may be one of multiple predetermined regions arranged at regular intervals, or one of multiple predetermined regions arranged randomly. The region dividing unit 120 may extract a predetermined region from the photographed image by pattern matching. The region dividing unit 120 may extract multiple predetermined regions from the photographed image. Figures 4A and 4B are explanatory diagrams of the process of extracting a predetermined region from an image of a cultivation area. Figure 4A shows the state before the extraction process is performed on an image 200 of a cultivation area. FIG. 4B shows the state after the extraction process is performed on the image 200 of the cultivation area, and a predetermined area 201 is extracted from the image 200.

[0023] The captured image may be displayed on the display unit 30, and the worker may use an input device to input the boundary of a predetermined area in the captured image displayed on the display unit 30. The area dividing unit 120 may extract a predetermined area from the captured image based on the boundary of the predetermined area in the captured image displayed on the display unit 30. Alternatively, the worker may print the captured image on paper, draw the boundary of the predetermined area in the captured image printed on paper, and then scan the captured image printed on paper with a scanning device. The area dividing unit 120 may extract a predetermined area from the captured image based on the boundary of the predetermined area in the captured image scanned by the scanning device.

[0024] In step S14, the region dividing unit 120 divides the predetermined region into multiple segmented regions based on distance information (distance data) included in the image data. The region dividing unit 120 may divide each of the multiple predetermined regions in the captured image into multiple segmented regions, or may divide one of the multiple predetermined regions in the captured image into multiple segmented regions. For example, the region dividing unit 120 may divide the predetermined region into multiple segmented regions by defining a group of pixels included in the image data whose depth values ​​fall within a certain range as one region (area). The region dividing unit 120 may also determine one predetermined region in the captured image as one segmented region. FIG. 5 is an explanatory diagram of the process of dividing the predetermined region into multiple segmented regions. In FIG. 5, the predetermined region 201 is divided into segmented regions 201A, 201B, and 201C. The segmented region 201A is an area formed by a group of pixels included in the image data whose depth values ​​are equal to or greater than 0 and less than 0.3. Segmented region 201B is an area made up of a group of pixels included in the image data whose depth values ​​are equal to or greater than 0.3 and less than 0.6. Segmented region 201C is an area made up of a group of pixels included in the image data whose depth values ​​are equal to or greater than 0.6 and less than 1.0.

[0025] The captured image may be displayed on the display unit 30, and the operator may use an input device to input the width and length of a predetermined area in the captured image displayed on the display unit 30. The area dividing unit 120 may divide the predetermined area into a plurality of segmented areas based on the width and length of the predetermined area. Tags or QR codes (registered trademark) may be placed in the field. The area dividing unit 120 may calculate the width and length of the predetermined area based on the tags or QR codes (registered trademark) in the captured image, and divide the predetermined area into a plurality of segmented areas based on the width and length of the predetermined area.

[0026] The captured image may be displayed on the display unit 30, and the operator may use an input device to input boundaries of multiple segment areas for the captured image displayed on the display unit 30. The region dividing unit 120 may divide the predetermined area into multiple segment areas based on the input boundaries of the multiple segment areas. Alternatively, the operator may print the captured image on paper, draw boundaries of the multiple segment areas for the captured image printed on paper, and then scan the printed image using a scanning device. The region dividing unit 120 may divide the predetermined area into multiple segment areas based on the boundaries of the multiple segment areas in the captured image scanned by the scanning device. Alternatively, the predetermined area in the captured image scanned by the scanning device may be divided into multiple segment areas. In this case, the processing of step S14 is omitted.

[0027] In step S15, the vegetative growth degree calculation unit 130 calculates the vegetative growth degree, which indicates the degree of vegetative growth of the crop in the captured image, for each of the multiple segmented regions based on the color information included in the image data. If a specific region in the captured image is determined to be one segmented region, the vegetative growth degree calculation unit 130 may calculate the vegetative growth degree for that segmented region based on the color information included in the image data. The vegetative growth degree may include, for example, a leaf area index (LAI), which indicates the ratio of the total area of ​​the cultivation area to the total leaf area in the cultivation area. The vegetative growth degree may also include other indices besides the LAI, such as the stem diameter at a specific location, the size of the leaves, or the size of the fruit. The vegetative growth degree calculation unit 130 may calculate the LAI for each of the multiple segmented regions by dividing the number of green pixels in that segmented region by the total number of pixels in that segmented region. Figure 6 shows the LAI for each segmented region. In FIG. 6, the LAI of the segmented area 201A is 0.5, the LAI of the segmented area 201B is 0.9, and the LAI of the segmented area 201C is 0.8.

[0028] In step S16, the reliability calculation unit 140 calculates a reliability indicating the likelihood of the vegetative growth of the crop in the captured image for each of the multiple segmented regions. If a specific region in the captured image is determined to be one segmented region, the reliability calculation unit 140 may calculate the reliability for that segmented region. The reliability is an index indicating the likelihood of vegetative growth calculated from the captured image. For example, if the crop is a strawberry and the vegetative growth includes LAI, the reliability is high when the cultivation area is photographed from above, but low when the cultivation area is photographed from an oblique or lateral angle. Furthermore, if the cultivation area is photographed from a long distance, the reliability decreases due to overlapping of the crop leaves at the front and rear. If the vegetative growth is the thickness of the crop stem, the closer the distance from the photographing position to the crop, the smaller the measurement error and the higher the reliability. Meanwhile, the farther the distance from the photographing position to the crop, the larger the measurement error and the lower the reliability.

[0029] The reliability calculation unit 140 may calculate the reliability based on the following formula 1: (Formula 1) Reliability = 1 - (average depth value of pixels in each divided region / distance D) The distance D is the length of each divided region in the depth direction, and in the case of a ridge, for example, it is the length of the side of the ridge in the depth direction.

[0030] FIG. 7 is an explanatory diagram of LAI and reliability. The left side of FIG. 7 shows a predetermined area 201 extracted from an image of a cultivation area photographed from an oblique angle. FIG. 7 also shows an image of the divided areas 201A, 202B, and 202C viewed from directly above. The image of the divided areas 201A, 202B, and 202C viewed from directly above shows the state of the crops in each of the divided areas 201A, 202B, and 202C. The LAI of divided area 201A is 0.5. If the average depth value of the pixels in divided area 201A is 0.1 and the distance D is 1.0, the reliability of divided area 201A is 1-(0.1 / 1) = 0.9. If the average depth value of the pixels in divided area 201B is 0.5 and the distance D is 1.0, the reliability of divided area 201B is 1-(0.5 / 1) = 0.5. If the average depth value of the pixels in the segmented region 201C is 0.9 and the distance D is 1.0, the reliability of the segmented region 201C is 1-(0.9 / 1)=0.1.

[0031] The LAI of the divided area 201A is 0.5, and the reliability of the divided area 201A is 0.9. The reliability of the divided area 201A is high because the object being photographed is close to the photographing location. Images of the cultivation area photographed from a sideways or oblique angle from a position close to the object being photographed include areas of the cultivation area where no crops are planted, resulting in a small discrepancy between the photographed image and the image. The LAI of the divided area 201B is 0.9, and the reliability of the divided area 201B is 0.5. The LAI of the divided area 201C is 0.8, and the reliability of the divided area 201C is 0.1. The reliability of the divided area 201C is low because the object being photographed is far from the photographing location. Images of the cultivation area photographed from a sideways or oblique angle from a position far from the object being photographed may not include areas of the cultivation area where no crops are planted. In such cases, the discrepancy between the photographed image and the image becomes larger. Thus, the higher the reliability, the smaller the discrepancy between the photographed image and the image, and the lower the reliability, the larger the discrepancy between the photographed image and the image. Therefore, the higher the reliability, the more accurately the LAI represents the actual crop condition, and the lower the reliability, the more likely it is that the LAI does not accurately represent the actual crop condition.

[0032] The method of calculating the reliability is not limited to Equation 1, and for example, the reliability of each partitioned area may be calculated by multiplying the average depth value of the pixels in each partitioned area by a predetermined coefficient.

[0033] In step S17, the output unit 150 outputs the vegetative growth degrees and reliability degrees associated with the multiple segmented regions. If one specified region in the captured image is determined to be one segmented region, the output unit 150 may output the vegetative growth degrees and reliability degrees associated with the single segmented region. The display unit 30 may display the vegetative growth degrees and reliability degrees associated with the multiple segmented regions. If one specified region in the captured image is determined to be one segmented region, the display unit 30 may display the vegetative growth degrees and reliability degrees associated with the single segmented region. The display unit 30 may display the vegetative growth degrees and reliability degrees associated with the multiple segmented regions together with the captured image. For example, the display unit 30 may display the captured image and display the vegetative growth degrees and reliability degrees associated with the multiple segmented regions next to the captured image. For example, the display unit 30 may display the captured image and superimpose the vegetative growth degrees and reliability degrees associated with the multiple segmented regions on the captured image. For example, the display unit 30 may display a map of the farm field, and superimpose a predetermined area divided into a plurality of segmented areas and vegetative growth degrees and reliability degrees associated with the plurality of segmented areas on the map of the farm field. When one predetermined area in the captured image is determined to be one segmented area, the display unit 30 may display the vegetative growth degrees and reliability degrees associated with the one segmented area together with the captured image. The display unit 30 may display the vegetative growth degrees as numbers, colors, or color saturation. The display unit 30 may display the reliability degrees as numbers, colors, or color saturation. Alternatively, when the display unit 30 displays the map of the farm field, the reliability degrees may be displayed as line thicknesses by changing the thickness of the lines indicating the boundaries between the plurality of segmented areas for each segmented area.

[0034] The output unit 150 outputs the vegetative growth degree and reliability associated with each of the multiple segmented regions, and the display unit 30 displays the vegetative growth degree and reliability associated with each of the multiple segmented regions. This allows the worker to easily grasp the overall growth level of the crop. The worker can also grasp how accurately the overall growth level of the crop is grasped. For example, when determining where to perform work on the crop based on the captured image, the worker can accurately grasp the location of the work on the crop. Therefore, the worker can accurately grasp the areas in the field where intervention should be focused or work should be prioritized, improving work efficiency and yield in the field. Furthermore, the cultivation support system 1 allows anyone to easily collect data, and as many users use it and data accumulation progresses, it can contribute to the spread of smart agriculture.

[0035] The output unit 150 may link the vegetative growth degree and the reliability to a plurality of segmented areas using an area identification ID and store them in the storage unit 40. The area identification ID is identification information for identifying a predetermined area and a plurality of segmented areas. The output unit 150 may link the vegetative growth degree and the reliability to a predetermined area and a plurality of segmented areas using an area identification ID and store them in the storage unit 40. The output unit 150 may link the vegetative growth degree and the reliability to a captured image, a predetermined area, and a plurality of segmented areas using an area identification ID and store them in the storage unit 40. The output unit 150 may link the vegetative growth degree and the reliability to a field map, a predetermined area, and a plurality of segmented areas using an area identification ID and store them in the storage unit 40.

[0036] The control unit 20 may calculate an evaluation value based on the vegetative growth degree and reliability for each of the multiple divided areas, and calculate an overall evaluation value (total score) for each of the multiple predetermined areas by adding up the evaluation values ​​calculated for each of the multiple divided areas. The control unit 20 is an example of an evaluation value calculation unit. The vegetative growth degree calculation unit 130, the reliability calculation unit 140, or the output unit 150 may calculate the overall evaluation value. The output unit 150 may output the overall evaluation value linked to each of the multiple predetermined areas. The control unit 20 may calculate the overall evaluation value based on the following formula 2. The control unit 20 may multiply the reliability of each divided area by a specific coefficient. (Formula 2) Overall evaluation value = vegetative growth degree of divided area A × reliability of divided area A + vegetative growth degree of divided area B × reliability of divided area B + ... + vegetative growth degree of divided area Z × reliability of divided area Z

[0037] The display unit 30 may display an overall evaluation value linked to each of the plurality of predetermined areas. The display unit 30 may display the overall evaluation value linked to each of the plurality of predetermined areas together with the captured image. For example, the display unit 30 may display the captured image and display the overall evaluation value linked to each of the plurality of predetermined areas next to the captured image. For example, the display unit 30 may display the captured image and superimpose the overall evaluation value linked to each of the plurality of predetermined areas on the captured image. For example, the display unit 30 may display a map of a farm field and superimpose the plurality of predetermined areas and the overall evaluation value linked to each of the plurality of predetermined areas on the farm field map.

[0038] The output unit 150 outputs the overall evaluation value associated with each of the multiple predetermined areas, and the display unit 30 displays the overall evaluation value associated with each of the multiple predetermined areas. For example, if the crop is strawberries, the predetermined area is a ridge, and if the crop is rice, the predetermined area is a rice field or a predetermined area. By checking the overall evaluation value for each of the multiple predetermined areas, the worker can appropriately identify areas in the field where intervention or work should be prioritized. For example, the worker can use the overall evaluation value as a basis for determining which cultivation areas to focus on when photographing the cultivation areas next. For example, the worker can improve the accuracy of measuring the vegetative growth rate by taking additional photographs of areas with low reliability using a method that increases reliability and updating the vegetative growth rate. Techniques for increasing reliability include taking the next photograph from a closer location than the previous photograph, or taking the next photograph at a higher resolution than the previous photograph.

[0039] The control unit 20 may calculate evaluation values ​​based on the vegetative growth degree and reliability for all partitioned areas included in the plurality of predetermined areas, and calculate an overall evaluation value by adding up the calculated evaluation values. This allows the operator to check the overall evaluation value for the entire field. The vegetative growth degree calculation unit 130, the reliability calculation unit 140, or the output unit 150 may calculate the overall evaluation value. The output unit 150 may output the overall evaluation value linked to the plurality of predetermined areas. The control unit 20 may multiply the reliability of each partitioned area by a specific coefficient.

[0040] The display unit 30 may display an overall evaluation value linked to a plurality of predetermined areas. The display unit 30 may display the overall evaluation value linked to a plurality of predetermined areas together with the captured image. For example, the display unit 30 may display the captured image and display the overall evaluation value linked to the plurality of predetermined areas next to the captured image. For example, the display unit 30 may display the captured image and superimpose the overall evaluation value linked to the plurality of predetermined areas on the captured image. For example, the display unit 30 may display a map of a farm field and superimpose the plurality of predetermined areas and the overall evaluation value linked to the plurality of predetermined areas on the farm field map.

[0041] The above describes an example of a process in which the photographing unit 10 photographs a predetermined area of ​​a field and generates image data including color information and distance information. However, the present invention is not limited to this example. The photographing unit 10 may photograph a predetermined area of ​​a field and generate image data including color information and position information (coordinates X, Y, and Z). That is, the photographing unit 10 may generate a photographed image (3D image) having color information and three-dimensional information (coordinates X, Y, and Z). The coordinates X, Y, and Z indicate the position of a point in a Cartesian coordinate system with a certain point as the origin. The coordinates X, Y, and Z are stored in a point cloud format or a mesh format. The region dividing unit 120 may divide the predetermined area into multiple segmented areas based on position information (position data) included in the image data. For example, the predetermined area may be divided into multiple segmented areas by defining a group of pixels whose coordinates X, Y, and Z of each pixel included in the image data fall within a certain range as one region. Alternatively, the region dividing unit 120 may convert position information included in the image data into distance information and use the converted distance information as an alternative value for the depth value. The reliability calculation unit 140 may calculate the reliability based on the position information included in the image data. For example, the reliability calculation unit 140 may use the distance information converted from the position information by the region dividing unit 120 as an alternative value for the depth value and calculate the reliability based on Equation 1.

[0042] (Veterinary Growth Degree) The above describes a process in which the vegetative growth degree calculation unit 130 calculates the vegetative growth degree based on color information included in image data. However, this process is not limited to this example. The vegetative growth degree calculation unit 130 may calculate the vegetative growth degree for each of a plurality of divided regions based on at least one of the length, area, and volume of the crop in the image and the color information included in the image data. The length of the crop may be, for example, the thickness of the stem at a specific position on the crop. For cherry tomatoes, there is a correlation between the stem diameter at a specific position and the size of the leaves that will grow in the future. Furthermore, the length of the crop may be, for example, the leaf length or width of a specific leaf on the crop. For strawberries, there is a correlation between the length or width of a specific leaf and photosynthetic products.

[0043] The area of ​​a crop is, for example, the size of the leaf area in a certain region. The size of the leaf area in a certain region may be expressed as the leaf area index (LAI). There is a correlation between the size of the leaf area and photosynthetic products. The vegetative growth degree calculation unit 130 may measure the degree of leaf withering. For example, the vegetative growth degree calculation unit 130 may calculate the degree of withering of the crop in each of the multiple divided regions by dividing the number of yellow and / or brown pixels in each divided region by the total number of pixels in each divided region. The vegetative growth degree calculation unit 130 may reduce the LAI of each divided region depending on the degree of withering of the crop in each divided region. Furthermore, the vegetative growth degree calculation unit 130 may calculate a vegetative growth degree for each divided region, including the LAI and the degree of withering of the crop. The area of ​​a crop is, for example, the total volume of the plant in a certain space.

[0044] (Reliability) The above describes the process of calculating reliability based on Equation 1, but this process is not limited to this. The reliability calculation unit 140 may calculate the reliability by increasing or decreasing the value calculated based on Equation 1 depending on the angle (photographing angle) between the line connecting the photographing device (e.g., the photographing unit 10) and the photographed object and the horizontal plane, the direction of gravity, the ground, etc. For example, when photographing a crop such as strawberries from directly above the plant, there is little overlapping of leaves, but when photographing the plant from the side or diagonally, there is a lot of overlapping of leaves. The photographed object includes a specific crop, crops within a segmented area, and the entire photographed range of the photographing device. In the case of a segmented area, the photographing angle may be the angle relative to the center or a representative point of the segmented area, or the average of the angles relative to all points within the segmented area. In the case of the entire photographed range, the photographing angle may be the angle relative to the center or a representative point of the entire photographed range, or the average of the angles relative to all points in the entire photographed range, or simply the angle between the photographing direction of the photographing device and the horizontal plane, the direction of gravity, the ground, etc. The photographing angle may be measured by an acceleration sensor or gyro sensor included in the photographing unit 10. The acquisition unit 110 may acquire the shooting angle at which the captured image was captured from the imaging unit 10. The reliability calculation unit 140 can calculate the reliability according to the shooting angle by increasing or decreasing the value calculated based on Equation 1 according to the shooting angle. If the shooting angle satisfies a predetermined criterion, the reliability calculation unit 140 may increase the value calculated based on Equation 1. If the shooting angle does not satisfy the predetermined criterion, the reliability calculation unit 140 may decrease the value calculated based on Equation 1.

[0045] The reliability calculation unit 140 may calculate the reliability by increasing or decreasing the value calculated based on Equation 1 according to the image resolution of the imaging device (for example, the imaging unit 10). The acquisition unit 110 may acquire the image resolution from the imaging unit 10. The reliability calculation unit 140 can calculate the reliability according to the image resolution of the imaging device by increasing or decreasing the value calculated based on Equation 1 according to the image resolution of the imaging device.

[0046] When the captured image is a 3D model image, the reliability calculation unit 140 may calculate the reliability based on the density of the point cloud included in the captured image. A 3D model image is an image in which a three-dimensional model (3D model) of a captured object is created by capturing multiple images of the captured object from different angles. The more captured images used to create the 3D model image, the clearer the 3D model image can be created. Furthermore, the longer the scanning time required to create the 3D model image, the clearer the 3D model image can be created. When the captured image is a 3D model image, the reliability calculation unit 140 may calculate the reliability based on at least one of the number of captured images used to create the 3D model image and the scanning time required to create the 3D model image.

[0047] Second Embodiment A cultivation support system 1 according to a second embodiment will be described. In the following description, the same components and processes as those in the first embodiment will be denoted by the same reference numerals, and detailed description thereof will be omitted. FIG. 8 is a block diagram showing the configuration of the cultivation support system 1. As shown in FIG. 8, the cultivation support system 1 includes an imaging unit 10, a control unit 20, a display unit 30, and a storage unit 40. The control unit 20 is configured to include an acquisition unit 110, an area division unit 120, a vegetative growth degree calculation unit 130, a reliability calculation unit 140, an output unit 150, and an update unit 160.

[0048] The vegetative growth degree and reliability may be linked to a predetermined area and a plurality of segmented areas using an area identification ID and stored in the storage unit 40. The vegetative growth degree and reliability may be linked to a predetermined area and a plurality of segmented areas using an area identification ID and stored in the storage unit 40. The vegetative growth degree and reliability may be linked to a photographed image, a predetermined area, and a plurality of segmented areas using an area identification ID and stored in the storage unit 40. The vegetative growth degree and reliability may be linked to a field map, a predetermined area, and a plurality of segmented areas using an area identification ID and stored in the storage unit 40.

[0049] (Example of Operation of Cultivation Support System) Fig. 9 is a flowchart showing an example of the operation of the cultivation support system 1 according to the second embodiment. An example of the operation of the cultivation support system 1 according to the second embodiment will be described with reference to the flowchart in Fig. 9. Steps S21 to S26 are similar to steps S11 to S16 in the flowchart in Fig. 2, and therefore their description will be omitted. In step S27, the update unit 160 acquires the vegetative growth degree and reliability and performs update processing.

[0050] FIG. 10 is a flowchart showing an example of the update process. An example of the update process will be described with reference to the flowchart of FIG. 10. In step S31, the update unit 160 determines whether or not the predetermined area in the captured image during current shooting is the same as the predetermined area stored in the storage unit 40. If the predetermined area in the captured image during current shooting is the same as the predetermined area stored in the storage unit 40 (step S31; YES), the process proceeds to step S32. If the predetermined area in the captured image during current shooting is not the same as the predetermined area stored in the storage unit 40 (step S31; NO), the process proceeds to step S28.

[0051] The operator may input information identifying the predetermined area (area identification information) using an input device when or after photographing the cultivation area. A tag or a QR code (registered trademark) may be placed in the field. The update unit 160 may automatically recognize the predetermined area. For example, a tag or a QR code (registered trademark) may be placed in the field, and the update unit 160 may calculate the width and length of the predetermined area based on the tag or the QR code (registered trademark) in the photographed image. The update unit 160 may identify the predetermined area based on the width and length of the predetermined area and output the area identification information. The update unit 160 may identify the predetermined area by pattern matching peripheral information (e.g., characteristics of a greenhouse) in the photographed image and output the area identification information. The area identification information is stored in the memory unit 40. The update unit 160 may determine whether the predetermined area in the photographed image at the current time of photographing is the same as the predetermined area stored in the memory unit 40, based on the area identification information at the current time of photographing and the area identification information stored in the memory unit 40.

[0052] In step S32, the update unit 160 determines whether or not there are any overlapping segmented areas (overlapping areas) between the multiple segmented areas in the captured image at the current time of shooting and the multiple segmented areas stored in the storage unit 40. If there are any overlapping segmented areas (step S32; YES), the process proceeds to step S33. If there are no overlapping segmented areas (step S32; NO), the process proceeds to step S28.

[0053] In step S33, the update unit 160 determines whether the reliability of the overlapping segmented areas in the captured image at the current time of shooting satisfies a predetermined standard. If the reliability of the overlapping segmented areas in the captured image at the current time of shooting is higher than the reliability of the segmented areas stored in the storage unit 40, the update unit 160 determines that the reliability of the overlapping segmented areas in the captured image at the current time of shooting satisfies the predetermined standard. On the other hand, if the reliability of the overlapping segmented areas in the captured image at the current time of shooting is lower than the reliability of the overlapping segmented areas stored in the storage unit 40, the update unit 160 determines that the reliability of the overlapping segmented areas in the captured image at the current time of shooting does not satisfy the predetermined standard. If the reliability of the overlapping segmented areas in the captured image at the current time of shooting satisfies the predetermined standard (step S33; YES), the process proceeds to step S34. If the reliability of the overlapping segmented areas in the captured image at the current time of shooting does not satisfy the predetermined standard (step S33; NO), the process proceeds to step S28.

[0054] In step S34, the update unit 160 updates the vegetative growth degree and reliability of the overlapping segmented areas stored in the storage unit 40 based on the vegetative growth degree and reliability of the segmented areas in the captured image at the time of current capture. After the processing of step S34 is performed, the processing proceeds to step S28. By updating the vegetative growth degree and reliability of the segmented areas stored in the storage unit 40, the vegetative growth degree and reliability of the segmented areas stored in the storage unit 40 become the latest values. This allows the operator to more appropriately grasp the areas in the field where intervention should be focused or work should be prioritized.

[0055] In step S28, the output unit 150 outputs the vegetative growth levels and reliability levels associated with the plurality of segmented regions. Step S28 is similar to step S17 in the flowchart of FIG. 2, and therefore a detailed description thereof will be omitted.

[0056] The update unit 160 may perform a process of lowering the reliability of the segmented area stored in the memory unit 40 when the elapsed time from the time when the segmented area was stored in the memory unit 40 exceeds a predetermined time. As time passes, the length, area, and volume of the crop change due to growth, etc. The update unit 160 may lower the reliability of the segmented area stored in the memory unit 40 by a larger amount as the elapsed time from the time when the segmented area was stored in the memory unit 40 increases. This allows the operator to more appropriately grasp the areas in the field where intervention should be focused or work should be prioritized.

[0057] The present invention may be understood as a cultivation assistance device, information processing device, or information processing system having at least some of the above-described configurations, means, and functions. The present invention may be understood as a cultivation assistance method or information processing method including at least some of the above-described processes. The present invention may be understood as a method in which a computer executes the above-described processes. The present invention may be understood as a program for causing a computer to execute the above-described processes, and the program may be provided to the computer via a network or from a computer-readable recording medium that non-temporarily stores data.

[0058] <Computer-readable recording medium> A program that causes an information processing device or other machine or device (hereinafter referred to as a computer, etc.) to realize any of the above functions can be recorded on a computer-readable recording medium. Then, the computer, etc. can provide the function by reading and executing the program from this recording medium. A computer-readable recording medium refers to a recording medium that stores information such as data and programs through electrical, magnetic, optical, mechanical, or chemical action and can be read by a computer, etc.

[0059] <Supplementary Note 1> A support device (1) comprising: an acquisition unit (110) that acquires an image generated by photographing a crop; a growth degree calculation unit (130) that calculates a vegetative growth degree indicating the degree of vegetative growth of the crop in a divided area within the image; a reliability calculation unit (140) that calculates a reliability indicating the likelihood of the vegetative growth degree in the divided area; and an output unit (150) that outputs the vegetative growth degree and the reliability in association with the divided area. <Supplementary Note 2> The support device (1) according to Supplementary Note 1 further comprises: an area division unit (120) that divides a predetermined area within the image into a plurality of divided areas, wherein the growth degree calculation unit (130) calculates the vegetative growth degree for each of the divided areas, the reliability calculation unit (140) calculates the reliability for each of the divided areas, and the output unit (150) outputs the vegetative growth degree and the reliability in association with the plurality of divided areas. <Supplementary Note 3> The support device (1) according to Supplementary Note 1, wherein the growth degree calculation unit (130) calculates the vegetative growth degree based on at least one of length, area, and volume of the crop in the image and color information of the image. <Supplementary Note 4> The support device (1) according to any one of Supplements 1 to 3, wherein the reliability calculation unit (140) calculates the reliability based on distance information in a depth direction of the image. <Supplementary Note 5> The support device (1) according to any one of Supplements 1 to 3, wherein the reliability calculation unit (140) calculates the reliability based on position information of the image having three-dimensional information. <Supplementary Note 6> The support device (1) according to any one of Supplements 1 to 5, wherein the acquisition unit acquires a shooting angle when the image was captured, and the reliability calculation unit calculates the reliability according to the shooting angle. <Supplementary Note 7> The support device (1) according to any one of Supplementary Notes 1 to 6, further comprising a display unit (30) that displays the vegetative growth level and the reliability linked to the segmented region together with the image.<Supplementary Note 8> The support device (1) according to any one of Supplementary Notes 1 to 7, wherein a plurality of predetermined regions within the image are divided into a plurality of segmented regions, and the support device (1) comprises an evaluation value calculation unit (20) that calculates an evaluation value based on the vegetative growth degree and the reliability for each of the plurality of segmented regions and calculates an overall evaluation value for each of the plurality of predetermined regions by adding up the evaluation values ​​calculated for each of the plurality of segmented regions, and the output unit (150) outputs the overall evaluation value linked to each of the plurality of predetermined regions. <Supplementary Note 9> The support device according to any one of Supplementary Notes 1 to 7, wherein a plurality of predetermined regions within the image are divided into a plurality of segmented regions, and the support device (1) comprises an evaluation value calculation unit (20) that calculates an evaluation value based on the vegetative growth degree and the reliability for all of the segmented regions included in the plurality of predetermined regions and calculates an overall evaluation value by adding up the calculated evaluation values, and the output unit (150) outputs the overall evaluation value linked to each of the plurality of predetermined regions. <Supplementary Note 10> The support device (1) according to any one of Supplementary Notes 1 to 9, comprising: a storage unit (40) that stores the vegetative growth degree and the reliability in association with a plurality of the segmented areas; and an update unit (160) that updates, for a segmented area that overlaps between the plurality of segmented areas in the image at the time the crop was photographed and the plurality of segmented areas stored in the storage unit (40), the vegetative growth degree and the reliability of the segmented area stored in the storage unit (40) based on the vegetative growth degree and the reliability of the segmented area in the image at the time the crop was photographed, if the reliability of the segmented area in the image at the time the crop was photographed satisfies a predetermined standard. <Supplementary Note 11> A support method comprising the steps of: photographing a crop and acquiring an image generated by photographing the crop (S12, S22); calculating a vegetative growth degree indicating the degree of vegetative growth of the crop in a divided area within the image (S15, S25); calculating a reliability indicating the likelihood of the vegetative growth degree in the divided area (S16, S26); and outputting the vegetative growth degree and the reliability in association with the divided area (S17, S28).<Supplementary Note 12> A program for causing a computer to execute the steps of: acquiring an image generated by photographing a crop (S12, S22); calculating a vegetative growth degree indicating the degree of vegetative growth of the crop in a divided area within the image (S15, S25); calculating a reliability indicating the likelihood of the vegetative growth degree in a plurality of the areas (S16, S26); and outputting the vegetative growth degree and the reliability in association with the divided area (S17, S28).

[0060] REFERENCE SIGNS LIST 1 Cultivation support system 10 Photography unit 20 Control unit 30 Display unit 40 Storage unit 110 Acquisition unit 120 Area division unit 130 Nutritional growth degree calculation unit 140 Reliability calculation unit 150 Output unit 160 Update unit

Claims

1. An assistance device comprising: an acquisition unit that acquires an image generated by photographing a crop; a growth degree calculation unit that calculates a nutritional growth degree indicating the degree of nutritional growth of the crop in a divided area within the image; a reliability calculation unit that calculates a reliability indicating the likelihood of the nutritional growth degree in the divided area; and an output unit that outputs the nutritional growth degree and the reliability in association with the divided area.

2. The support device described in claim 1, further comprising an area division unit which divides a specified area within the image into a plurality of the divided areas, wherein the growth rate calculation unit calculates the vegetative growth rate for each of the plurality of divided areas, the reliability calculation unit calculates the reliability for each of the plurality of divided areas, and the output unit outputs the vegetative growth rate and the reliability in association with the plurality of divided areas.

3. The support device according to claim 1, wherein the growth degree calculation unit calculates the vegetative growth degree based on at least one of the length, area, and volume of the crop in the image and color information of the image.

4. The support device according to claim 1, wherein the reliability calculation unit calculates the reliability based on distance information in a depth direction of the image.

5. The support device according to claim 1, wherein the reliability calculation unit calculates the reliability based on position information of the image having three-dimensional information.

6. The support device according to claim 1, wherein the acquisition unit acquires a shooting angle when the image was captured, and the reliability calculation unit calculates the reliability according to the shooting angle.

7. The support device according to claim 1, further comprising a display unit that displays the vegetative growth rate and the reliability associated with the divided area together with the image.

8. The support device of claim 1, further comprising an evaluation value calculation unit that calculates an evaluation value based on the nutritional growth rate and the reliability for each of the plurality ...

9. The support device described in claim 1, wherein a plurality of specified areas in the image are divided into a plurality of the partitioned areas, and an evaluation value calculation unit is provided that calculates an evaluation value based on the nutritional growth rate and the reliability for all the partitioned areas included in the plurality of the specified areas and calculates an overall evaluation value by adding up the calculated evaluation values, and the output unit outputs the overall evaluation value in association with the plurality of the specified areas.

10. An assistance device as described in any one of claims 1 to 9, comprising: a memory unit that stores the vegetative growth degree and the reliability in association with a plurality of the divided areas; and an update unit that updates, for a divided area that overlaps between the plurality of the divided areas in the image at the time the crop was photographed and the plurality of the divided areas stored in the memory unit, based on the vegetative growth degree and the reliability of the divided area in the image at the time the crop was photographed if the reliability of the divided area in the image at the time the crop was photographed satisfies a predetermined standard.

11. An assistance method comprising the steps of: acquiring an image generated by photographing a crop; calculating a vegetative growth degree indicating the degree of vegetative growth of the crop in a divided area within the image; calculating a reliability indicating the likelihood of the vegetative growth degree in the divided area; and outputting the vegetative growth degree and the reliability in association with the divided area.

12. A program for causing a computer to execute the steps of: acquiring an image generated by photographing a crop; calculating a vegetative growth degree indicating the degree of vegetative growth of the crop in a divided area within the image; calculating a reliability indicating the likelihood of the vegetative growth degree in the divided area; and outputting the vegetative growth degree and the reliability in association with the divided area.

Images