Agricultural information processing device, agricultural information processing method, and agricultural information processing program

Abstract

To provide a technology that can effectively cultivate grapes.SOLUTION: Provided is an agricultural information processing device 200 comprising: an image data reception unit 201 that receives the image data of a photographed image obtained by photographing the flower spike part of a grape; a flowering determination unit 204 that determines whether or not the flower spike is in flower based on the photographed image; a full flower determination unit 205 that determines whether or not the flower spikes are in full flower based on the photographed image; and a flower spike shaping presence / absence determination unit 206 that determines whether or not flower spike shaping is performed on the flower spike. The determination by the full flower determination unit 205 is performed when it is determined that the flower spike shaping is performed by the flower spike shaping presence / absence determination unit 206, and the determination by the flowering determination unit 204 is performed when it is determined that no flower spike shaping is performed by the flower spike shaping presence / absence determination unit 206.SELECTED DRAWING: Figure 2

Description

[Technical Field]

[0001] The present invention relates to techniques used in grape cultivation. [Background technology]

[0002] During the grape blossom season, various tasks are concentrated and extremely strict schedule management is required. Therefore, it is important to determine and predict when the grapes will bloom and when they will be in full bloom. Currently, the determination and prediction of when the grapes will bloom and when they will be in full bloom relies on the experience and intuition of skilled workers. Patent Document 1 describes a method for determining when strawberries will bloom based on time-lapse images. Patent Document 2 describes orchard management using UAVs. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2020-54289 [Patent Document 2] Special Publication No. 2020-64663 Summary of the Invention [Problem to be solved by the invention]

[0004] In the cultivation of high-quality grapes, it is necessary to shape the flower spikes from the beginning of flowering until before gibberellin treatment at full bloom, and then to apply gibberellin treatment at a specific time from full bloom to a few days after full bloom. Of course, various other treatments are also necessary, but the timing of the above two treatments is particularly important.

[0005] For example, it would be convenient to be able to determine whether a flower is in bloom or in full bloom from a photographed image. One possible method for doing this would be to use an AI image judgment program created using deep learning to determine whether a flower is in bloom or in full bloom from a photographed image. However, the AI ​​image judgment program referred to here refers to machine learning, such as one that uses learning data and training data, and types other than deep learning can also be used.

[0006] Incidentally, the flower spikes that are candidates for full bloom judgment are those that have been shaped, but because shaping the flower spikes significantly changes the appearance of the flower spikes, it is necessary to use different AI for flower bloom judgment and AI for full bloom judgment. For example, the flower spikes that are the subject of full bloom judgment are those that have been shaped, and naturally, the images used during deep learning are images of flower spikes that have been shaped. Therefore, even if the full bloom judgment AI is asked to judge photographed images of flower spikes that are the subject of bloom judgment, it will judge images that are different from the images it has learned, increasing the possibility of incorrect judgment.

[0007] This inconvenience also occurs when photographs of the flower spikes to be judged as fully bloomed (which, of course, are after flower spike shaping) are judged by AI for flowering judgment (the flower spikes to be judged as fully bloomed are before flower spike shaping).

[0008] Research is also being conducted into technology that uses the germination date as a starting point and statistically predicts the flowering and full bloom dates based on meteorological information. This technology can be used as a guideline for determining the timing of various tasks, but there is a demand for technology with higher accuracy.

[0009] In this context, the present invention aims to provide a technique useful for grape cultivation. [Means for solving the problem]

[0010] The present invention is an agricultural information processing device that includes an image data receiving unit that receives image data of a captured image obtained by photographing a portion of a grapevine flower spike, a flower bloom determination unit that determines whether the flower spike is in bloom based on the captured image, a full bloom determination unit that determines whether the flower spike is in full bloom based on the captured image, and a flower spike trimming determination unit that determines whether the flower spike has been trimmed.If the flower spike trimming determination unit determines that the flower spike has been trimmed, a determination is made by the full bloom determination unit, and if the flower spike trimming determination unit determines that the flower spike has not been trimmed, a determination is made by the flower bloom determination unit.

[0011] The present invention may be embodied in an embodiment comprising a notification unit that issues a notification to encourage spike shaping treatment when the flowering determination unit determines that the plant is in flower. The present invention may be embodied in an embodiment comprising a gibberellin treatment presence / absence determination unit that determines the presence or absence of gibberellin treatment, and a notification unit that issues a notification to encourage gibberellin treatment when the full bloom determination unit determines that the plant is in full bloom and the gibberellin treatment presence / absence determination unit determines that the gibberellin treatment has not been performed.

[0012] In the present invention, whether or not the flower spike is shaped of The determination may be made based on an image of the work of shaping the flower spikes. In the present invention, the image of the work of shaping the flower spikes may include a code display that identifies the work of shaping the flower spikes, and the presence or absence of the work of shaping the flower spikes may be determined based on the code display.

[0013] In one embodiment of the present invention, the presence or absence of gibberellin treatment is determined based on an image of the gibberellin treatment, in which a code indication identifying the gibberellin treatment is captured on the image, and the presence or absence of gibberellin treatment is determined based on the code indication.

[0014] In the present invention, the flowering rate is defined as the ratio of flower buds that have bloomed to all flower buds on the flower spike, and the flowering judgment unit is created by machine learning using image data of a flowering rate of 0% before flower spike shaping and a flowering rate of 1 to 100% before flower spike shaping, and the full bloom judgment unit is created by machine learning using image data of a flowering rate of 1 to 99% after flower spike shaping and a flowering rate of 100% after flower spike shaping.

[0015] In the present invention, a flowering prediction unit is provided that performs flowering prediction on the grapevine based on the germination time and weather information. The flowering prediction is performed individually in each of the first region and the second region. When flowering in the first region is determined by the function of the flowering determination unit, the result of the flowering prediction in the second region is corrected based on the determination of the flowering. Examples of the first region and the second region include the first field and the second field, the first area and the second area, the first part and the second part divided in one field, one side and the other side obtained by dividing a certain area into two, etc. Also, a form in which two or more fields are included in one region is also possible.

[0016] In the present invention, a flowering prediction unit is provided that performs flowering prediction and full bloom prediction on the grapevine based on the germination time and weather information. When flowering is determined by the function of the flowering determination unit, the result of the full bloom prediction is corrected based on the determination of the flowering.

[0017] In the present invention, a position acquisition unit that acquires the position of the grape flower cluster based on the captured image, and a map creation unit that creates a map showing the position of the grape flower cluster in the field based on the position of the grape flower cluster are provided. Examples of the map include those in which the determination results of the flowering determination unit and the full bloom determination unit are displayed.

[0018] In the present invention, a first flowering determination unit that determines whether the flowering rate of the flower cluster is less than X% or not less than X% based on the captured image, a second flowering determination unit that determines whether the flowering rate of the flower cluster is less than Y% or not less than Y% based on the captured image with X < Y, and a flowering rate calculation unit that obtains the flowering rate of the flower cluster based on X and Y are provided.

[0019] In the present invention, a full bloom prediction unit that predicts full bloom based on the change of the flowering rate with time is provided.

[0020] The present invention is an agricultural information processing method that receives image data of a photographed image obtained by photographing a portion of a grapevine flower spike, and based on the photographed image, makes a flowering judgment to determine whether the flower spike is in flower or not, or a full bloom judgment to determine whether the flower spike is in full bloom or not, and judges whether the flower spike is shaped or not, and if it is determined in the judgment whether the flower spike is shaped or not that the flower spike is shaped or not, makes the full bloom judgment, and if it is determined in the judgment whether the flower spike is shaped or not that the flower spike is not shaped, makes the flowering judgment.

[0021] The present invention is a program that is read and executed by a computer, which causes the computer to operate as an image data receiving unit that receives image data of a photographed image obtained by photographing a portion of a grapevine spike, a blooming judgment unit that determines whether the spike is in bloom based on the photographed image, a full bloom judgment unit that determines whether the spike is in full bloom based on the photographed image, and a spike trimming presence / absence judgment unit that determines whether spike trimming has occurred on the spike, and if the spike trimming presence / absence judgment unit determines that spike trimming has occurred, a judgment is made by the full bloom judgment unit, and if the spike trimming presence / absence judgment unit determines that spike trimming has not occurred, a judgment is made by the flowering judgment unit. [Effects of the Invention]

[0022] According to the present invention, a technique useful for grape cultivation can be obtained. [Brief explanation of the drawings]

[0023] [Figure 1] FIG. 1 is a conceptual diagram showing an overview of a farm field. [Figure 2] FIG. 1 is a block diagram of an agricultural information processing device. [Figure 3] This is a conceptual diagram of an AI model for determining whether a flower is in bloom and another for determining whether it is in full bloom. [Figure 4] This is a photograph used as a drawing to show the flowering status. [Figure 5] This is a photograph used as a drawing to show the flowering status. [Figure 6](A) is a photograph showing the flower spike before shaping, and (B) is a photograph showing the flower spike after shaping. [Figure 7] 10 is a flowchart illustrating an example of a processing procedure. [Figure 8] FIG. 1 is a diagram showing a flowering map. DETAILED DESCRIPTION OF THE INVENTION

[0024] 1. First embodiment (Field overview) FIG. 1 is a conceptual diagram showing a portion of a field in which the invention is implemented. FIG. 1 shows a grapevine 100. The trunk 100 is supported on a trellis, and branches 101 are spread out in an H-shape when viewed from above. In addition to the H-shape, there are also other shapes known as I-shape and WH-shape. Numerous branches 102 extend from the branches 101, and multiple leaves 103 and flower spikes 104 are formed on the branches 102. Note that flower spikes 104 are shaped so that a portion of the flower spike remains and a portion is removed during cluster thinning, ultimately forming clusters of grapes.

[0025] Although one trunk 100 is shown in FIG. 1, many trunks are planted in an actual field. Many identification targets 111 are placed on the members (rod-shaped members, wires, etc.) that make up the shelves. The identification targets have a role similar to that of orientation targets in aerial photogrammetry. Technology for placing identification targets in a field is described, for example, in JP 2021-139749 A and JP 2022-4884 A.

[0026] A mobile robot 120 is operating in the field shown in Fig. 1. The mobile robot 120 is equipped with multiple cameras 121 facing in various directions, and patrols the field while taking pictures. This technology is described in, for example, Japanese Patent Application No. 2022-4884.

[0027] By taking the photographs as described above, photographed images of the flower spike 104 are obtained. The photographs are taken so that the nth and n+1th photographed images overlap when viewed with one camera. These two overlapping photographed images become stereo photographic images, making three-dimensional photographic measurement possible.

[0028] Instead of the mobile robot 120, a person can patrol a cart equipped with many cameras by pushing it around, or a person can patrol the site equipped with many cameras (cameras attached to a helmet or work vest) and take pictures. Also, workers performing various tasks can wear cameras on their heads or work vests to take pictures of their hands as they work, and the work progress can be recorded.

[0029] 1 shows an agricultural information processing device 200. The agricultural information processing device 200 is configured using a PC (personal computer) and performs processes related to determining whether the spike 104 is in flower and full bloom, as well as other processes. Details of the processes will be described later.

[0030] (Overview of flowering and full bloom determination) Photographed images of grape flower spikes are used to determine whether the grape flowers are in bloom and whether they are in full bloom. Here, the bloom determination involves determining whether the flowers are in bloom, in other words, whether they have just started to bloom or not. The determination is made using an AI determination model obtained through deep learning. Here, in the cultivation of grapes that is the subject of this invention, flower spikes are shaped after flowering, and some of the flowered parts are removed. For this reason, the appearance of the flower spikes before flower spike shaping (see Figure 6(A)) and after flower spike shaping (see Figure 6(B)) is significantly different.

[0031] Therefore, we will prepare a flowering judgment AI that is specialized in determining whether the flower has bloomed before the flower spike has been shaped, and a full bloom judgment AI that is specialized in determining whether the flower spike has bloomed after the flower spike has been shaped. Details of the flowering judgment AI and full bloom judgment AI will be described later.

[0032] Here, for each flower spike, we use the photographed images to determine whether it is in bloom or in full bloom, but as mentioned above, it is necessary to use different AIs for determining whether it is in bloom or in full bloom.

[0033] Here, when focusing on a particular flower spike, we use AI to determine whether it has bloomed or not and AI to determine whether it is in full bloom, depending on whether the flower spike has been shaped or not.

[0034] The presence or absence of flower spike shaping is determined using captured images. For example, the worker shaping the flower spikes wears a camera on his / her head and a wristband with a barcode on it for identifying the flower spike shaping. In addition, an identification target for image recognition is placed near each flower spike. During the work, the camera repeatedly takes pictures of the worker's hand and the flower spike.

[0035] This captured image is analyzed, and the target flower spike is identified using the above-mentioned identification target. Alternatively, the target flower spike is identified using AI image recognition of the flower spike itself. In addition, the flower spike shaping work is identified from the barcode displayed on the wristband. This allows a determination to be made as to whether or not the flower spike in question has been shaped.

[0036] (Hardware configuration) FIG. 2 is a block diagram of the agricultural information processing device 200 in FIG. 1. The agricultural information processing device 200 is configured using a PC (personal computer). The PC has a CPU, a storage device, a communication interface, a user interface, and other configurations and functions that a normal PC has. It is also possible to prepare a dedicated computer as the agricultural information processing device 200 or to use a data processing server.

[0037] The agricultural information processing device 200 includes an image data receiving unit 201, a work content specifying unit 202, a spike specifying unit 203, a flowering determination unit 204, a full-bloom determination unit 205, a spike shaping presence / absence determination unit 206, a gibberellin treatment presence / absence determination unit 207, a notification unit 208, a flowering prediction unit 209, a position acquisition unit 210, and a map creation unit 211. These functional units are realized by the CPU of the agricultural information processing device 200 executing an operating program. It is also possible to realize some or all of these functional units using dedicated hardware.

[0038] The image data receiving unit 201 receives image data of a photographed image obtained by photographing a portion of a grape flower spike with a camera.

[0039] The work content identification unit 202 identifies the content of the work performed on the specific flower spike to be processed, based on the image data received by the image data reception unit 201. Here, flower spike shaping and gibberellin treatment are identified based on the photographed image of the flower spike.

[0040] The method for specifying the work content will be explained below. First, the specification of inflorescence shaping will be explained. In inflorescence shaping, the tip of the inflorescence is left, for example, 3.5 cm to 4 cm, and the other accessory spikes and pedicels are removed (see Figure 6). Note that the length of the inflorescence tip to be left is not necessarily limited to the above range.

[0041] Inflorescence shaping is carried out by removing unnecessary accessory spikes and peduncles from the inflorescence at the flowering stage to prevent flower shatter (physiological flower drop), stabilize quality, and adjust the cluster shape. Details of inflorescence shaping are described, for example, in JP 2007-75014 A.

[0042] In this embodiment, for example, a worker performing the flower spike shaping wears a wristband on which a barcode identifying the flower spike shaping task is displayed, or the barcode is displayed on a tool used for the flower spike shaping.

[0043] In addition, the worker wears a camera on his head, which captures an image of his hands while he is working on shaping the flower spikes. The barcode is read from the captured image, and the contents of the code are detected. If the code is detected in the captured image, it is recognized that shaping of the flower spikes in the image has been performed. In this way, it is recognized whether or not the work of shaping the flower spikes has been performed based on the captured image.

[0044] Another method is to transmit an identification signal when shaping the flower spike, and compare the reception record with the photographed record taken by the camera, thereby associating the photographed image with the flower spike shaping.

[0045] Next, we will explain how to identify gibberellin treatment. Gibberellin treatment is carried out by immersing the flower spike in a gibberellin solution (it can also be sprayed). In pesticide registration, the object to be immersed in the gibberellin solution is referred to as the "flower cluster," but here we will use the term "flower spike." Gibberellin is a general term for certain plant hormones, and gibberellin treatment at full bloom is essential for seedless grape cultivation in order to eliminate seeds, stabilize berry set, and promote berry enlargement.

[0046] The timing of gibberellin treatment varies depending on the grape variety, but for example, in the case of diploid European varieties intended to eliminate seeds and promote berry size, gibberellin treatment must be carried out within three days of full bloom. If this timing is missed, the effectiveness of seedlessness and stable berry set will be reduced, and there is a high possibility of a deterioration in quality. In addition, the period for gibberellin treatment is regulated by law, which also poses problems.

[0047] The principle of determining whether or not gibberellin treatment has been performed is the same as that for determining spike formation. For example, a worker who performs gibberellin treatment wears a wristband with a barcode indicating that gibberellin treatment has been performed. Alternatively, the barcode is displayed on the tool used to perform gibberellin treatment.

[0048] The worker wears a camera on his head, which captures an image of his hands while he is working on the gibberellin treatment. The barcode is read from the captured image, and the contents of the code are detected. If the code is detected, it is recognized that the gibberellin treatment has been performed on the flower spike in the image. In this way, the gibberellin treatment is identified based on the captured image, that is, a process is performed to recognize whether or not the gibberellin treatment has been performed.

[0049] As another method, an identification signal can be transmitted when gibberellin treatment is carried out, and the received signal can be compared with the photographed image recorded by the camera, thereby associating the photographed image with the gibberellin treatment.

[0050] The flower spike identification unit 203 identifies and specifies the flower spikes that appear in the captured image. Multiple grapevines are planted in the field, and each tree has many new shoots, with flower spikes attached to those shoots. When a specific flower spike (or the part that will become the flower spike) is photographed, it is necessary to identify which part of the field the flower spike is located in before it can be managed as data.

[0051] The identification of the flower spike is performed as follows: First, an identification target is placed around the flower spike (reference numeral 111 in FIG. 1). The flower spike is identified and specified by this identification target that appears in the image.

[0052] For example, suppose that 100 identification targets 111 are arranged on the trunk 100 in Fig. 1, and each identification target is assigned an identification number from 1 to 100. Now, suppose that in a photographed image of a certain flower spike, identification targets with identification numbers 11 and 12 are captured on both sides of the flower spike. In this case, the flower spike is identified as the flower spike located between the identification number 11 and the identification target with identification number 12.

[0053] In this embodiment, a 3D model of the field is created using Structure from Motion (SFM). The above-mentioned identification targets are captured in the captured images used in this SFM, and the position of each identified flower spike can be identified using this. In this way, the location of each flower spike is digitized. This technology is described, for example, in Japanese Patent Application Nos. 2021-210942 and 2022-4884.

[0054] It is also possible to identify flower spikes without using an identification target. In this case, buds (before they become flower spikes) are detected from the captured image, and their positions are identified using SFM. After that, photographs are taken periodically, and the growth from bud to flower spike is tracked in the image. This allows the flower spikes to be identified using their position data. In other words, each flower spike is identified as being at position (X1, Y1, Z1), the flower spike at position (X2, Y2, Z2), etc., and managed digitally.

[0055] A method for digitizing and managing the position of flower spikes (fruits) in a field is described, for example, in Patent Application No. 2021-210942.

[0056] The flowering determination unit 204 determines whether the flowering rate is between 1 and 100% based on the image data received by the image data receiving unit 201. This determination is made using AI image recognition technology created using deep learning. Note that an image determination model obtained by machine learning other than deep learning can also be used.

[0057] Figure 3(A) shows a conceptual diagram of the AI ​​determination model for determining flowering used by the flowering determination unit 204. This AI determination model for determining flowering is created by deep learning using image data before flowering (flowering rate: 0%) and image data after flowering (flowering rate: 1 to 100%) for each flower spike before flower spike shaping. The flowering rate here is defined as the ratio of flower buds that have opened to all flower buds on the flower spike before flower spike shaping.

[0058] Photographs of the flowering process are shown in Figures 4 and 5. The sample images of the flowering process used for deep learning to create an AI judgment model for flowering judgment are those taken before flower spike shaping.

[0059] In this embodiment, flowering is judged to know the timing of spike shaping. Therefore, the target of flowering judgment is the stage before spike shaping. In other words, it is necessary to judge the state of flowering at the stage before spike shaping.

[0060] Figure 6(A) shows the state of the flower spike before flower spike shaping, and Figure 6(B) shows the state of the flower spike after flower spike shaping. As shown in Figure 6, the appearance of the images before and after flower spike shaping is significantly different. Note that the flower spike is left at the base in Figure 6(B) to serve as a marker to indicate whether or not gibberellin treatment has been performed later (this is not necessary in the present invention).

[0061] For the reasons stated above, the training data used for deep learning of the AI ​​judgment model for flowering judgment must be data before flower spike formation.

[0062] The full bloom determination unit 205 determines whether the grapevine flower spike shown in the captured image is in full bloom based on the captured image. The AI ​​determination model for full bloom determination is created by deep learning using data on flowering (1-99%) after flower spike shaping and full bloom (100%) after flower spike shaping. The flowering rate here is defined as the ratio of flower buds that have bloomed to all flower buds on the flower spike after flower spike shaping.

[0063] The full bloom judgment must be made after the flower spike formation, because in order to produce high-quality fruit, it is desirable to complete the flower spike formation before full bloom.

[0064] The spike shaping presence / absence determination unit 206 determines whether spike shaping is being performed at the time of determination for the spike of interest. For example, if spike shaping has been specified (recorded) before the time of determination in the season, it is determined that spike shaping has been performed.

[0065] The gibberellin treatment presence / absence determination unit 207 determines whether or not gibberellin treatment has been performed on the target spike. For example, when this determination is made in the season, if the execution of gibberellin treatment has been specified (recorded) before, it is determined that gibberellin treatment has been performed.

[0066] The notification unit 208 issues notifications to encourage the implementation of spike shaping treatment and gibberellin treatment. The timing of notifications will be described later.

[0067] The flowering prediction unit 209 predicts the flowering date based on the germination date and meteorological conditions related to temperature and sunlight. This prediction model is described, for example, in "Naohiko Motonaga, Shinichi Fujimaki, and Tatsuya Matsumoto [Method for Predicting Flowering of Kyoho Grapes]," Niigata Prefectural Agricultural Research Institute Research Report No. 2 (March 20, 2000), pp. 71-72; Mamoru Sato and Kunio Takezawa [Prediction of the Germination, Flowering, and Veraison Periods of the Tetraploid Seedless Grape "Azumashizuku"], Horticultural Research No. 13 (September 30, 2014), pp. 193-201; and Masahiro Uemori, Yuka Miwa, Takeshi Isobe, and Akihiro Hosomi [Prediction Model for the Germination and Full-Bloom Dates of Delaware Grapes Based on Daily Average Temperature], Horticultural Research No. 19 (June 30, 2020), pp. 175-181."

[0068] The position acquisition unit 210 acquires the position of the flower spike in the captured image using the SFM principle. This technology is described in, for example, Japanese Patent Application Nos. 2021-210942 and 2022-4884.

[0069] An example of processing in the position acquisition unit 210 will be described below. Each flower spike is photographed from multiple different viewpoints so that the photographed objects overlap in the photographed images. Here, assume that a certain flower spike is photographed from two viewpoints, and two photographed images are obtained. These two photographed images become stereo images, and the relative positional relationship between the flower spike and the two camera positions is determined based on the principle of stereophotometry. This corresponds to relative orientation in photogrammetry.

[0070] If these two images are included in a large number of images used for SFM, and if these images contain multiple reference targets whose positions in an absolute coordinate system are identified, then the scale and absolute coordinates are given to the relative positional relationship. This corresponds to absolute orientation in photogrammetry.

[0071] This technology is described in, for example, Japanese Patent Application Laid-Open No. 2013-186816. Photogrammetry using identification targets in farm fields is described in Japanese Patent Application Laid-Open No. 2021-139749.

[0072] The absolute positional relationship between the flower spike and the two camera positions is determined, thereby determining the position of the flower spike in the field. This process is performed by the position acquisition unit 209.

[0073] The map creation unit 211 creates a flowering map, an example of which is shown in Figure 8. The flowering map in Figure 8 is updated at appropriate times. The flowering map in Figure 8 displays the flowering status of each flower spike in the field at that time. In addition, this flowering map displays information regarding flower spike shaping and gibberellin treatment.

[0074] The agricultural information processing device 200 also includes a storage unit 212 and a communication unit 213. The storage unit 212 is a storage device (hard disk storage device or semiconductor memory) of the PC used, and stores the operation program of the agricultural information processing device 200 (a program for executing the functions of each functional unit in FIG. 2), data required for operation, data obtained during and as a result of operation, and other data. An external storage device can also be used as the storage unit 212.

[0075] The communication unit 213 is a communication interface of the PC to be used, and performs communication with external devices using a wireless LAN or a telephone line.

[0076] (Processing Procedure) An example of the processing procedure is shown below. Fig. 7 is a flowchart showing an example of the processing. A program for executing this flowchart is stored in a PC constituting the agricultural information processing device 200 and executed by the PC. This program can also be stored in an applicable storage medium or a server on the Internet and downloaded from there for use.

[0077] The following process is performed as a prerequisite for the process in Figure 7. First, as explained in relation to Figure 1, a mobile robot equipped with a camera is patrolled to take pictures and acquire image data of images of workers at work. The mobile robot takes pictures repeatedly, such as once or twice a day.

[0078] Every time new image data is obtained, a process of identifying and specifying each spike is performed by the function of the spike identification unit 203. As a result, the images of each spike are associated and converted into data.

[0079] Then, each time new image data is obtained, the process in Figure 7 is performed for each spike. There is also a method of performing the process in Figure 7 at set intervals, such as once or twice a day. In this case, the process in Figure 7 is performed using the latest image data at that time.

[0080] When the process starts, first, image data of the photographed image of the flower spike to be processed is acquired (step S101). Next, the image data received in step S101 is analyzed to identify the work content (step S102). This process is performed by the work content identification unit 202 in FIG. 2. Note that if the image does not show any work to be identified, no particular work content is identified in this process. In this case, the result of the process is "no work identified."

[0081] Next, it is determined whether there is a record of spike shaping already being performed (step S103). That is, it is determined whether there is a record of a judgment that "spike shaping has been performed" at this point in time for the target spike. If spike shaping is identified in step S102 of the flow, the judgment in step S103 will be "spike shaping has been performed." The judgment in step S103 is performed by the spike shaping presence / absence judgment unit 206 in FIG. 2.

[0082] If spike shaping has been performed, proceed to step S104; if not, proceed to step S108. In S104, it is determined whether or not there is a record of a determination that "gibberellin treatment has been performed" at this point. Note that if gibberellin treatment has been identified in step S102 of the flow, the determination in step S104 will be "gibberellin treatment has been performed." The processing of step S104 is performed by the gibberellin treatment presence / absence determination unit 207 in FIG. 2.

[0083] If it is determined in step S104 that "gibberellin treatment was performed," the process ends and the start of the next flow is awaited.

[0084] If it is determined in step S104 that "gibberellin treatment was not performed," the process proceeds to step S105. In step S105, a determination is made as to whether or not the flower spikes shown in the captured image are in full bloom (full bloom determination) based on the image acquired in step S101. This process is performed by the full bloom determination unit 205 in FIG. 2.

[0085] If the result of the determination is not full bloom, the process ends and waits for the start of the next flow. If the result of the determination is full bloom, the process proceeds from step S106 to step S107, and an alarm process is performed to prompt the operator to perform gibberellin treatment. For example, an alarm signal is sent to a smartphone carried by the operator, and an alarm sound is output from the smartphone and an alarm screen is displayed. After step S107, the process ends and the system prepares for the next process.

[0086] The notification screen in step S110 may display the "full bloom date" and the "gibberellin treatment deadline." For example, in the case of a diploid European variety cultivated and treated twice with gibberellin, the deadline for the gibberellin treatment at full bloom is set to within three days of full bloom.

[0087] When the process proceeds from step S103 to step S108, a flowering determination is performed using an AI determination model for determining flowering on the image acquired in step S101. This process is performed by the flowering determination unit 204 in FIG. 2. In this example, the flowering determination determines whether the flowering rate is between 1 and 100%. If the determination is that the flowering has occurred (the flowering rate is between 1 and 100%), the process proceeds from step S109 to step S110; if not, the process ends and preparations for the next process are made.

[0088] In step S110, a notification process is performed to prompt the user to perform the spike shaping process, and the process is then terminated. The process in step S110 is performed by the notification unit 208 in FIG.

[0089] In the notification process in step S110, for example, a notification signal is sent to a smartphone carried by the worker, and a notification sound is output from the smartphone and a notification screen is displayed.

[0090] The notification screen may also display the "flowering confirmation date" and "flower spike shaping deadline." It is preferable to shaping flower spikes as soon as possible after flowering. The flower spike shaping deadline is set, for example, to the flowering date + 2 days.

[0091] (superiority) By using a judgment model for determining whether grapes have bloomed and a judgment model for determining whether grapes have fully bloomed, the accuracy of determining whether grapes have bloomed and full bloom can be improved.

[0092] Furthermore, the timing of flower spike shaping and gibberellin treatment is extremely important, but even on a single tree, the timing of flowering and full bloom can vary depending on the flower spike. Even in this case, by repeatedly performing the process shown in Figure 7 on each flower spike, flowering and full bloom can be detected without missing any, and flower spike shaping and gibberellin treatment can be performed on each flower spike at the appropriate time. This also avoids the problems of performing flower spike shaping before the plant has bloomed or performing gibberellin treatment before full bloom.

[0093] 2. Second embodiment For example, suppose there are six fields in a certain region, designated Field 1 to Field 6. Here, the range of the region is selected as a range where the weather conditions do not vary greatly. However, since the weather conditions of areas separated by mountains or the sea can differ greatly, it is not desirable to treat these as a single region, and they should be considered as separate regions.

[0094] In this embodiment, a prediction model for grapes is used to predict the flowering date using germination dates and meteorological information. Naturally, the prediction model is created for a target grape variety. This process is performed by the flowering prediction unit 209 in Figure 2.

[0095] Here, the flowering date is defined as the "date when flowering was first confirmed" in the target area (for example, the target field). It is also possible to define the flowering date as a date when the flowering rate is 10% or 20%. This is a matter of definition. It is also possible to create a model to predict the full bloom date. The method for defining the full bloom date is the same as for the flowering date.

[0096] In this embodiment, meteorological elements are measured by sensors in each of fields 1 to 6. Using these measured values, the flowering date is predicted for each of fields 1 to 6 using the prediction model described above.

[0097] Flowering prediction is performed as follows. First, germination is detected using images captured by a camera. The detection of germination is based on the time when germination is detected in each of fields 1 to 6. Once germination is detected, the date and time are entered into the prediction model. This is performed for each of fields 1 to 6.

[0098] At the initial germination stage, subsequent meteorological factors are unknown, so average values ​​based on past meteorological data for the region are input into the prediction model to determine the predicted flowering date. As days pass, actual measurement data is obtained. Therefore, at specific intervals, such as every seven days, the actual measurement data is input into the prediction model and a new prediction is made. The interval between re-predictions can be narrowed as the predicted flowering date approaches.

[0099] Meanwhile, in each field, flowering is determined by image determination as explained in the first embodiment. Here, if flowering is determined in any field, that date is designated as the flowering date. This flowering date is then compared with the predicted flowering date for that field at that time, and the difference is calculated. For example, if the flowering date is May 1st and the predicted flowering date is May 3rd, a correction value of -2 days is calculated. The correction value is negative if the predicted flowering date is later than the flowering date, and positive if earlier.

[0100] This correction value is then used to correct the predicted values ​​for the other fields. For example, suppose that in the first field, the flowering date is May 1st and the predicted flowering date is May 3rd. In this case, the predicted flowering dates for each of the second to sixth fields are set to -2 days. In other words, the predicted flowering dates for each of the fields 2 to 6 are brought forward by two days.

[0101] The above process is performed for the following reasons. First, fields 1 to 6 were selected as areas where weather conditions do not vary significantly. However, due to differences in elevation, topography, and sunlight exposure, predictions are made for each of fields 1 to 6 individually.

[0102] Incidentally, flowering is thought to be influenced by a variety of factors other than meteorological elements. One such factor is diurnal changes in the weather environment. Diurnal changes in the weather environment could mean cloudy and cold in the morning, but sunny and hot in the afternoon.

[0103] Here, it is unlikely that there will be a large difference in weather between Fields 1 to 6. For example, it is unlikely that Field 1 will have sunny weather and Field 2 will have rainy weather throughout the day.

[0104] That is, it can be assumed that the weather changes are similar in fields 1 to 6. Therefore, it can be assumed that the error factors in the predicted flowering date in field 1 are not significantly different from those in the other fields.

[0105] Therefore, it is assumed that the difference between the predicted flowering date in the field where flowering was first determined and the actual flowering date will also occur in other fields, and the predicted flowering date is corrected as described above.

[0106] In this embodiment, an example is given in which there are six fields, but there is no limit to the number of fields. The unit for flowering prediction is not limited to the field unit. For example, it is possible to divide a large field with differences in elevation into multiple areas and make flowering predictions individually for each divided area. This is effective when there are differences in the weather environment due to differences in elevation. It is also possible to make predictions individually for each section of the field that has differences in sunlight.

[0107] In this embodiment, assuming two regions with similar weather conditions, it is also possible to use the results of flowering determination in the first region to correct the flowering prediction in the second region. In this embodiment, an example has been described in which the flowering date is the target, but the method of this embodiment can also be applied to a specific flowering rate, full bloom date, or specific full bloom rate. This is also the case in the third embodiment described below. The full bloom rate is defined as the ratio of flower spikes that are in full bloom to all flower spikes in the range to be determined.

[0108] 3. Third embodiment For example, suppose there is an area where grape cultivation is popular and there are many fields for grape cultivation there. In reality, it may be difficult to implement the first embodiment in all the fields in the area from a cost perspective. An example of applying the present invention to such a case will be described below.

[0109] First, the field in the region that empirically has the earliest flowering is selected, and the first embodiment is implemented in that field (hereinafter referred to as the flowering judgment field). Furthermore, meteorological elements and germination dates are measured in each field, including the flowering judgment field, and flowering predictions are made as described in the second embodiment. Flowering predictions may also be made collectively for fields with similar environments.

[0110] When flowering is determined in the flowering determination field, the difference between that day (flowering date) and the predicted flowering date is evaluated to obtain the correction value described in the second embodiment. This correction value is then used to correct the predicted flowering dates of other fields.

[0111] Although this method is affected by differences in weather conditions within the region, it can achieve real-time correction of predicted values ​​based on actual measurements at a relatively low cost.

[0112] This embodiment can also be applied to regions instead of fields. For example, suppose there are two regions: a △□ region in XX prefecture and a ◇▽ region in XX prefecture. The two regions have similar (or can be considered to have similar) weather conditions, and it is known statistically that grapes bloom earlier in the △□ region than in the ◇▽ region.

[0113] In this case, the first embodiment is implemented in the △□ region to determine flowering. Also, flowering prediction is performed in the ◇▽ region. The results of the flowering determination in the △□ region are then used to correct the flowering prediction in the ◇▽ region. This embodiment can also be applied to the flowering rate, full bloom date, and full bloom rate.

[0114] 4. Fourth Embodiment It is assumed that the prediction model of the second embodiment predicts the flowering date and full bloom date of grapes in a farm field, and that the flowering determination described in the first embodiment is performed in this farm field.

[0115] In this case, let's say the predicted flowering date is April 25th, and the flowering date determined by flowering judgment is April 23rd. In this case, since the predicted date is two days later than the actual measurement date, the prediction is considered to be two days later, and the predicted full bloom date is shifted by -2 days (two days earlier). For example, if the predicted full bloom date is May 4th, the predicted full bloom date is set to May 2nd.

[0116] Here, it is assumed that the factors that caused errors in the prediction of the flowering date also affect the prediction of full bloom, and that errors similar to those in the prediction of the flowering date also occur in the prediction of full bloom, and the above processing is carried out.

[0117] 5. Fifth Embodiment According to the first embodiment, the location in the field, the flowering date, and the full-bloom date for each flower spike are known. An example of mapping this information is shown in Figure 8. Figure 8 shows an example of a flowering map as of May 8th and an example of a flowering map as of May 12th.

[0118] 6. Sixth Embodiment Below, we will explain an example of predicting the full bloom of grape spikes using captured images. First, we obtain the relationship based on statistical data between the time from the flowering date to full bloom and the flowering rate for the grapes of the target variety. This relationship can be found as a relational equation (y = f(x)) where the horizontal axis is elapsed time (variable x) and the vertical axis is the flowering rate y. This relational equation is found using statistical data.

[0119] In addition to the AI ​​judgment model for determining flowering and full bloom described in the first embodiment, we also provide a first AI judgment model for determining flowering that determines whether the flowering rate is above or below 10%, a second AI judgment model for determining flowering that determines whether the flowering rate is above or below 20%, a third AI judgment model for determining flowering that determines whether the flowering rate is above or below 30%, a fourth AI judgment model for determining flowering that determines whether the flowering rate is above or below 40%, and a fifth AI judgment model for determining flowering that determines whether the flowering rate is above or below 50%.

[0120] The first AI judgment model for determining whether the flowering rate is above or below 10% is created by deep learning using image data of flowering rates below 10% (flowering rate of 1% to 9%) and flowering rates of 10% or above (flowering rate of 10% to 100%). In other words, the AI ​​judgment model for determining whether the flowering rate is above or below X% is created by deep learning using image data of flowering rates below X% (1% to (X-1)%) and flowering rates of X% or above (X% to 100%). All image data used has been shaped from flower spikes.

[0121] Here, image data determined to have flower spikes shaped is subjected to a determination using an AI determination model for determining each of the above flower spike flowering rates. For example, suppose a determination using the first AI determination model for flower spike flowering on a photographed image of a certain flower spike determines that the flower spike flowering rate is less than 10%. In this case, the flower spike flowering rate is in bloom but is less than 10%. In this case, the intermediate value is used as the representative value, and the flower spike flowering rate is set to 5%.

[0122] Also, suppose that the first AI model for determining flowering determines that the flowering rate is 10% or higher, and the second AI model for determining flowering determines that the flowering rate is less than 20%. In this case, the flowering rate of the inflorescence is 10% to less than 20%. In this case, the flowering rate of the inflorescence is set to 15%.

[0123] This determination is repeated continuously, and a flowering rate plot is created with the time elapsed since flowering began on the horizontal axis and the flowering rate on the vertical axis. In the above case, this flowering rate plot is a diagram with plot points for flowering rates of 5%, 15%, 25%, 35%, and 45%.

[0124] This flowering rate plot is then compared with the curve shown by the relational equation y=f(x) calculated from the statistical data described above, and the full-bloom date is predicted by extrapolation. For example, let's assume that the relational equation y=f(x) is a linear function (a linear equation). In that case, we assume that the flowering rate plot is approximated by a straight line, and predict the full-bloom date by extrapolation.

[0125] In this way, for each flower spike, the full bloom date can be predicted based on the photographed images when the flowering rate is less than 50%. It is also possible to predict full bloom based on a lower flowering rate. However, in this case, the error will be larger. On the other hand, it is also possible to predict full bloom based on a higher flowering rate. In this case, the accuracy of the full bloom date prediction will be higher, but since the prediction is made when the full bloom date is approaching, the significance of the prediction in terms of advance prediction is reduced.

[0126] The slope of the line fitted to the flowering rate plot of this embodiment represents the flowering rate of the target flower spike from flowering to full bloom. Of course, the flowering rate is not necessarily constant. By combining this flowering rate with other observation parameters such as meteorological factors, it is possible to make various predictions, including the full bloom date.

[0127] When this embodiment is implemented, a flowering determination unit corresponding to the determination of each flowering rate is provided in the agricultural information processing device 200 of Fig. 2. In addition, a flowering rate calculation unit is provided that calculates the flowering rate of the inflorescence captured in the captured image based on the two determined flowering rates. For example, in the above case, for an inflorescence having a flowering rate of 10% or more and less than 20%, the flowering rate calculation unit calculates that the flowering rate of this inflorescence is between 20% and less than 30%, and sets 25% as the representative value.

[0128] In addition, a full bloom prediction unit is provided that predicts full bloom based on the change in the flowering rate over time (flowering speed). This full bloom prediction unit predicts the full bloom date based on, for example, the flowering rate plot chart and the relational expression y=f(x) calculated from statistical data.

[0129] An application of this embodiment is to display the predicted full bloom date on the flowering map of FIG.

[0130] 7. Other embodiments It is also possible to implement one or more of the above-described embodiments in combination. [Explanation of symbols]

[0131] 100...grape trunk, 101...grape branch, 102...branch, 103...leaf, 104...flower spike, 120...mobile robot, 121...camera.

Claims

1. an image data receiving unit that receives image data of a photographed image obtained by photographing a portion of a grape flower spike; a flowering determination unit that determines whether the flower spike is in a flowering state based on the captured image; a full bloom determination unit that determines whether the flower spike is in full bloom based on the captured image; a spike trimming presence / absence determination unit that determines whether or not the spike is trimmed; Equipped with When the presence / absence of flower spike shaping determination unit determines that the flower spike is shaped, the full bloom determination unit performs a determination, An agricultural information processing apparatus in which, when the presence / absence determination unit for determining whether or not the flower spike is shaped has determined that the flower spike is not shaped, a determination is made by the flowering determination unit.

2. The agricultural information processing device according to claim 1 , further comprising a notification unit that issues a notification to prompt a user to perform a spike shaping process when the determination by the flowering determination unit indicates that the plant is in flower.

3. a gibberellin treatment presence / absence determination unit for determining the presence / absence of gibberellin treatment; a notification unit that notifies the user to perform gibberellin treatment when the full bloom determination unit determines that the plant is in full bloom and the gibberellin treatment presence / absence determination unit determines that the plant is not in gibberellin treatment; The agricultural information processing device according to claim 1 or 2, comprising:

4. 4. The agricultural information processing device according to claim 1, wherein the determination of whether or not the flower spikes have been shaped is made based on an image of the flower spike shaping work.

5. 5. The agricultural information processing device according to claim 4, wherein the image of the work of shaping the flower spikes shows a code display for identifying the work of shaping the flower spikes, and whether or not the work of shaping the flower spikes has been performed is determined based on the code display.

6. The agricultural information processing device according to claim 3, wherein the determination of whether or not the gibberellin treatment has been performed is made based on an image obtained by photographing the gibberellin treatment work.

7. 7. The agricultural information processing device according to claim 6, wherein the image of the gibberellin treatment includes a code display identifying the gibberellin treatment, and the presence or absence of the gibberellin treatment is determined based on the code display.

8. The flowering rate is the ratio of flower buds that have opened to the total flower buds in the flower spike. The flowering judgment unit is created by machine learning using image data of a flowering rate of 0% before flower spike shaping and a flowering rate of 1 to 100% before flower spike shaping, The agricultural information processing device according to any one of claims 1 to 7, wherein the full bloom determination unit is created by machine learning using image data of a flowering rate of 1 to 99% after flower spike shaping and a flowering rate of 100% after flower spike shaping.

9. a flowering prediction unit that predicts flowering of the grapevine based on germination time and weather data; The flowering prediction is performed separately for each of the first region and the second region; 9. The agricultural information processing device according to claim 1, wherein when the flowering determination unit determines that flowering has occurred in the first region, the result of the flowering prediction in the second region is corrected based on the determination of flowering.

10. a flowering prediction unit that predicts flowering and full bloom of the grapevine based on germination time and weather information; 9. The agricultural information processing device according to claim 1, wherein when flowering is determined by the function of the flowering determination unit, the result of the full bloom prediction is corrected based on the determination of flowering.

11. a position acquisition unit that acquires the position of the grape spike based on the captured image; a map creation unit that creates a map indicating the positions of the grapevine flower spikes in the field based on the positions of the grapevine flower spikes; Equipped with The agricultural information processing device according to any one of claims 1 to 10, wherein the map displays the results of the determination by the flowering determination unit and the results of the determination by the full bloom determination unit.

12. a first flowering determination unit that determines whether the flowering rate of the spike is less than X% or equal to or greater than X% based on the captured image; a second flowering determination unit that determines whether the flowering rate of the spike is less than Y% or Y% or more based on the photographed image, assuming that X<Y; a flowering rate calculation unit that calculates the flowering rate of the spike based on the X and the Y; The agricultural information processing device according to any one of claims 1 to 11, comprising:

13. The agricultural information processing device according to claim 12 , further comprising a full bloom prediction unit that predicts full bloom based on the change in the flowering rate over time.

14. Accepting image data of photographed images of grape flower spikes, Based on the photographed image, a blooming determination is made as to whether the flower spike is in bloom or not, or a full bloom determination is made as to whether the flower spike is in full bloom or not; Determining whether or not the spike is shaped; In the determination of whether or not the flower spike is shaped, if it is determined that the flower spike is shaped, the determination of full bloom is made; An agricultural information processing method in which the flowering determination is performed when it is determined that the flowering is not formed in the determination of whether or not the flower spike is formed.

15. A program to be read and executed by a computer, Computer an image data receiving unit that receives image data of a photographed image obtained by photographing a portion of a grape flower spike; a flowering determination unit that determines whether the flower spike is in a flowering state based on the captured image; a full bloom determination unit that determines whether the flower spike is in full bloom based on the captured image; a spike trimming presence / absence determination unit that determines whether or not the spike is trimmed; and run it, When the presence / absence of flower spike shaping determination unit determines that the flower spike is shaped, the full bloom determination unit performs a determination, The agricultural information processing program includes a step of causing the flowering determination unit to make a determination when the unit for determining whether or not the flower spike is shaped determines that the flower spike is not shaped.

Images