Information processing device, information processing method, and program

Abstract

This information processing device calculates a rate relating to plant growth on the basis of sensing data, and performs control for displaying sample position images, which are arranged in accordance with sensing positions of respective pieces of sensing data, in a map area on a user interface screen and in image modes which are determined in accordance with the rates calculated from the respective pieces of sensing data.

Description

[Technical field]

[0001] The present technology relates to an information processing device, an information processing method, and a program, and in particular to a technology for presenting information suitable for managing a farm field. [Background technology]

[0002] In recent years, as farms have become larger, automatic planting of crops has become common. In automatic planting, it is very important to check whether the crops have been planted correctly and whether they are growing as expected. If there are areas where the crops have not been planted or grown as expected, the farm staff must decide whether or not to replant. Replanting means replanting (re-seeding, etc.) areas that have already been planted.

[0003] On the other hand, with the development of aerial imaging technology using cameras mounted on drones and other flying objects, attempts are being made to simplify and automate the work of manually inspecting farmland on large-scale farms. Detection of planting defects as mentioned above is also becoming automated using such aerial imaging technology. Patent Document 1 discloses a technique for capturing images of farm fields and performing remote sensing of vegetation conditions. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] Patent No. 5162890 Summary of the Invention [Problem to be solved by the invention]

[0005] Incidentally, although it is possible to check for planting defects from captured images, in reality, even if there are defects, the degree of the defects varies from place to place, and not all places require replanting. Also, replanting naturally incurs costs, and even if the number of seeds that germinate is slightly less than expected, there are cases where it is better to let the seeds grow as they are without replanting. Of course, there are also cases where replanting is better even if it incurs costs.

[0006] There is also a demand to make the decision to replant at an early stage, because, for convenience during harvesting, it is desirable to harvest both the already sprouted parts and the parts to be replanted without causing too much difference in crop growth.

[0007] In view of the above, the present disclosure proposes a technique for presenting appropriate information for a prompt replant decision. [Means for solving the problem]

[0008] The information processing device according to the present technology includes a calculation unit that calculates a rate related to plant growth from sensing data, and a user interface control unit that performs control to display sample position images arranged according to the sensing positions of each sensing data in a map area of ​​a user interface screen in an image form determined according to the rate calculated from each sensing data. The sensing data is, for example, various data obtained by sensing a farm field. For example, the sensing data is image data obtained by capturing an image of the farm field, and data from sensors such as a thermosensor and an ultrasonic sensor. The farm field broadly includes farmland where crops are cultivated, such as cultivated land, hydroponic cultivation land, and greenhouse cultivation land. The rate of plant growth is rate information about plant growth in a broad sense, such as the rate of crops that have germinated after planting, or the rate of crops that are growing normally. Note that "crops" refers to things that are harvested artificially, but "plants" here also include other things such as fruit trees and weeds. The sample position image is an image for user interface display showing the position on the map image where the image data of the sample was captured or the captured position within the field. This sample position image is displayed in an image mode according to the rate. The image mode refers to elements that make up the sample position image, such as color, brightness, shape, and contour line. The sample position image displayed as an image in the user interface may be an image of text or numerical values, an image of color only, or any other mode.

[0009] In the information processing device related to the above-mentioned present technology, the sensing data is image data of a farm field, and the rate is considered to be the ratio between the number of crops per unit area determined from the image data and the number of planting instructions per unit area for the imaging area of ​​the image data. When planting in a field, information (instruction number of plantings) instructing the number of seeds / stalks / seedlings of crops to be planted per acre for each location in the field where planting is to be performed is given to the tractor, and planting is performed. The ratio of the number of crops per unit area that have actually germinated / grown and are calculated based on image data to such instruction number of plantings is regarded as the rate of crop growth.

[0010] In the information processing device related to the above-mentioned present technology, the sensing data is image data of a farm field, and the rate is considered to be the ratio between the number of crops per unit area determined from the image data and the actual number of plantings per unit area for the imaging area of ​​the image data. The ratio of the number of crops per unit area calculated based on the image data to the number of crops actually planted by the tractor is taken as the rate of crop growth.

[0011] In the information processing device related to the above-mentioned technology, the sensing data is image data of a farm field, and it is possible to select whether the rate is the ratio between the number of crops per unit area calculated from the image data and the number of planting instructions per unit area for the imaging area of ​​the image data, or the ratio between the number of crops per unit area calculated from the image data and the number of actual plantings per unit area for the imaging area of ​​the image data. In other words, the user can select whether the standard value for the rate of germination rate is the number of seeds planted by the tractor or the number of seeds planted by the tractor.

[0012] In the information processing device related to the above-mentioned present technology, the user interface control unit may control the display of a rate map image in the map area, the rate map image showing differences in values ​​used to calculate the rate within the field, together with the sample position image. For example, the planting instruction number for a tractor or the actual planting number, which are the base values ​​for the rate, may differ for each location in the field. A rate map image, which is an image to display this on the user interface, is displayed together with the sample position image. The rate map image is also an image displayed on the user interface, and may be an image of text or numbers, an image of only color, or any other type of image.

[0013] In the information processing device according to the present technology described above, it is considered that the user interface control unit performs control to display, in the map area, an area definition image that defines areas obtained by dividing a farm field. As the area definition image, for example, an image such as a grid pattern is displayed in the map area, allowing the user to recognize the definition of each area into which the field is divided.

[0014] In the information processing device relating to the above-mentioned present technology, it is considered that the user interface control unit performs control so that each area shown in the area definition image is displayed in an image format determined according to the rate calculated for that area. Each area defined by the area definition image is displayed in an image mode corresponding to the rate. In this case, the image mode refers to elements constituting the area definition image, such as color, brightness, contour line, grid pattern, etc.

[0015] In the information processing device related to the above-mentioned present technology, it is considered that the calculation unit calculates the rate of each area indicated in the area definition image by an interpolation operation using a crop growth rate for the sensing data related to that area. The rate for each area defined by the area definition image is calculated using the rate for sensing data such as associated image data captured near the center point of the area.

[0016] In the information processing device related to the present technology described above, the sensing data related to a certain area, which is used to refer to a rate related to crop growth in the interpolation calculation, is considered to be sensing data sensed within a predetermined distance from the center point of the certain area. For example, the rate for the area is calculated by an interpolation calculation using the rate for image data captured within a predetermined distance from the center point of the area.

[0017] In the information processing device related to the above-mentioned present technology, the sensing data related to a certain area, which is referenced in the interpolation calculation for the rate related to crop growth, may be a predetermined number of sensing data in order of imaging area proximity to the center point of the certain area. The rate of the imaging area is calculated by an interpolation calculation using the rates for a predetermined number of sensing data in ascending order of distance from the center point of the area.

[0018] In the information processing device relating to the above-mentioned present technology, it is considered that the calculation unit performs the interpolation calculation by weighting the rate for sensing data related to a certain area, the sensing data sensing an area having the same number of planting instructions or number of planting records as the certain area. If there is related sensing data, for example image data of an area with the same planting instructions or planting results as the area in question, an interpolation calculation is performed by weighting the rate for that image data to calculate the rate for the area.

[0019] In the information processing device according to the present technology described above, it is considered that the user interface control unit performs interface control for performing a setting operation of the area definition image. For example, the size of the area defined in the area definition image can be specified by the user.

[0020] In the information processing device according to the present technology described above, it is considered that the user interface control unit performs interface control for performing an operation of designating a replant candidate area for each area defined in the area definition image. For example, the user may be allowed to designate any area as a replant candidate on the area definition image.

[0021] In the information processing device related to the above-mentioned present technology, the user interface control unit may perform interface control for designating a candidate area for replanting for each area defined in the area definition image, and may also perform control to present cost information related to replanting in response to the designation operation of the candidate area for replanting. For example, when a user designates an arbitrary area as a replant candidate area on the area definition image, a cost corresponding to the area is presented.

[0022] In the information processing device according to the present technology described above, it is considered that the user interface control unit performs interface control that enables each area defined in the area definition image to be individually designated as a replant candidate area. For example, the user can specify an arbitrary area by clicking on the area definition image.

[0023] In the information processing device according to the present technology described above, it is considered that the user interface control unit performs interface control that enables a plurality of areas to be designated as replant candidate areas by a range designation operation on the area definition image. For example, a user can specify a range on an area definition image, thereby specifying a plurality of areas included in the range.

[0024] The information processing device according to the present technology described above may include a file generating unit that generates a replant instruction file in response to determination of a replant execution area in the area definition image. For example, a user designates an arbitrary area as a replant candidate on an area definition image, and then determines the area to be replanted by a predetermined operation, whereby an instruction file for executing replanting is generated.

[0025] The user interface control unit may perform control to present a determination result as to whether or not replanting is required for each area shown in the area definition image. A determination is made as to whether each area defined in the area definition image should be replanted or not, and the determination result is displayed. For example, each area is displayed in an image form according to the determination result. In this case, the image form refers to elements that make up the grid image, such as color, brightness, and contour lines.

[0026] The information processing method according to the present technology calculates a rate related to plant growth from sensing data, and performs control to display, in a map area of ​​a user interface screen, sample position images arranged according to the sensing positions of each sensing data, in an image format determined according to the rate calculated from each sensing data. This allows the information processing device to present information suitable for making a replant decision. A program according to the present technology causes an information processing device to execute the process of the above-mentioned method, thereby making it easy to realize an information processing device that presents information suitable for making a replant decision. [Brief description of the drawings]

[0027] [Figure 1] FIG. 1 is an explanatory diagram of a sensing system according to an embodiment of the present technology. [Diagram 2] FIG. 2 is an explanatory diagram of rows and an imaging range in a farm field. [Diagram 3] FIG. 1 is a block diagram of a hardware configuration of an information processing apparatus according to an embodiment. [Figure 4] FIG. 2 is an explanatory diagram of a functional configuration of the information processing apparatus according to the embodiment; [Diagram 5] 4 is a flowchart showing a work procedure for farm field management according to the embodiment. [Figure 6] FIG. 4 is an explanatory diagram of a user interface screen according to the embodiment. [Figure 7] FIG. 13 is an explanatory diagram of a display state of an instruction rate map according to an embodiment. [Figure 8] FIG. 13 is an explanatory diagram of a display state of an actual rate map according to an embodiment. [Figure 9] FIG. 13 is an explanatory diagram of a display state of a stand count in the embodiment. [Figure 10] 11 is an explanatory diagram of a display state of a stand rate according to an embodiment. FIG. [Figure 11] 13 is an explanatory diagram of a rate map selection dialog according to an embodiment. [Figure 12] 11 is an explanatory diagram of the display state of a stand count and a rate map image in the embodiment. FIG. [Figure 13] 11 is an explanatory diagram of the display state of a stand rate and a rate map image in the embodiment. FIG. [Figure 14] 13 is an explanatory diagram of another display state of the stand rate and rate map image of the embodiment. FIG. [Figure 15] 13 is an explanatory diagram of a display state when a sample is specified on a stand counter according to an embodiment. FIG. [Figure 16] FIG. 2 is an explanatory diagram of an image display state according to an embodiment. [Figure 17] 11 is an explanatory diagram of a display state when a sample is specified on a stand rate according to an embodiment. FIG. [Figure 18] FIG. 13 is an explanatory diagram of a display state of a grid according to an embodiment. [Figure 19] FIG. 13 is an explanatory diagram of a grid setting dialog according to the embodiment. [Figure 20] 11A and 11B are explanatory diagrams of the display state of a grid and a stand rate according to an embodiment. [Figure 21] FIG. 11 is an explanatory diagram of a display state of a grid and a rate map according to an embodiment. [Figure 22] 11 is an explanatory diagram of the display state of a grid, stand rate, and rate map image in the embodiment. FIG. [Figure 23] 11 is an explanatory diagram of area specification by frame selection on a grid according to an embodiment. FIG. [Figure 24] 11 is an explanatory diagram of area specification by frame selection on a grid according to an embodiment. FIG. [Diagram 25] FIG. 13 is an explanatory diagram of an area designation state on a grid according to an embodiment. [Figure 26] FIG. 13 is an explanatory diagram of area specification by range specification on a grid according to an embodiment. [Figure 27] 11 is a flowchart of a preparation process according to an embodiment. [Figure 28] 11 is a flowchart of a user interface process according to the first embodiment. [Figure 29] 11 is a flowchart of a user interface process according to the first embodiment. [Diagram 30]11 is a flowchart of a user interface process according to the first embodiment. [Diagram 31] 11 is a flowchart of a user interface process according to the first embodiment. [Diagram 32] 11 is a flowchart of a user interface process according to the first embodiment. [Diagram 33] 13 is a flowchart of a rate calculation process according to an embodiment. [Diagram 34] 13 is a flowchart of a grid setting process according to the embodiment. [Diagram 35] FIG. 4 is an explanatory diagram of a data interpolation method according to the embodiment; [Diagram 36] FIG. 11 is an explanatory diagram of presentation of replant candidate areas on a grid in the second embodiment. [Figure 37] 13 is a flowchart of a user interface process according to the second embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0028] The embodiments will be described below in the following order. <1. Sensing system configuration> <2. Configuration of information processing device> <3. Work Procedure> <4. User interface screen> <5. Preparation for the embodiment> <6. User Interface Processing of the First Embodiment> <7. User Interface Processing of the Second Embodiment> <8. Summary and variations>

[0029] Here, some terms used in the embodiments will be explained. A "stand" refers to each individual crop (plant) that has been planted and germinated in a field. "Number of stands" or "stand count" refers to the number of stands measured from image data. "Population" refers to the number of stands measured from image data converted to the number of stands per unit area (e.g., 1 acre). This population can also be considered as the germination rate per unit area. "Stand rate" refers to the proportion of grown crops, such as the germination rate, per unit area, and in this embodiment, it is the ratio of the number of stands (population) per unit area to the number of planting instructions or the number of planting results per unit area shown in the rate maps (instructed rate map and actual rate map) described later. Stand rate can also be said to be the germination achievement rate.

[0030] <1. Sensing system configuration> First, a sensing system according to an embodiment will be described. 1 shows an information processing device 1 constituting a sensing system, and an imaging device 220 mounted on a small flying object 200 such as a drone. Also shown is a tractor 270 that performs planting.

[0031] The flying object 200 can move above the field 300, for example, by radio control by an operator or by automatic control. An imaging device 220 is set on the flying object 200 so as to capture, for example, an image of a downward direction. When the flying object 200 moves above the farm field 300 along a predetermined route, the imaging device 220 captures still images, for example, periodically.

[0032] It should be noted that, by flying object 200 at a relatively low altitude (for example, at an altitude of about 10 to 20 m), only a portion of farm field 300 appears in one captured image. By capturing still images at short intervals, it is possible to stitch the captured images together to obtain a composite image of the entire field. However, in the case of sensing in this embodiment, this is not necessarily required. For example, it is sufficient if discrete areas in the field 300 are captured as sensing samples to obtain multiple image data. Hereinafter, image data for each captured still image will also be referred to as a "sample."

[0033] The imaging device 220 mounted on the flying object 200 may be a visible light image sensor (an image sensor that captures visible light of R (red), G (green), and B (blue)), a camera for capturing NIR (Near Infra Red) images, a multi-spectrum camera that captures images in multiple wavelength bands, a hyper-spectrum camera, a Fourier transform infrared spectrophotometer (FTIR), an infrared sensor, etc. Of course, multiple types of cameras (sensors) may be mounted on the flying object 200. A multispectral camera may be used that captures, for example, NIR and R (red) images and can calculate the NDVI (Normalized Difference Vegetation Index) from the images obtained. NDVI is a vegetation index that indicates the degree of vegetation, and can be used as an index showing the distribution and activity of vegetation. NDVI can be calculated from R and NIR images. That is, the NDVI value is: NDVI=(NIR-R) / (NIR+R) It is required as follows.

[0034] Tag information is added to the images captured by the imaging device 220. The tag information includes imaging date and time information, position information (latitude / longitude information) as GPS (Global Positioning System) data, information on the flying altitude of the flying object 200 at the time of capturing the image, imaging device information (camera individual identification information, model information, etc.), and information on each image data (image size, wavelength, imaging parameters, etc.).

[0035] Image data and tag information captured by the imaging device 220 attached to such an aircraft 200 are acquired by the information processing device 1. For example, image data and tag information are exchanged via wireless communication or network communication between the imaging device 220 and the information processing device 1. Examples of the network include the Internet, a home network, a LAN (Local Area Network), a satellite communication network, and various other networks. Alternatively, the image data and tag information are transferred to the information processing device 1 in such a manner that a recording medium (eg, a memory card) attached to the imaging device 220 is read by the information processing device 1 side.

[0036] The information processing device 1 uses the image data and tag information to generate analytical information for measuring the farm field 300. The information processing device 1 also performs processing to present the analysis results to the user as an image. Specifically, the crops captured in the image data can be counted, and based on that, information that can be used to manage the field 300, such as the number of crops, the number of crops per unit area, germination rate, predicted harvest yield, crop ratio, etc., can be generated and presented to the user. The information processing device 1 also controls the execution of a display that presents the positions of each sample (image data) on a map image including the farm field 300, and a display of information related to replanting.

[0037] The information processing device 1 is realized as, for example, a personal computer (PC), a field-programmable gate array (FPGA), or a terminal device such as a smartphone or a tablet. In FIG. 1, the information processing device 1 is separate from the imaging device 220. However, for example, a calculation device (such as a microcomputer) serving as the information processing device 1 may be provided in a unit including the imaging device 220.

[0038] In the case of the sensing system of this embodiment, for example, the locations in the farm field 300 that are deemed to be "row" on the image data are automatically determined, and the number of planted crops is counted based on that determination. A row is a line along which crops are planted. For example, the ridges formed in the field 300 for planting seeds are a type of row. Rows are not limited to raised soil such as ridges, but also include lines formed when seeds are sown on flat land. For example, planting lines formed when sowing seeds with a tractor (seeder) are called rows.

[0039] 2 shows a schematic diagram of rows 301 formed in a farm field 300. For example, as time passes after sowing, the seeds germinate and the rows of crop leaves form rows 301 as shown in the figure. In this embodiment, the image capturing device 220 captures images of the farm field 300 while the flying object 200 is moving, and images of various locations are captured at appropriate timing, for example, as in the image capturing ranges SP1 and SP2 in the figure. For example, image data of the image capturing range SP1 and image data of the image capturing range SP2 are obtained by the image capturing device 220 as a still image captured image of one frame, and these are each imported into the information processing device 1 as a sample.

[0040] 2 shows linear rows 301, the rows 301 are not necessarily always straight. Straight rows 301 may bend at the edge of the field 300. Depending on the shape of the field 300, the route taken during sowing, obstacles, etc., the rows 301 may be partially curved, or may be formed in a spiral or concentric shape, for example.

[0041] The tractor 270 plants seeds and the like (seeds, seedlings, saplings, etc.) in the field 300. The tractor 270 automatically travels and plants based on planting instruction data provided by, for example, the information processing device 1. As a result, the above-mentioned rows 301 are formed in the field 300. A computer device is mounted on the tractor 270, and the computer device controls the traveling and planting operations based on instruction data. For example, by importing a data file including planting instruction data (rate map data including the number of planting instructions per unit area for each location) as shown in an instruction rate map described below, the computer device controls the traveling and planting operations of the tractor 270 based on the instruction data. Furthermore, the results of the planting work by the tractor 270 (the number of plants planted per unit area at each location) are provided to the information processing device 1 as result data.

[0042] <2. Configuration of information processing device> An information processing device 1 that acquires and processes image data from the imaging device 220 in the above sensing system will be described.

[0043] 3 shows a hardware configuration of the information processing device 1. The information processing device 1 includes a central processing unit (CPU) 51, a read only memory (ROM) 52, and a random access memory (RAM) 53. The CPU 51 executes various processes according to a program stored in the ROM 52 or a program loaded from the storage unit 59 to the RAM 53. The RAM 53 also stores data and the like necessary for the CPU 51 to execute various processes. The CPU 51, the ROM 52, and the RAM 53 are interconnected via a bus 54. An input / output interface 55 is also connected to this bus 54.

[0044] To the input / output interface 55, a display unit 56 consisting of a liquid crystal panel or an organic EL (Electroluminescence) panel, an input unit 57 consisting of a keyboard, a mouse, etc., a speaker 58, a storage unit 59, a communication unit 60, etc. can be connected.

[0045] The display unit 56 may be integrated with the information processing device 1 or may be a separate device. The display unit 56 displays captured images, various calculation results, and the like on the display screen based on instructions from the CPU 51. Furthermore, the display unit 56 displays various operation menus, icons, messages, and the like, that is, a GUI (Graphical User Interface), based on instructions from the CPU 51.

[0046] The input unit 57 refers to an input device used by a user of the information processing device 1. For example, the input unit 57 may be various types of operators or operation devices such as a keyboard, a mouse, keys, a dial, a touch panel, a touch pad, or a remote controller. An operation by the user is detected by the input unit 57, and a signal corresponding to the input operation is interpreted by the CPU 51.

[0047] The storage unit 59 is composed of a storage medium such as a hard disk drive (HDD) or a solid-state memory. The storage unit 59 stores, for example, image data captured by the imaging device 220, tag information, and various other information. The storage unit 59 is also used to store program data for various processes.

[0048] The communication unit 60 performs communication processing via networks including the Internet, and communication with various peripheral devices. The communication unit 60 may be a communication device that communicates with the imaging device 220 or the tractor 270, for example.

[0049] A drive 61 is also connected to the input / output interface 55 as required, and a storage device 62 such as a memory card is attached to it, for writing and reading data. For example, a computer program read from the storage device 62 is installed in the memory unit 59 as necessary, and data processed by the CPU 51 is stored therein. Of course, the drive 61 may be a recording / reproducing drive for removable storage media such as a magnetic disk, an optical disk, or a magneto-optical disk. These magnetic disks, optical disks, magneto-optical disks, etc. are also aspects of the storage device 62.

[0050] The information processing device 1 according to the embodiment is not limited to being configured as a single information processing device (computer device) 1 having a hardware configuration as shown in Fig. 3, but may be configured as a system of multiple computer devices. The multiple computer devices may be systemized using a LAN or the like, or may be located in a remote location using a VPN (Virtual Private Network) or the like using the Internet or the like. The multiple computer devices may include a computer device that can be used by a cloud computing service. 3 can be realized as a desktop or notebook personal computer, a tablet terminal, a smartphone, or other portable terminal. Furthermore, the information processing device 1 of this embodiment can be mounted in electronic devices having the function of the information processing device 1, such as a measuring device, a television device, a monitor device, an imaging device, and a facility management device.

[0051] For example, an information processing device 1 having such a hardware configuration has a calculation function by a CPU 51, a storage function by a ROM 52, a RAM 53, and a storage unit 59, a data acquisition function by a communication unit 60 and a drive 61, and an output function by a display unit 56, etc., and is provided with a functional configuration as shown in Figure 4 by the installed software functioning.

[0052] That is, the information processing device 1 is provided with an image acquisition unit 2, a calculation unit 5, a user interface control unit 6 (hereinafter, "user interface" will be abbreviated as "UI"), a recording control unit 7, a communication control unit 8, and a file generation unit 9 shown in FIG. These processing functions are realized by software executed by the CPU 51 . The programs constituting the software are downloaded from a network or read from a storage device 62 (e.g., a removable storage medium) and installed in the information processing device 1 in Fig. 3. Alternatively, the programs may be pre-stored in the storage unit 59 or the like. When the programs are started in the CPU 51, the functions of the above-mentioned units are realized. The calculation process and results of each function are stored, for example, in the memory area of ​​the RAM 53 or the memory unit 59.

[0053] The image acquisition unit 2 is a function for acquiring image data to be processed and tag information associated with the image data. For example, image data captured by the imaging device 220 is stored in the storage unit 59, and the CPU 51 reads out specific image data and uses it as a target for calculation processing.

[0054] The calculation unit 5 performs various calculations based on the image data. For example, the calculation unit 5 detects crops in the image data of the image of the field 300, i.e., the image data processed by the image acquisition unit 2, and determines the rows 301 (the part of the image in which the rows 301 are shown) in the image data based on the results of the crop detection. Then, the number of stands is calculated by counting the crop parts on the rows. By detecting the rows 301 and then counting the number of stands, weeds outside the rows can be excluded from the count, enabling highly accurate counting. The calculation unit 5 also calculates the number of crops (population) per unit area (e.g., 1 acre), germination rate (stand rate), predicted yield, replant cost, etc. based on the count number of crops captured in the image data.

[0055] The UI control unit 6 has a function of, for example, controlling the display of the display unit 56 and acquiring operation information input by the input unit 57. That is, the UI control unit 6 presents images and information calculated by the calculation unit 5, recognizes user operations, and the like. In particular, the UI control unit 6 controls the execution of display for setting the number of stands, stand rate, and replant area on the map image. Specifically, the UI control unit 6 controls the display of sample position images arranged according to the imaging position of each image data in the map area of ​​the UI screen in an image format determined according to the rate calculated from each image data.

[0056] The file generating unit 9 is a function that generates an instruction file including instruction data for replanting when the area to be replanted is determined by a user operation using the UI screen. This instruction file is provided to the tractor 270.

[0057] The recording control unit 7 has a function of controlling the storage in the memory unit 59 of various information such as the number of stands, population, stand rate, predicted harvest yield, cost information, UI screen setting information, etc. calculated by the calculation unit 5, as well as instruction files generated by the file generation unit 9. The communication control unit 8 has a function of controlling the transmission of information from the calculation unit 5 and the file generation unit 9 to an external device via the communication unit 60.

[0058] <3. Work Procedure> Figure 5 shows an example of a workflow for managing a farm. In step S1, the flying object 200 flies over the farm field 300, and images are captured by the imaging device 220. In this way, image data is obtained as a large number of samples.

[0059] In step S2, the information processing device 1 acquires image data. For example, the information processing device 1 imports image data captured during flight through wired or wireless communication from the imaging device 220, or through delivery of a storage device 62 such as a memory card. The information processing device 1 stores the image data in, for example, the storage unit 59.

[0060] In step S3, the information processing device 1 performs an analysis process based on the image data. For example, stand counting is performed using each image data. A specific example of the process will be described later as a preparation process.

[0061] In step S4, the information processing device 1 presents the status of the field 300 and various information to a user (e.g., farm staff) on the UI screen to help them decide whether to replant, and responds to user operations. Specifically, the information processing device 1 presents information to help the user decide whether to replant or not, or which area to replant, and performs processing to accept the designation of the area expected to be replanted.

[0062] When replantation is performed as a result of input / output on the UI screen, the process proceeds from step S5 to step S6, and the information processing device 1 performs processing to generate an instruction file including an ROI (Region of Interest) for replantation. This instruction file is provided to the tractor 270, and the ROI is information indicating the area to be replanted. Then, in step S7, the information processing device 1 exports the instruction file. The instruction file is received by the tractor 270, and then the actual replant is carried out.

[0063] In the above-mentioned procedure, the information processing device 1 according to the present embodiment presents information suitable for the user's decision on replanting, particularly in the input / output on the UI screen in step S4. Below, we will explain specific examples of UI screens.

[0064] <4. User interface screen> Fig. 6 shows an example of a UI screen displayed on, for example, the display unit 56. In Fig. 6, a map image is displayed on the UI screen, but no information for replanting is being presented.

[0065] The UI screen is provided with a map area 21. A map image including a farm field 300 is displayed in the map area 21. Fig. 6 simply shows a state in which a map of the surrounding area of ​​the farm field 300 is displayed. The scale of the map image in the map area 21 can be arbitrarily set by the user using the zoom buttons 28. The range of the map displayed in the map area 21 can be moved by the user by dragging the cursor 90 or the like.

[0066] The UI screen is provided with a map selection section 22, which allows the user to select a rate map using, for example, a pull-down menu. The rate map may be an instruction rate map or a performance rate map, which will be described later.

[0067] The UI screen provides a map display checkbox 23, a count display checkbox 24, a rate display checkbox 25, and a grid display checkbox 26, and the user can check each checkbox to select whether or not to display the rate map, stand count, stand rate, or grid.

[0068] The UI screen is provided with a date and time selection section 27, which allows the user to select the date and time of the sample to be processed (date and time of flight / image capture) using, for example, a pull-down menu.

[0069] The UI screen is provided with a grid generation button 30, a rate calculation button 31, and an ROI generation button 32, which can be operated by the user. The grid generation button 30 is a button for performing an operation to set a grid, which will be described later. The rate calculation button 31 is a button for instructing calculation of the stand rate. The ROI generation button 32 is a button for performing an operation for generating an instruction file. Examples of the display state and processing corresponding to the operation of each button will be described later.

[0070] The UI screen is provided with a population display section 35, a stand rate display section 36, and a sample image display section 37. When a sample is specified, the population display section 35 displays the population for that sample. When a sample is specified, the stand rate display section 36 displays the number of stands for that sample. Images of the samples are displayed in the sample image display section 37. Specific examples of each will be described later.

[0071] FIG. 7 shows a state in which an instruction rate map 351 is displayed in the map area 21. When the user checks the map display checkbox 23, the rate map selected in the map selection unit 22 at that time is displayed. In Fig. 7, "Variable Rate Map 2019" indicating the instruction rate map 351 is selected in the map selection unit 22, and therefore, by checking the map display checkbox 23, the instruction rate map 351 corresponding to "Variable Rate Map 2019" is displayed. When another rate map is selected in the map selection unit 22, the other rate map is displayed by checking the map display check box 23. Fig. 8 shows an example in which "As Applied Map 2019", which indicates an actual rate map, is selected in the map selection unit 22, and an actual rate map 352 corresponding to "As Applied Map 2019" is displayed in the map area 21.

[0072] The recommended rate map 351 shown in FIG. 7 indicates the recommended planting numbers for each area in the field 300, and the actual rate map 352 in FIG. Planting recommendations are set as the number of stands per unit area (eg, acre).

[0073] In the farm field 300, the planting instructions may be different for each section, such as 10 acres, 100 acres, etc. The instruction rate map 351 is a map showing the planting instruction numbers formulated in the planting plan, such as setting the number of stands per unit area (i.e., population) to "35,000" in one area and setting the number of stands per unit area (population) to "30,000" in another area. In other words, it is a map showing the population values ​​instructed to the tractor 270 when planting for each area in the field 300.

[0074] On the other hand, actual rate map 352 shows the actual planting results that tractor 270 has performed in accordance with the instructions shown in command rate map 351. Tractor 270 plants in each district according to the number of planting instructions per unit area shown in the command file, but some deviations occur due to the performance of tractor 270 and the accuracy of position detection. Actual rate map 352 shows the number of plants per unit area (actual number of plantings) as the result of planting. Therefore, the boundaries between districts may be slightly different between command rate map 351 and actual rate map 352.

[0075] In the designated rate map 351 in Fig. 7 and the actual rate map 352 in Fig. 8, each area is displayed in a different color according to the number of stands per unit area. Although not shown, the area may be displayed in different colors such as green, yellow, red, etc. This enables the user to easily recognize the number of planting instructions or the number of plantings actually performed in each area.

[0076] The recommended planting numbers per unit area may vary for different areas due to various reasons, such as the soil and geology of each area, differences in planting seasons, harvesting schedules, etc. Of course, there are also cases where planting is performed with the same recommended planting numbers for the entire field 300.

[0077] 9 shows an example of a circular sample position mark 350 in the map area 21, and shows an example in which the sample position mark 350 is displayed in a manner according to the population. When the user checks the count display check box 24, such a sample position mark 350 is displayed. In order to avoid complicating the drawing, the reference numeral "350" is attached to only some of the sample position marks in Fig. 9. The same applies to the other drawings.

[0078] Each sample position mark 350 shown as a circular image in the map area 21 indicates the imaging position of one sample (image data), that is, the position of the imaging device 220 at the time of imaging. Alternatively, the sample position mark 350 may indicate the imaged position, that is, the position of the location designated as the imaging range SP1, SP2, etc. on the farm field 300 side. Since imaging is performed, for example, periodically during the flight of the air vehicle 200 as described above, each sample position mark 350 will be displayed at each position so as to indicate the flight trajectory.

[0079] In this case, each sample position mark 350 is displayed in a different color according to the population, for example, green (white part in the figure), yellow (shaded part in the figure), or red (black part in the figure). The population at each sample position mark 350 is the number of stands calculated from the image data of that sample position mark 350, converted into the number of stands per unit area, such as one acre. The imaging range of one image data is a very small portion of the entire field 300, as shown in Fig. 2 as imaging ranges SP1 and SP2. For example, the number of stands is calculated from the image data of the imaging range SP1, but the number of stands varies depending on the size of the imaging range SP1. In other words, it varies depending on conditions such as the altitude of the flying object 200 and the angle of view of the imaging device 220. Therefore, instead of the number of stands calculated from the image data itself, a population value converted from the number of stands calculated from the image data is used. The population of the imaging range SP1 can be calculated by finding the ratio between the area of ​​the imaging range SP1 and the unit area, and multiplying the ratio by the number of stands calculated from the image. The area can also be calculated from the ratio of a specific length on the image to the actual length. For example, the length between two rows is input (10m), and the number of stands per certain length of the row is calculated by image recognition (30 stands per 10m). Then, 30 stands / (10m x 10m) = 0.3 (stands / m 2 ) the population can be calculated as

[0080] The sample position mark 350 is displayed in a different color depending on the population. Of course, this differs depending on the type of crop, etc., but for example, if the population is 35,000 or more, it is displayed in green, if it is less than 35,000 and is greater than or equal to 33,000, it is displayed in yellow, and if it is less than 33,000, it is displayed in red. Note that the three-level color coding is merely an example. This color-coded display allows the user to check the population at each sample position, i.e., the growth status, such as germination.

[0081] 10 shows an example in which a circular sample position mark 350 is shown in the map area 21 and the sample position mark 350 is displayed in a manner corresponding to the stand rate. When the user checks the rate display check box 25, such a sample position mark 350 is displayed.

[0082] As in the case of Fig. 9, each sample position mark 350 shown as a circular image in the map area 21 indicates the imaging position of one sample (image data), but in the case of Fig. 10, each sample position mark 350 is displayed in a different color according to the stand rate. For example, the marks are displayed in green (white circle in the figure), yellow (shaded circle in the figure), and red (black circle in the figure). The stand rate at each sample position mark 350 can be considered as the ratio of the population calculated from the image data of that sample position mark 350 to the population expected at that location. The planned population is the number of planting instructions shown in the directed rate map 351 or the number of plantings shown in the actual rate map 352. In other words, it is the number of crops that are planned to be grown. For example, if the number of stands per unit area is specified to be 35,000 and the population calculated from the image data is 35,000, the stand rate will be 100%. If the number of stands per unit area is specified to be 35,000 and the population calculated from the image data is 34,000, the stand rate will be 97%.

[0083] The sample position mark 350 is displayed in different colors according to the stand rate. For example, if the stand rate is 98% or more, it is displayed in green (white circle in the figure), if it is less than 98% but more than 90%, it is displayed in yellow (shaded circle in the figure), and if it is less than 90%, it is displayed in red (black circle in the figure). Note that the three-level color coding is merely an example. Such color coding allows the user to see the stand rates, such as germination rate, at each sample location.

[0084] The stand rate is calculated based on the expected population at each sample position. This standard is the expected population (the number of plantings per unit area) at the sample position shown in the indicated rate map 351, or the expected population (the number of plantings per unit area) at the sample position shown in the actual rate map 352. The user can select which standard to use.

[0085] As a processing example, it is conceivable that the calculation of the stand rate corresponding to each sample position mark 350 is performed when the user operates the rate calculation button 31. When the user operates the rate calculation button 31, a selection dialog 34 is displayed as shown in FIG. 11, allowing the user to select a reference rate map, for example, by using a pull-down menu. For example, a pull-down menu may display a prescribed rate map 351 named "Variable Rate Map 2019" and an actual rate map 352 named "As Applied Map 2019," allowing the user to select any rate map. In response to the selection, the CPU 51 calculates the stand rate corresponding to each sample position mark 350 and displays it in a different color as shown in FIG. When the designated value is constant or when there is no designated data, calculations may be performed assuming that the entire farm field 300 has a uniform value. Furthermore, the user may be able to select the rate map in other ways, such as by providing an input form for numerical data so that the user can make a selection, without being limited to a pull-down menu.

[0086] Next, FIG. 12 shows a case where the user checks both the map display check box 23 and the count display check box 24. In this case, the selected rate map (eg, the instruction rate map 351) is displayed in the map area 21, and furthermore, sample position marks 350 color-coded according to population are displayed superimposed thereon. This image allows the user to easily compare the planned population for each area with the population calculated from the image data (the actual population).

[0087] FIG. 13 shows a case where the user has checked both the map display check box 23 and the rate display check box 25 . In this case, the selected rate map (eg, the indicated rate map 351) is displayed in the map area 21, and furthermore, a sample position mark 350 color-coded according to the stand rate is displayed superimposed thereon. This image makes it easy for the user to compare the planned population for each area with the stand rate at the location of each image data.

[0088] The population display using sample position marks 350 in Fig. 12 is a color-coded display based on the number of stands, while the stand rate display using sample position marks 350 in Fig. 13 is a color-coded display based on the rate, so the information has different meanings. The user can easily grasp the status of the farm field 300 by selecting between these displays as desired.

[0089] In addition, although an example is shown in Figures 12 and 13 in which the sample position mark 350 is displayed superimposed on the rate map, as shown in Figure 14, the sample position mark 350 and the rate map (e.g., the instruction rate map 351) may be displayed separately. Alternatively, the user may be allowed to select between a separate display as in FIG. 14 and an overlapping display as in FIGS.

[0090] FIG. 15 shows a case where the user designates one sample position mark 350 in a state where a population is displayed using the sample position marks 350. For example, it is assumed that the user performs an operation to specify one sample position mark 350 by clicking the cursor 90 while the cursor 90 is placed on one of the sample position marks 350 . In this case, the population value for the specified sample position mark 350 is displayed in the population display section 35. In the figure, an example is shown in which the population value "31000" is displayed as "31K".

[0091] The sample image display unit 37 displays an image corresponding to the sample position mark 350, i.e., an image captured at the position of the sample position mark 350. Here, an example is shown in which an RGB image 41, a stand count image 42, and an NDVI image 43 are displayed. The RGB image 41 is an image captured when the imaging device 220 is an RGB camera, for example. The NDVI image 43 is an image of NDVI calculated from R and NIR. The stand count image 42 is an image processed, for example, for stand calculation.

[0092] The user can also enlarge these images. For example, by performing an operation such as clicking on a portion of the RGB image 41, the RGB image 41 is enlarged and displayed as shown in FIG. The user can operate the exit button 45 to end the enlarged display and return to the state shown in FIG. 15 and 16, the user can check the population value or image at any point (at the sample position mark 350). In other words, more detailed information than the rough population indicated by the color coding of the sample position mark 350 can be confirmed.

[0093] FIG. 17 shows a case where a stand rate is displayed using sample position marks 350 and the user designates one sample position mark 350. For example, when the user performs an operation to specify one sample position mark 350 with the cursor 90, the population for the specified sample position mark 350 is displayed in the population display section 35, and the stand rate is displayed in the stand rate display section 36. The figure shows an example in which the population value "32000" is displayed as "32K" and the stand rate "88.9%" is also displayed.

[0094] In this case, too, images corresponding to the sample position mark 350, such as the RGB image 41, the stand count image 42, and the NDVI image 43, are displayed in the sample image display section 37. When these images are designated, the images are enlarged and displayed, as in the cases of Figs. 15 and 16. In the case of FIG. 17, the user can check the stand rate value in addition to the rough stand rate based on the color coding of the sample position marks 350.

[0095] FIG. 18 shows a state in which a grid 355 of a checkerboard pattern is displayed in the map area 21. When the user checks the grid display check box 26, such a grid 355 is displayed. This grid 355 is an area definition image, and is a display that defines the area of ​​the farm field 300 with lattice lines. That is, each area into which the farm field 300 is divided is presented to the user as a range separated by the lattice (grid squares). The grid 355 is primarily used to designate the area to be replanted.

[0096] Each area represented by a square in the grid 355 is displayed in an image manner determined according to the stand rate calculated for that area, here color-coded. Note that each area of ​​the grid 355 and each sample (image data indicated by the sample position marks 350) do not necessarily correspond one-to-one. For this reason, the stand rate of each area of ​​the grid 355 is calculated by an interpolation calculation using neighboring sample position marks 350, etc. A specific example will be described later.

[0097] Each area of ​​the grid 355 is displayed in a different color, for example, if the stand rate is 98% or more, it is green (white squares in the figure), if it is less than 98% but more than 90%, it is yellow (shaded squares in the figure), if it is less than 90%, it is red (black squares in the figure), etc. Note that the three-level color coding is just one example. This color coding allows the user to see the stand rates in each area defined by the grid 355.

[0098] For grid 355, the size of the frame can be set by the user. When the user operates the grid generation button 30, a setting dialog 33 for setting the size of the grid is displayed, as shown in Fig. 19. This setting dialog 33 allows the user to arbitrarily set the size of one side of the grid. The grid 355 displayed as in FIG. 18 is an image of a lattice pattern that forms squares of the size thus set.

[0099] When the grid 355 is displayed, a calculation button 71, a planning cost display section 72, a selected area display section 73, an average population display section 74, an average stand rate display section 75, and a replant cost display section 76 appear, as shown in the figure. However, when no particular area on the grid 355 is selected, no specific numerical values ​​are displayed, as in FIG.

[0100] FIG. 20 shows a case where the user has checked both the grid display check box 26 and the rate display check box 25 . In this case, the map area 21 displays sample position marks 350 color-coded according to stand rate and a grid 355 superimposed thereon. This image makes it easy for the user to understand the relationship between each area defined by the grid 355 and the sample positions.

[0101] FIG. 21 shows a case where the user has checked both the grid display check box 26 and the map display check box 23 . In this case, the selected rate map (eg, the command rate map 351) is displayed in the map area 21, and a grid 355 is further displayed superimposed thereon. This provides the user with an image that allows them to simultaneously see the planned population for each district and the area defined by the grid 355.

[0102] FIG. 22 shows a case where the user has checked the grid display check box 26, the map display check box 23, and the rate display check box 25. In this case, the selected rate map (eg, the indicated rate map 351) is displayed in the map area 21, a grid 355 is displayed overlaid thereon, and sample position marks 350 color-coded according to the stand rate are further displayed overlaid thereon. This provides the user with an image that allows them to simultaneously see the planned population for each district, the area defined by the grid 355, and the sample locations.

[0103] Although not shown in the figure, if the user checks on the grid display checkbox 26, the map display checkbox 23, and the count display checkbox 24, the rate map, grid 355, and sample position marks 350 color-coded by population will be displayed superimposed. Furthermore, if the user checks both the grid display check box 26 and the count display check box 24, the grid 355 and the sample position marks 350 color-coded according to population are displayed superimposed on each other.

[0104] In each case, both the count display checkbox 24 and the rate display checkbox 25 can be checked, in which case the sample position mark 350 will be colored according to either the stand rate or the population. Alternatively, the count display check box 24 and the rate display check box 25 may be set so that they cannot be checked at the same time.

[0105] As mentioned above, the grid 355 is used to specify the area to be replanted. 23 shows an operation of specifying a replant candidate area using a grid 355. For example, a single replant candidate area is selected by the user clicking an area with the cursor 90. In this case, as shown in the figure, the square of the area is shown to be in a thick line, for example, to indicate that the area is selected. Of course, the selected state may be indicated by changing the color or highlighting the area. FIG. 24 shows an example in which neighboring areas are successively selected by clicking or the like. In this way, the user can arbitrarily select an area as a replant candidate area. FIG. 25 also shows a state in which the user has selected eight other areas as replant candidate areas.

[0106] When one or more areas have been selected, specific numerical values ​​are displayed in the selected area display section 73, average population display section 74, average stand rate display section 75, and replant cost display section 76 according to the area selected. 23, when one area is selected, numerical values ​​according to the selection state are displayed in the selected area display section 73, the average population display section 74, and the average stand rate display section 75. That is, the selected area display section 73 displays the area, the average population display section 74 displays the average population value of the selected area, and the average stand rate display section 75 displays the average stand rate of the selected area. When multiple areas are selected as in FIG. 24, these numerical values ​​correspond to the multiple areas.

[0107] The planning cost display section 72 also allows the user to input the replant cost per unit area, for example, per acre. In the replant cost display section 76, as shown in FIG. 25, for example, the replant cost according to the area of ​​the selected area is displayed based on the replant cost per acre. As shown in the display examples of Figures 23, 24, and 25, the user can select an area and have the total area, average population, and average stand rate of the candidate replant areas displayed. Costs are also displayed, allowing the user to select an area while checking costs, etc.

[0108] When the calculation button 71 is operated, a predetermined calculation window (not shown) is displayed, and various conditions such as the region, date and time, current population, and target population can be input, and a calculation can be instructed to be executed. This causes a calculation to be made as to whether or not replanting is necessary, and the result of the judgment is displayed. The user can refer to such a display to determine whether or not to perform replanting.

[0109] The ROI generation button 32 is a button that the user operates when determining the replant candidate area as the area to be replanted. For example, when replanting is desired in a state where the replant candidate areas are selected as shown in Fig. 25, the user operates the ROI generation button 32. As a result, one or more areas that have been determined as the replant candidate areas are determined as areas to be replanted, and an instruction file for this purpose is generated. Thereafter, the instruction file generated by the information processing device 1 is exported and handed over to the tractor 270, whereby the tractor 270 can execute the replanting work.

[0110] In the examples of Figures 23, 24, and 25, the user selects the areas (cells) shown in grid 355 one by one, but it is also possible to select them all at once by specifying a range, as shown in Figure 26. For example, multiple areas can be specified by dragging the cursor 90 from the dashed line position (cursor 90P), which makes it possible to facilitate the selection of replant candidate areas. Another possible operation method is to provide a slider or input box on the UI screen, and have the user input a threshold stand rate, so that areas with stand rates below the threshold are selected as candidate areas for replanting.

[0111] <5. Preparation for the embodiment> An example of processing performed by the information processing device 1 to realize the above-described UI screen will now be described. First, the preparation process performed at step S3 in FIG. 5 will be described with reference to FIG. 27 shows a process that the CPU 51 of the information processing device 1 executes for each sample after capturing image data as the sample. This process is realized by the CPU 51 having the functions shown in FIG. 4, mainly the function as the calculation unit 5.

[0112] In step S101, the CPU 51 identifies image data to be processed. That is, one of the images captured from the air by the imaging device 220 mounted on the flying object 200 is identified as the image to be processed. In this case, the image data may be color image data captured by an RGB (red, green, blue) camera, or may be image data in the form of IR (near infrared) data and R (red) data. The image data may also be the above-mentioned NDVI image data obtained from the IR (near infrared) data and R (red) data.

[0113] In step S102, the CPU 51 performs a process of calculating the number of stands for the image data to be processed, that is, detects crop areas from the image, and counts the number of crops. There are various methods for calculating the number of stands, but it is preferable to detect rows 301 in the image and detect plants growing on the rows 301. Also, because crops are arranged at regular intervals on the rows 301, it is preferable to count taking these intervals into consideration. By performing this type of counting, the number of stands can be counted with relatively high accuracy, regardless of weeds, etc.

[0114] The process of counting the number of stands involves detecting the parts of the image that are determined to be crops and counting them. For example, when using images captured by an RGB camera, G pixel detection is assumed to be the detection method. Crops are mainly composed of leaves, and leaves are generally green, so green areas are detected from the RGB color image. To detect green areas, for example, each RGB component of a pixel in the image may be compared, and only the pixel with the highest G (green) may be detected. Alternatively, RGB can be converted to a lightness, saturation, and hue space such as HSV, and pixels having a hue within the region can be detected by setting a region with a certain width in the green region of the hue value.

[0115] In addition, a detection method using near-infrared and red images is a threshold detection method for NDVI images. NDVI, an index showing the distribution of crops (plants), is calculated from near infrared and red light and has a range of ±1, with crops having high NDVI values ​​and soil having low NDVI. Therefore, an NDVI image can be created from IR data and R data, and pixels in the crop area can be detected by threshold processing and counted. If an NDVI image already exists, the pixels in the crop area can be detected by threshold processing. Plants can be detected by these G pixel detection and NDVI threshold detection, and then by taking into account the detection of row 301 and the planting intervals of crops as described above, it is possible to count without including weeds.

[0116] After counting the number of stands, the CPU 51 calculates the population in step S103. In this case, the CPU 51 calculates the area of ​​the imaging range of the image to be processed. The area can be calculated from altitude information of the flying object 200 at the time of imaging and information on the angle of view of the imaging device 220. After the area of ​​the imaging range is calculated, the ratio to the unit area is calculated, and the population is calculated by multiplying the number of stands by this ratio. Of course, the unit area may be calculated from the ratio between a specific length on the image and the actual length using image recognition. As mentioned earlier

[0117] In step S104, the CPU 51 sets the display mode according to the population value. This is processing for setting the color of the sample position mark 350 when the population display as shown in Fig. 9 is performed for the image data to be processed. In step S105, the CPU 51 performs mapping setting. That is, the CPU 51 reads out latitude and longitude information from tag information attached to the image data, and sets the latitude and longitude information as a position for displaying the sample position mark 350 in the map area 21.

[0118] In step S106, the CPU 51 performs a storage process, that is, stores the number of stands, the population value, the color and display position settings of the sample position mark 350, and the like, for the image data that is processed as one sample. This allows the sample position marks 350 to be displayed according to the population when a display is requested on the above-mentioned UI screen.

[0119] In step S107, CPU 51 determines whether the above processing has been completed for all samples, and if there are unprocessed samples, the process returns to step S101 and the same processing is performed on the unprocessed samples. When all samples have been processed, the preparation process in FIG. 27 ends. Note that "all samples" here may refer to all image data captured in one flight, or may refer to all necessary samples selected in advance. For example, useless image data may be captured at the beginning or end of a flight, or the drone may fly outside the field 300 during the flight. It is not necessary to perform the process of FIG. 27 on image data in such cases.

[0120] By carrying out the preparation process of FIG. 27, it becomes possible to display the sample position marks 350 for each sample in the map area 21 in a color according to the population.

[0121] <6. User Interface Processing of the First Embodiment> Next, the UI processing performed by the information processing device 1 will be described with reference to FIGS. 28, 29, 30, 31, and 32 show the UI processing performed by the CPU 51 using the functions of the UI control unit 6, and the processing performed during that process using the functions of the calculation unit 5, file generation unit 9, recording control unit 7, and communication control unit 8. In each figure, the connections between the flowcharts are indicated by symbols "c1" to "c5." The UI processing shown in Figures 28 to 32 is processing carried out from step S4 to step SS6 in Figure 5, and is processing that realizes the display of the UI screen and the correspondence with user operations as described in Figures 6 to 27.

[0122] The information processing device 1 starts displaying a UI image in step S200 of Fig. 28. For example, the CPU 51 causes the display unit 56 or the like to execute a display as shown in Fig. 6. When the UI image starts to be displayed, the UI image may be displayed according to the on / off state of the check box when the UI image was last displayed.

[0123] During display control of a UI image, the CPU 51 repeatedly executes the monitoring process from step S201 to step S219 in Fig. 32. Hereinafter, the monitoring processes from step S201 in Fig. 28 to step S219 in Fig. 32 will be collectively referred to as a "monitoring loop."

[0124] In step S201 of Fig. 28, the CPU 51 determines whether to end the UI display. For example, the CPU 51 determines to end the UI display in response to a user's end operation or the occurrence of some end trigger, and performs a process of ending the display of the UI image in step S230 to end the series of UI control processes.

[0125] In step S202, the CPU 51 monitors a rate map selection operation. If it detects that the user has operated the map selection unit 22, the CPU 51 proceeds from step S202 to step S231 and performs processing corresponding to the selection operation. For example, the CPU 51 displays a pull-down menu in the map selection unit 22 or detects a selection operation. If the CPU 51 detects an operation by the user to change the current rate map, the CPU 51 proceeds from step S232 to step S233 and performs processing to change the selected rate map. For example, the CPU 51 switches the selected state of the instructed rate map 351 to the selected state of the actual rate map 352. If the user performs an operation such as displaying a pull-down menu but does not end up issuing an instruction to change the rate map, the CPU 51 returns to the monitoring loop from step S232.

[0126] In step S203, the CPU 51 monitors the operation of the rate calculation button 31. If it detects that the user has operated the rate calculation button 31, the CPU 51 proceeds from step S203 to step S240 and controls the display of the selection dialog 34 as shown in FIG. If it is detected that the user has performed a cancel operation on the selection dialog 34, the CPU 51 proceeds from step S241 to step S244, where it ends the selection dialog 34 and returns to the monitoring loop. When it is detected that the user has selected a reference rate map in the selection dialog 34 and then performed an OK operation, the CPU 51 proceeds from step S242 to step S243 and performs a rate calculation process.

[0127] This rate calculation process is a process for obtaining a stand rate for each sample. An example of the rate calculation process is shown in FIG.

[0128] In step S301, the CPU 51 specifies a sample (image data) to be calculated. This is processing to specify one of the samples displayed as the sample position mark 350 as the target of calculation processing.

[0129] In step S302, the CPU 51 obtains a population value corresponding to the position of the sample to be calculated from the currently selected rate map (e.g., either the command rate map 351 or the actual rate map 352), and sets this as the target population, i.e., the population expected at the location of the sample.

[0130] In step S303, the CPU 51 calculates a stand rate from the population calculated in the preparation process for the sample to be calculated and the value of the target population.

[0131] In step S304, the CPU 51 sets the display mode according to the value of the stand rate. This is processing for setting the color of the sample position mark 350 when the rate of the image data to be processed is displayed.

[0132] In step S305, the CPU 51 performs a storage process, that is, stores the value of the stand rate, the color setting of the sample position mark 350 when the rate is displayed, and the like, for the image data that is the subject of calculation as one sample.

[0133] In step S306, CPU 51 determines whether the above process has been completed for all samples to be calculated, and if there are unprocessed samples, the process returns to step S301 and the same process is performed for the unprocessed samples. When the processing has been completed for all necessary samples, the rate calculation processing of FIG. 33 ends. At this point, it is possible to display the sample position mark 350 for each sample in a color corresponding to the stand rate. In other words, when a rate display such as that shown in Fig. 10 is requested on the above-mentioned UI screen, it becomes possible to display the sample position mark 350 according to the stand rate.

[0134] After performing the above rate calculation process in step S243 in FIG. 28, the CPU 51 proceeds to step S244 to end the selection dialogue 34 and return to the monitoring loop.

[0135] 29, the CPU 51 monitors the operation of the grid generation button 30. If it detects that the user has operated the grid generation button 30, the CPU 51 proceeds from step S204 to step S250 and performs display control of the setting dialog 33 as shown in FIG. If it is detected that the user has performed a cancel operation on the setting dialog 33, the CPU 51 proceeds from step S251 to step S254, where it ends the setting dialog 33 and returns to the monitoring loop. When it is detected that the user has selected or input a value for the grid size in the setting dialog 33 and then performed an OK operation, the CPU 51 proceeds from step S252 to step S253 and performs a grid setting process.

[0136] This grid setting process is a process for setting the size of the squares of the grid 355, setting each area into which the farm field 300 is divided, and determining the stand rate of each area. An example of the grid setting process is shown in FIG.

[0137] In step S401, the CPU 51 sets the grid size. That is, the size of the grid is set in accordance with a user input in the setting dialog 33. In step S402, CPU 51 specifies an area to be calculated. The area is indicated by one cell of grid 355. A plurality of cells are set by the lattice of grid 355, and one of these cells is specified as the area to be calculated.

[0138] In step S403, the CPU 51 calculates the stand rate of the calculation target area (the square of the grid 355) by interpolation processing. This calculation can use, for example, an inverse distance weighting method with area related samples. That is, the rate for the area being calculated is calculated using the stand rate for image data related to that area, and is calculated using the inverse distance weighting method. There are several possible examples for collecting and calculating samples for use in the inverse distance weighting method.

[0139] As a first example, it is possible to use image data captured within a predetermined distance from the center point of an area as a sample (image data) related to the area to be calculated. 35A shows three areas AR1, AR2, and AR3 as areas defined by a grid 355. Now, assume that the area AR1 is the target of stand rate calculation. The range of distance d1 from the center point of area AR1 is indicated by circle CC. Additionally, each of the surrounding samples is indicated by sample position marks 350X, 350Y, and 350Z. The samples within the circle CC indicated by the sample position marks 350X, 350Y, and 350Z are regarded as samples related to the area AR1 to be calculated. The stand rates of these samples are used to calculate the stand rate of the area AR1.

[0140] As a second example, a predetermined number (for example, three) of image data items may be used as samples (image data) related to an area that is the subject of calculation, in order of proximity to the center point of that area. 35B shows three samples (sample position marks 350X, 350Y, and 350Z) in order of distance from the center point of the area AR1 to be calculated. The stand rates of these samples are used to calculate the stand rate of the area AR1.

[0141] The third example uses samples from the first and second examples, but weights specific samples. In other words, the rate of image data relating to the one area being calculated that captures an area with the same number of planting instructions or planting results as the one area is weighted, and an interpolation calculation is performed. 35C, for example, the image of FIG. 35B is shown superimposed with the instruction rate map 351. Here, the area AR1 and the samples indicated by the sample position marks 350X have the same planting instruction number indicated by the instruction rate map 351. Therefore, the stand rates of the samples indicated by the sample position marks 350X, 350Y, and 350Z are used, but the stand rate of the sample indicated by the sample position mark 350X is weighted to calculate the stand rate of the area AR1.

[0142] In step S403 in FIG. 34, the CPU 51 performs an interpolation calculation in the above example, and calculates the stand rate for the target area by the inverse distance weighting method.

[0143] In step S404, the CPU 51 sets the display mode in accordance with the calculated stand rate value, which is a process of setting the color of the area that is the subject of the calculation when the grid 355 is displayed.

[0144] In step S405, the CPU 51 performs a storage process, that is, stores the grid size, the color settings of the area to be calculated, and the like.

[0145] In step S406, the CPU 51 determines whether or not the stand rate calculation has been completed for all areas to be calculated, and if there is an area that has not been calculated, the process returns to step S402 and the same process is carried out for the unprocessed area. 34 ends when the processing has been completed for all areas of the grid 355. At this point, it becomes possible to display the grid 355 with each area color-coded as shown in FIG.

[0146] After performing the above grid setting process in step S253 in FIG. 29, the CPU 51 proceeds to step S254 to end the setting dialogue 33 and return to the monitoring loop.

[0147] In step S205, the CPU 51 monitors the zoom in or zoom out operation using the zoom in or zoom out button 28. If it detects that the user has performed a zoom in or zoom out operation, the CPU 51 proceeds from step S205 to step S261, and controls the zoom in or zoom out of the display of the map image in the map area 21. Then, the process returns to the monitoring loop. It should be noted that if a rate map, sample position marks 350, or grid 355 are displayed on the map image in the map area 21, they will also be enlarged or reduced in size in accordance with the map image.

[0148] In step S206, the CPU 51 monitors an operation for scrolling the map, such as a drag operation in the map area 21. If it detects that the user has performed a scroll operation, the CPU 51 proceeds from step S206 to step S262, and scrolls the map image in the map area 21. Then, the process returns to the monitoring loop. It should be noted that if a rate map, sample position mark 350, or grid 355 is displayed on the map image in the map area 21, these will also be moved in accordance with the map image.

[0149] 30, the CPU 51 monitors whether the user has checked the map display check box 23. If it detects that the user has checked the check box, the CPU 51 proceeds from step S207 to step S263, and performs control to display the currently selected rate map (for example, either the command rate map 351 or the actual rate map 352) in the map area 21. Then, the CPU 51 returns to the monitoring loop.

[0150] In step S208, the CPU 51 monitors whether the user has checked off the map display check box 23. If it detects that the user has checked off the check box 23, the CPU 51 proceeds from step S208 to step S264 and performs control to terminate the display of the rate map on the map area 21. Then, the process returns to the monitoring loop.

[0151] In step S209, the CPU 51 monitors whether the user has checked the count display check box 24. If it detects that the user has checked the check box, the CPU 51 proceeds from step S209 to step S265, and performs control to display the sample position mark 350, the color of which is set by the population, in the map area 21. Then, the process returns to the monitoring loop.

[0152] In step S210, the CPU 51 monitors whether the user has checked off the count display check box 24. If it detects that the user has checked off the count display check box 24, the CPU 51 proceeds from step S210 to step S266, and performs control to terminate the display of the sample position marks 350, the color of which has been set according to the population in the map area 21. Then, the process returns to the monitoring loop.

[0153] In step S211, the CPU 51 monitors whether the rate display check box 25 has been checked. If it detects that the user has checked the rate display check box 25, the CPU 51 proceeds from step S211 to step S267, and performs control to display a sample position mark 350, the color of which is set according to the stand rate, in the map area 21. Then, the process returns to the monitoring loop.

[0154] In step S212, the CPU 51 monitors whether the rate display check box 25 has been checked off. If it detects that the user has performed a check off operation, the CPU 51 proceeds from step S212 to step S268, and performs control to terminate the display of the sample position mark 350, the color of which has been set according to the stand rate, in the map area 21. Then, the process returns to the monitoring loop.

[0155] In step S213 in Fig. 31, the CPU 51 monitors whether the grid display check box 26 has been checked. If it detects that the user has performed a check-on operation, the CPU 51 proceeds from step S213 to step S269, and performs control to display a grid 355 in the map area 21. In this case, each area of ​​the grid 355 is displayed in the color set in the grid setting process in Fig. 34 described above. Then, the process returns to the monitoring loop. Note that a grid setting process must be performed before the grid 355 can be displayed. Therefore, even if the user checks the grid display check box 26 before operating the grid generation button 30, it may be possible to invalidate the check.

[0156] In step S214, the CPU 51 monitors whether the user has checked off the grid display check box 26. If it detects that the user has checked off the check box, the CPU 51 proceeds from step S214 to step S270, and performs control to end the display of the grid 355 in the map area 21. Then, the process returns to the monitoring loop.

[0157] In step S215, the CPU 51 monitors whether any of the displayed sample position marks 350 is designated (clicked, etc.). When an operation to designate a certain sample position mark 350 is detected, the CPU 51 proceeds from step S215 to step S271, and performs control to perform detailed display of that sample position mark 350. For example, as shown in Fig. 15 and Fig. 17, detailed numerical values ​​are displayed in the population display unit 35 and the stand rate display unit 36, and an image corresponding to that sample position mark 350 is displayed in the sample image display unit 37. Then, the process returns to the monitoring loop.

[0158] 15 and 17, the user may perform an operation to specify an image when an image is displayed in sample image display portion 37. In step S216, CPU 51 monitors the image specification operation. If an operation for designating a certain image is detected, CPU 51 advances from step S216 to step S272, and performs control so as to enlarge and display the designated image, as shown in FIG. During the enlarged image display, CPU 51 monitors the operation of end button 45, and when it detects the operation of end button 45, the process returns to the monitoring loop from step S273.

[0159] In step S217, the CPU 51 monitors an area selection operation of the grid 355. For example, this is an area click operation as described in Fig. 25 and Fig. 26, or a range designation operation as described in Fig. 28. Alternatively, some other selection operation may be performed. When an operation to select an area is detected, the CPU 51 proceeds from step S217 to step S280, and executes a display to indicate that the area has been selected, as a selection-enabled display, such as displaying the frame of the area in a thick line as shown in FIG. In step S281, the CPU 51 temporarily sets the designated area as a replant candidate area.

[0160] In step S282, the CPU 51 calculates replant-related values. For example, the CPU 51 calculates values ​​to be displayed in the selected area display section 73, the average population display section 74, and the average stand rate display section, and calculates cost values ​​to be displayed in the replant cost display section 76 in response to an input in the planning cost display section 72. The CPU 51 also displays windows and performs calculations in response to the calculation button 71. In step S283, the CPU 51 controls the display of these calculation results, and then returns to the monitoring loop.

[0161] 32, the CPU 51 monitors the operation of the ROI generation button 32. If the operation of the ROI generation button 32 is detected, the CPU 51 proceeds from step S218 to step S290 and generates an instruction file including instruction data of the replant area. This is a process of determining that one or more areas currently set as replant candidate areas are areas for replanting, and generating an instruction file for the tractor 270 to execute the replanting. In step S291, the CPU 51 performs a process for saving the generated instruction file, and then returns to the monitoring loop.

[0162] In step S219, the CPU 51 monitors the export operation. If a predetermined export operation is detected, the CPU 51 proceeds from step S219 to step S292, and performs output processing of the instruction file. For example, this is output processing for transferring the instruction file to the tractor 270. Then, the process returns to the monitoring loop.

[0163] By carrying out the above-mentioned processing, information is provided to the user by displaying the above-mentioned UI image, and an instruction file for replanting is generated in response to the user's operation. Note that the above processing example is just an example, and other processing examples are also possible. For convenience of explanation, not all processing regarding the UI screen and operation detection is shown, and other processing is also actually performed.

[0164] <7. User Interface Processing of the Second Embodiment> As a second embodiment, an example of recommending an area for replanting in a grid 355 will be described. For example, when the grid 355 is displayed, a recommended replant area 360 is presented as shown in FIG. For example, the CPU 51 determines areas with low stand rates or areas with low stand rates that are relatively concentrated, and when displaying the grid 355, displays a recommended area 360 as an area where replanting is recommended to the user.

[0165] In this case, the CPU 51 performs, for example, the process of Fig. 37 instead of the process of Fig. 31. That is, the process of step S275 is performed after the process of step S269. In the process of step S 275 , the CPU 51 sets a recommended replant area, and performs control to display this as a recommended area 360 on the grid 355 . For example, an area where the stand rate is equal to or lower than a predetermined value is determined as an area where replanting is recommended. Alternatively, if there are a predetermined number or more adjacent areas where the stand rate is equal to or lower than a predetermined value, these areas are determined as areas where replanting is recommended. The recommended area may be set, for example, when the grid setting process in FIG. 34 is performed.

[0166] By presenting the recommended area 360, the user can confirm the area in which replanting should be performed when the grid 355 is displayed, making it easier to make a decision on replanting.

[0167] <8. Summary and variations> According to the above embodiment, the following effects can be obtained. The information processing device 1 of the embodiment includes a calculation unit 5 that calculates a rate related to crop growth (e.g., a stand rate) from image data captured of a farm field 300, and a UI control unit 6 that controls the display of sample position marks 350, which are placed according to the imaging position of each image data in the map area 21 of the UI screen, in an image form (e.g., a color determined according to the rate) determined according to the rate calculated from each image data. For example, when the rate display check box 25 is checked and rate display is performed as in Figures 10 and 13, a sample position image showing the sample position is displayed on the map area 21, and each sample position image is displayed in a different color depending on the stand rate calculated for the area including their imaging range. This allows a user (for example, farm staff) to clearly and easily know the state of germination and growth in the vicinity of each sample position in the field 300. This is extremely useful information for making a decision on replanting. In the above example, the sample position image is a circular sample position mark 350, and the color differs depending on the stand rate, but the mark shape may differ depending on the stand rate, such as a square / triangle / circle. Also, even if the mark shape is the same, the brightness may differ depending on the stand rate. Of course, the image may have a composite color, brightness, and shape that differ depending on the stand rate. In the above example, the sample position images are displayed in three stages according to the stand rate, but they may be displayed in two stages or four or more stages, with each sample position image being displayed in a different image format. Although the stand rate is given as an example of a rate related to crop growth, the rate is not limited to this. For example, various rates can be calculated and displayed in a manner that reflects the type of crop, the growing environment, the planting environment, etc., such as the flowering rate, the growth rate, and the sun-exposed leaf rate (the ratio of the leaf area exposed to the sun). The field 300 may be not only cultivated land in outdoor farmland, but also land used for hydroponic cultivation or greenhouse cultivation, and the technology of the embodiment can be used to present information about crops grown in various locations. Furthermore, the technology of the embodiment can be used to present sensing results relating to the growth of, for example, fruit trees, weeds, etc., other than crops. Therefore, it can also be applied to remote sensing of forests, vacant lots, etc. Although the image data captured by the imaging device 220 has been given as an example of sensing data, the data is of course not limited to this, and may be detection data from various sensors such as detection data from a thermosensor or an ultrasonic sensor. Furthermore, the technology of the embodiment can be applied not only to plants but also to sensing the quality of soil in the farm field 300 and the like.

[0168] In the embodiment, an example has been given in which the stand rate is the ratio between the number of crops per unit area obtained from image data and the number of planting instructions per unit area for the imaging region of the image data. If information on the number of planting recommendations per acre for each location in the field 300 is present as the recommendation rate map 351, the rate of the number of stands per unit area relative to the number of planting recommendations per unit area can be calculated as a rate related to crop growth. By setting this as the stand rate, the germination / growth rate at each location in the field 300 can be checked on the UI screen against the farm staff's original expectations.

[0169] In the embodiment, an example has been given in which the stand rate is the ratio between the number of crops per unit area obtained from image data and the actual planting number per unit area for the imaging region of the image data. If information on the actual number of plantings per acre for each location in the field 300 exists as an actual rate map, the rate of the number of stands per unit area relative to the actual number of plantings per unit area can be calculated as a rate related to crop growth. By setting this as the stand rate, the germination / growth rate in relation to the actual planting number at each location in the field 300 can be checked on the UI screen.

[0170] In the embodiment, an example is given in which it is possible to select whether the stand rate is the ratio between the number of crops per unit area determined from image data and the number of planting instructions per unit area for the imaging area of ​​that image data, or the ratio between the number of crops per unit area determined from image data and the actual number of plantings per unit area for the imaging area of ​​that image data. In other words, it is possible to select whether the value serving as the basis for the rate is the instructed planting number based on the instructed rate map 351 or the actual planting number based on the actual rate map 352. The user can select the designated rate map 351 and the actual rate map 352 to display in the map area 21 (see Figs. 7 and 8) and use them as the basis for rate calculation. In other words, the user can select the basis for the rate calculation according to the purpose of the user.

[0171] In the embodiment, an example has been described in which the UI control unit 6 performs control to display, in the map area 21, a rate map image that shows the differences in values ​​used to calculate the stand rate in the field 300, together with the sample position mark 350. 12 and 13, for example, by displaying sample position marks 350 together with an indicated rate map 351 or an actual rate map 352, the user can check the population status and stand rate status by the sample position marks 350 while recognizing each area shown on the rate map. Therefore, information is presented that makes it easy to understand the growth status at each location in the field 300.

[0172] In the embodiment, an example has been described in which the UI control unit 6 performs control to display the grid 355 in the map region 21 as an area definition image that defines areas obtained by dividing the farm field 300. For example, as shown in Figures 18, 20, 21, etc., a grid 355 in a lattice pattern is displayed as an area definition image. This allows each area into which the field 300 is divided to be presented to the user as a range partitioned by the lattice (grid squares). In the above example, the area definition image is a grid pattern image (a lattice-like image) and square areas are defined as the cells of the lattice, but the image may also define areas as triangles, rectangles, polygons with 5 or more sides, or shapes with curves.

[0173] In the embodiment, an example has been described in which the UI control unit 6 performs control such that each area shown in the grid 355 is displayed in an image format determined according to the rate calculated for that area. For example, as shown in Figs. 18, 20, and 21, each square (i.e., each area) of the grid 355 is displayed in a color according to the rate calculated for that area. This allows the user to clearly and easily know the state of germination, etc. of each area. This is extremely useful information when considering where to replant. In other words, the user can use the grid 355 to plan the replant. In the image aspect of the grid 355, the color of each area represented by a square differs depending on the rate, but the image aspect is not limited to presenting the rate by color. For example, it is also possible to vary the brightness of the squares and their outlines, or to vary the type or color of the outlines of the squares. In the above example, the color of the squares is displayed in three stages according to the rate, but it may be displayed in two stages or four or more stages, with each area being displayed in a different image format.

[0174] In the embodiment, an example has been described in which the calculation unit 5 calculates the rate of each area indicated by the grid 355 by an interpolation calculation using the rate of crop growth for the image data related to that area (see FIG. 34). The grid 355 as the area definition image is displayed in different colors according to the stand rate, so that the growth status of each area can be recognized at a glance, and the stand rate can be calculated using the rate for the related sample (sensing data such as image data). This makes it possible to accurately express the rate for each area indicated by the grid 355, even if the imaging area of ​​each sample and the area divided and set by the grid 355 do not completely match. This also means that image data for each sample may be image data obtained by discretely capturing images of the field 300. In other words, there is no need to capture images of the entire field 300, and discrete images are sufficient, which reduces the processing load during flight and the amount of data by reducing the number of image data (samples), thereby improving the efficiency of processing.

[0175] In the embodiment, an example has been described in which sensing data related to a certain area to be calculated, for which a stand rate is referenced in the interpolation calculation, is image data captured within a predetermined distance from the center point of the certain area (see FIG. 35A). The stand rate of each area represented by a square in the grid 355 can be calculated with high accuracy by performing an interpolation calculation from the stand rates of all sample position images within a predetermined distance d1 from the center point of that area.

[0176] In the embodiment, an example has been described in which the sensing data related to a certain area to be calculated, which refers to a stand rate in the interpolation calculation, is a predetermined number of image data in the order of imaging areas closest to the center point of the certain area (see Figure 35B). The stand rate for each area represented by a square on the grid 355 can be calculated with high accuracy by performing an interpolation calculation from the stand rates of a predetermined number of samples in order of proximity to the center point of the area. In this case, the predetermined number of samples used for the calculation can be secured without fail.

[0177] In the embodiment, an example has been described in which the calculation unit 5 performs an interpolation calculation by weighting the rate of sensing data (image data) related to a certain area to be calculated, the sensing data (image data) capturing an area having the same number of planting instructions or number of actual plantings as the certain area (see Figure 35C). Image data captured in an area with the same planting instruction number or planting record number has similar conditions to the area for which the rate is calculated, so the rate for that image data is expected to be close to the rate for that area. Therefore, by weighting, the rate calculation for the area can be made more accurate.

[0178] In the embodiment, an example has been described in which the UI control unit 6 performs interface control for performing setting operations for the grid 355. For example, the setting dialog 33 in Fig. 19 is presented so that the user can specify the size of one side of the grid 355, which is an area definition image. This allows the user to set the area according to their convenience, and the grid 355 can be displayed in a way that is suitable for the replant plan.

[0179] In the embodiment, an example has been described in which the UI control unit 6 performs interface control for performing a designation operation for each area defined by the grid 355 as a replant candidate area. For example, as described with reference to FIGS. 23 to 26, a replant area candidate can be set by specifying a square in the grid 355 or by specifying a range. Users can select the replant locations while checking the rates displayed for each area using colors, etc., creating an interface suitable for the replant concept.

[0180] In the embodiment, an example has been described in which the UI control unit 6 performs interface control to designate each area defined by the grid 355 as a candidate area for replantation, and also performs control to present cost information related to replantation in response to the designation operation of the candidate area for replantation. For example, as described in Fig. 23 to Fig. 26, when considering a replant candidate area, the cost is displayed. This allows the user to consider the scope of replant and whether to perform it while referring to the cost. The information processing device 1 can provide the user with extremely useful information when considering replant.

[0181] In the embodiment, an example has been described in which the UI control unit 6 performs interface control that enables each area defined by the grid 355 to be individually designated as a replant candidate area. For example, as explained in FIG. 23 and FIG. 24, the area to be replanted can be selected by specifying a square in the grid 355, which is suitable for the task of considering the range to be replanted.

[0182] In the embodiment, an example has been described in which the UI control unit 6 performs interface control that enables a plurality of areas to be designated as replant candidate areas by a range designation operation on the image of the grid 355. For example, by specifying the range as described in FIG. 26, multiple areas to be replanted can be easily specified.

[0183] The information processing device 1 according to the embodiment includes a file generating unit 9 that generates a replant instruction file in response to determination of a replant execution area in the grid 355 (see step S290 in FIG. 32). This generates file data including the ROI to be transferred to the tractor 270. In other words, a control file for executing the replant planned using the grid 355 on the UI screen is automatically generated. Although the file output in step S292 in FIG. 32 is output to the tractor 270, it is also possible to generate an instruction file for the flying object 200 to fly again and transfer it to the flying object 200 as control data. For example, this technology can also be applied when re-imaging an area that has been determined to be a replant candidate area to confirm the situation.

[0184] In the second embodiment, an example has been described in which the UI control unit 6 performs control to present the determination result as to whether or not replanting is to be performed for each area shown in the grid 355. That is, an example has been described in which a recommended replanting area 360 is displayed. For example, as explained in Fig. 36 and Fig. 37, a display that clearly indicates areas where replanting is recommended may be provided. This serves as a guide for the user in deciding whether to replant and selecting an area, and also makes it easier for the user to decide whether to replant and select an area.

[0185] In the embodiment, an example has been given in which the imaging device 220 is mounted on the flying object 200 that can be wirelessly controlled or automatically controlled. Examples of the flying object 200 that can be radio-controlled or automatically controlled include so-called drones, small radio-controlled fixed-wing airplanes, and small radio-controlled helicopters.

[0186] The program of the embodiment is a program that causes the information processing device 1 to execute a process of calculating rates related to crop growth (stand rate, etc.) from image data captured of the field 300, and a control process of displaying sample position marks 350, which are placed according to the capturing position of each image data, in the map area 21 of the UI screen in an image format determined according to the rate calculated from each image data. That is, it is a program for causing an information processing device to execute the processes described with reference to FIGS. 27 to 34 and 36.

[0187] Such a program makes it easy to realize the information processing device 1 of the present embodiment. Such a program can be stored in advance in a recording medium built into a device such as a computer device, or in a ROM in a microcomputer having a CPU. Alternatively, the program can be temporarily or permanently stored (stored) in a removable recording medium such as a semiconductor memory, a memory card, an optical disk, a magneto-optical disk, or a magnetic disk. Such a removable recording medium can be provided as a so-called package software. Furthermore, such a program can be installed in a personal computer or the like from a removable recording medium, or can be downloaded from a download site via a network such as a LAN or the Internet.

[0188] It should be noted that the effects described in this specification are merely examples and are not limiting, and other effects may also be obtained.

[0189] The present technology can also be configured as follows. (1) a calculation unit that calculates a rate related to plant growth from the sensing data; a user interface control unit that controls displaying a sample position image arranged according to the sensing position of each sensing data in a map area of ​​a user interface screen in an image form determined according to a rate calculated from each sensing data. Information processing device. (2) The sensing data is image data obtained by capturing an image of a farm field, The rate is The number of crops per unit area calculated from the image data, The ratio of the number of planting instructions per unit area to the imaging area of ​​the image data. The information processing device according to (1) above. (3) The sensing data is image data obtained by capturing an image of a farm field, The rate is The number of crops per unit area calculated from the image data, The ratio of the number of plantings per unit area to the imaging area of ​​the image data. The information processing device according to (1) above. (4) The sensing data is image data obtained by capturing an image of a farm field, The rate is the ratio between the number of crops per unit area obtained from the image data and the number of planting instructions per unit area for the image capture region of the image data; The ratio between the number of crops per unit area obtained from the image data and the number of planted crops per unit area for the image capture region of the image data, or It is possible to select The information processing device according to (1) above. (5) The user interface control unit includes: In the map area, a rate map image showing differences in values ​​used to calculate the rate in the field is displayed together with the sample position image. An information processing device according to any one of (1) to (4) above. (6) The user interface control unit includes: In the map area, control is performed to display an area definition image that defines an area into which the farm field is divided. An information processing device according to any one of (1) to (5) above. (7) The user interface control unit includes: Each area shown in the area definition image is controlled to be displayed in an image format determined according to the rate calculated for that area. The information processing device according to (6) above. (8) The calculation unit is The rate of each area shown in the area definition image is calculated by an interpolation calculation using the rate of crop growth of the sensing data related to the area. The information processing device according to (7) above. (9) The sensing data relating to an area, which refers to a rate of crop growth in the interpolation calculation, This is sensing data obtained by sensing within a predetermined distance from the center point of the area. The information processing device according to (8) above. (10) The sensing data relating to an area, which refers to a rate of crop growth in the interpolation calculation, The imaging area is a predetermined number of sensing data in order of proximity to the center point of the one area. The information processing device according to (8) above. (11) The calculation unit performs the interpolation calculation by weighting a rate for sensing data obtained by sensing an area having the same planting instruction number or planting record number as the one area, among sensing data related to the one area. An information processing device according to any one of (8) to (10) above. (12) The user interface control unit includes: The interface control is performed to perform the setting operation of the area definition image. An information processing device according to any one of (6) to (11) above. (13) The user interface control unit includes: For each area defined in the area definition image, an interface control is performed to specify a replant candidate area. An information processing device according to any one of (6) to (12) above. (14) The user interface control unit includes: For each area defined in the area definition image, an interface control is performed to perform an operation of designating a replant candidate area, and in response to the operation of designating a replant candidate area, a control is performed to present cost information related to replant. The information processing device according to (13) above. (15) The user interface control unit includes: An interface control is performed to allow each area defined in the area definition image to be individually designated as a replant candidate area. The information processing device according to (13) or (14) above. (16) The user interface control unit includes: An interface control is performed to allow a plurality of areas to be designated as replant candidate areas by a range designation operation on the area definition image. An information processing device according to any one of (13) to (15) above. (17) A file generating unit that generates a replant instruction file in response to determination of a replant execution area in the area definition image. An information processing device according to any one of (6) to (16) above. (18) The user interface control unit includes: For each area shown in the area definition image, control is performed to present a determination result of whether or not to replant. An information processing device according to any one of (6) to (17) above. (19) Calculate the rate of plant growth from the sensing data, In the map area of ​​the user interface screen, control is performed to display the sample position images arranged according to the sensing positions of the sensing data in an image format determined according to the rate calculated from the sensing data. Information processing methods. (20) A process of calculating a rate of plant growth from the sensing data; a control process for displaying, in a map area of ​​a user interface screen, sample position images arranged according to the sensing positions of the sensing data, in an image format determined according to a rate calculated from the sensing data; A program for causing an information processing device to execute the above. [Explanation of symbols]

[0190] 1. Information processing device 2. Image acquisition section 5 Calculation section 6 UI control section 7 Recording control section 8. Communication Control Unit 9 File Generation Section 21 Map Area 22 Map Selection Section 23 Map display checkbox 24 Count display checkbox 25 Rate display checkbox 26 Grid display checkbox 27 Date and time selection section 28 Zoom button 30 Grid generation button 31 Rate calculation button 32 ROI generation button 35 Population display section 36 Stand rate display unit 37 Sample image display section 51 CPU 71 Calculation button 72 Planning cost display section 73 Selected area display section 74 Average Population Display 75 Average Stand Rate Display 76 Replant cost display unit 200 Flying Objects 220 Imaging Device 270 Tractor 300 fields 350 Sample Position Mark 351 Indicated Rate Map 352 Actual Rate Map 355 Grid 360 Recommended Area

Claims

1. A method and device for detecting a rate of plant growth from sensing data obtained by capturing an image of a farm field, comprising: a user interface control unit that controls displaying, in a map area of ​​a user interface screen, sample position images arranged according to positions at which each piece of sensing data was captured, in an image format determined according to a rate calculated from each piece of sensing data; The rate is The number of crops per unit area calculated from the image data, is the ratio of the number of planting instructions per unit area to the imaging area of ​​the image data, The user interface control unit includes: In the map area, a rate map image showing differences in values ​​used to calculate the rate in the field is displayed together with the sample position image. Information processing device.

2. A calculation unit that calculates a rate of plant growth from sensing data as image data of an image of a farm field; a user interface control unit that controls displaying, in a map area of ​​a user interface screen, sample position images arranged according to positions at which each piece of sensing data was captured, in an image format determined according to a rate calculated from each piece of sensing data; The rate is The number of crops per unit area calculated from the image data, The ratio of the number of plantings per unit area to the imaging area of ​​the image data, The user interface control unit includes: In the map area, a rate map image showing differences in values ​​used to calculate the rate in the field is displayed together with the sample position image. Information processing device.

3. A calculation unit that calculates a rate of plant growth from sensing data as image data of an image of a farm field; a user interface control unit that controls displaying, in a map area of ​​a user interface screen, sample position images arranged according to positions at which each piece of sensing data was captured, in an image format determined according to a rate calculated from each piece of sensing data; The rate is the ratio between the number of crops per unit area obtained from the image data and the number of planting instructions per unit area for the image capture region of the image data; The ratio between the number of crops per unit area obtained from the image data and the number of planted crops per unit area for the image capture region of the image data, or It is possible to select The user interface control unit includes: In the map area, a rate map image showing differences in values ​​used to calculate the rate in the field is displayed together with the sample position image. Information processing device.

4. The user interface control unit includes: In the map area, control is performed to display an area definition image that defines an area into which the farm field is divided. The information processing device according to claim 1 .

5. The user interface control unit includes: Each area shown in the area definition image is controlled to be displayed in an image format determined according to the rate calculated for that area. The information processing device according to claim 4.

6. The calculation unit is The rate of each area shown in the area definition image is calculated by an interpolation calculation using the rate of crop growth of the sensing data related to the area. The information processing device according to claim 5 .

7. The sensing data relating to an area, which refers to a rate of crop growth in the interpolation calculation, This is sensing data obtained by sensing within a predetermined distance from the center point of the area. The information processing device according to claim 6.

8. The sensing data relating to an area, which refers to a rate of crop growth in the interpolation calculation, The imaging area is a predetermined number of sensing data in order of proximity to the center point of the one area. The information processing device according to claim 6.

9. The calculation unit performs the interpolation calculation by weighting a rate for sensing data obtained by sensing an area having the same planting instruction number or planting record number as the one area, among sensing data related to the one area. The information processing device according to claim 6.

10. The user interface control unit includes: The interface control is performed to perform the setting operation of the area definition image. The information processing device according to claim 4.

11. The user interface control unit includes: For each area defined in the area definition image, an interface control is performed to specify a replant candidate area. The information processing device according to claim 4.

12. The user interface control unit includes: For each area defined in the area definition image, an interface control is performed to perform an operation of designating a replant candidate area, and in response to the operation of designating a replant candidate area, a control is performed to present cost information related to replant. The information processing device according to claim 11.

13. The user interface control unit includes: An interface control is performed to allow each area defined in the area definition image to be individually designated as a replant candidate area. The information processing device according to claim 11.

14. The user interface control unit includes: An interface control is performed to allow a plurality of areas to be designated as replant candidate areas by a range designation operation on the area definition image. The information processing device according to claim 11.

15. A file generating unit that generates a replant instruction file in response to determination of a replant execution area in the area definition image. The information processing device according to claim 4.

16. The user interface control unit includes: For each area shown in the area definition image, control is performed to present a determination result of whether or not to replant. The information processing device according to claim 4.

17. A method for calculating a rate of plant growth, which is a ratio between the number of crops per unit area determined from the image data and the number of planting instructions per unit area for an imaged region of the image data, from sensing data in the form of image data of a farm field; displaying, in a map area of ​​a user interface screen, sample position images arranged according to the captured positions of the respective sensing data in an image format determined according to a rate calculated from the respective sensing data; A control for displaying, in the map area, a rate map image showing differences in values ​​used to calculate the rate in the field together with the sample position image. The information processing method is executed by an information processing device.

18. A process for calculating a rate of plant growth, which is a ratio between the number of crops per unit area determined from the image data and the number of planting instructions per unit area for the imaged region of the image data, from sensing data as image data of an image of a farm field; a control process for displaying, in a map area of ​​a user interface screen, sample position images arranged according to positions where the respective sensing data were captured, in an image format determined according to a rate calculated from the respective sensing data; a control process for displaying, in the map area, a rate map image presenting differences in values ​​used to calculate the rate in the farm field together with the sample position image; A program for causing an information processing device to execute the above.

19. A method for calculating a rate of plant growth, which is a ratio between the number of crops per unit area determined from the image data and the number of actual plantings per unit area relative to the imaged region of the image data, from sensing data in the form of image data of a farm field; displaying, in a map area of ​​a user interface screen, sample position images arranged according to the captured positions of the respective sensing data in an image format determined according to a rate calculated from the respective sensing data; A control for displaying, in the map area, a rate map image showing differences in values ​​used to calculate the rate in the field together with the sample position image. The information processing method is executed by an information processing device.

20. A process for calculating a rate of plant growth, which is a ratio between the number of crops per unit area determined from the image data and the number of actual plantings per unit area relative to the imaged region of the image data, from sensing data as image data of an image of a farm field; a control process for displaying, in a map area of ​​a user interface screen, sample position images arranged according to positions where the respective sensing data were captured, in an image format determined according to a rate calculated from the respective sensing data; a control process for displaying, in the map area, a rate map image presenting differences in values ​​used to calculate the rate in the farm field together with the sample position image; A program for causing an information processing device to execute the above.

Images