Determination device, determination method, and program

Abstract

To provide a detection device that can easily determine the presence or absence of plants. [Solution] The determination device (1) includes a first acquisition unit (13) for acquiring a vegetation index image, a second acquisition unit (14) for acquiring vegetation information, a pre-processing unit (16) for performing processing to acquire the maximum value of the vegetation index within a plant area indicating the location where a plant is planted, a determination unit (17) for determining representative values ​​of the vegetation index for multiple plant areas, and a determination unit (19) for determining whether or not each plant area is missing a plant.

Description

Technical Field

[0001] The present invention relates to a determination device, a determination method, and a program.

Background Art

[0002] In recent years, the introduction of AI (Artificial Intelligence) technology into agriculture has been promoted. For example, Non-Patent Document 1 describes a method for counting the number of corn plants by inputting an aerial image acquired using a drone into a trained model that uses a neural network. This makes it possible to estimate the number of missing corn plants.

Prior Art Documents

Non-Patent Documents

[0003]

Non-Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] However, the method for counting the number of plants in Non-Patent Document 1 has a problem that it takes time and effort to create a trained model.

[0005] An object of the present invention is to provide a determination device, a determination method, and a program that enable easy determination of missing plants.

Means for Solving the Problems

[0006] To solve the above problems, a determination device according to one aspect of the present invention is a determination device for determining missing plants of crops in a field, comprising: a first acquisition unit that acquires a vegetation index image representing the vegetation index of the field; a second acquisition unit that acquires vegetation information indicating a vegetation area including a plurality of rows in which the crops are planted, the starting and ending positions of planting the crop plants for each row, and the spacing between the plants for each row; a pre-processing unit that performs processing to obtain the maximum value of the vegetation index in a plant area indicating the location where the plants are planted from the vegetation index image based on the vegetation information; a determination unit that determines representative values ​​of the vegetation index values ​​for a plurality of plant areas; and a determination unit that determines whether each plant area is missing plants by comparing each of the representative values ​​determined by the determination unit with a predetermined threshold.

[0007] A determination method according to one aspect of the present invention is a determination method for determining missing plants of crops in a field using a determination device, comprising: a first acquisition step of acquiring a vegetation index image with the first acquisition unit; and a second acquisition step of acquiring vegetation information with the second acquisition unit. The preprocessing unit performs a processing operation based on the vegetation information to obtain the maximum value of the vegetation index in a plant area indicating the location where the plant is planted from the vegetation index image; the determination unit determines representative values ​​of the vegetation index in a plurality of plant areas; and the determination unit determines whether each plant area is missing a plant by comparing each of the representative values ​​determined in the determination step with a predetermined threshold. [Effects of the Invention]

[0008] According to one aspect of the present invention, it is possible to easily determine whether or not a plant is missing. [Brief explanation of the drawing]

[0009] [Figure 1] This is a block diagram showing the overall configuration of the determination device according to Embodiment 1 of the present invention. [Figure 2]This flowchart shows an example of the processing flow by the control unit of the determination device according to Embodiment 1. [Figure 3] Figure 2 is a flowchart showing an example of the process for determining the representative value of the vegetation index. [Figure 4] Figure 2 is a flowchart showing an example of the process for determining missing plants. [Figure 5] This figure shows an example of an orthomosaic image and a vegetation index image generated by the determination device according to Embodiment 1. [Figure 6] This is an illustrative diagram illustrating the process by which the determination device according to Embodiment 1 determines a representative value of the vegetation index within each plant area. [Figure 7] This is a diagram illustrating the process of marking the location of a plant using the determination device according to Embodiment 1. [Figure 8] This graph shows an example of the density distribution of representative values ​​of the vegetation index determined by the threshold setting unit of the determination device according to Embodiment 1. [Figure 9] This figure shows an example of how the missing plant count is calculated by the missing plant calculation unit of the determination device according to Embodiment 1. [Figure 10] This figure shows an example of the location of a missing plant identified by the location identification unit of the determination device according to Embodiment 1. [Figure 11] This flowchart shows an example of the process flow for determining a representative value of the vegetation index by the control unit of the determination device according to Embodiment 2. [Figure 12] This figure illustrates the process of creating a mesh in step S18 of Figure 11. [Modes for carrying out the invention]

[0010] [Embodiment 1] Hereinafter, the determination device 1 according to Embodiment 1 of the present invention will be described with reference to Figures 1 to 10.

[0011] [Configuration of the judgment device] Figure 1 is a block diagram showing the overall configuration of a determination device 1. As shown in Figure 1, the determination device 1 includes a control unit 10, a storage unit 30, an input unit 40, and a display unit 50. The control unit 10 is electrically connected to the storage unit 30, the input unit 40, and the display unit 50. The determination device 1 is a device for determining missing plants of crops planted in a field.

[0012] Examples of the crops include corn, lettuce and the like, which have a lower missing plant compensation capability, that is, the capability to compensate for missing plants and develop leaves, than rice and the like. Missing plants in such corn, lettuce and the like with low compensation capability directly affect the crop yield. The determination device 1 is used for estimating the number of missing plants of crops such as corn and lettuce, and predicting the crop yield, etc.

[0013] Note that one or more ridges are formed in the field. Each ridge is provided with one or more rows. It is assumed that a plurality of plants of a specific crop are planted in each row at predetermined intervals. Note that the ridges may be omitted, as long as one or more rows are provided in the field.

[0014] The control unit 10 is configured to include, for example, a CPU (Central Processing Unit) and the like. The control unit 10 controls the operation of each part of the determination device 1 by executing the control program stored in the storage unit 30.

[0015] The storage unit 30 is configured to include, for example, a ROM (Read Only Memory), a RAM (Random Access Memory) and the like. The storage unit 30 stores the above-mentioned control program. The storage unit 30 also stores, for example, an aerial image acquired by a third acquisition unit 15, or a threshold value used when a determination unit 19 performs missing plant determination.

[0016] The input unit 40 is an interface for performing operations on or inputting information to the determination device 1. For example, the input unit 40 is configured to include a keyboard, a mouse, a touch panel, a tablet, and the like that accept operations on the determination device 1 from a worker. The input unit 40 inputs, for example, an aerial image of a farm field captured by a camera mounted on an aircraft 60 to the third acquisition unit 15.

[0017] The input unit 40 and the aircraft 60 may be connected wirelessly or by wire. Further, the input unit 40 may input the aerial image to the third acquisition unit 15 via a terminal device such as a tablet terminal equipped with an application that receives and displays the aerial image transmitted from the aircraft 60.

[0018] The display unit 50 is a display panel that displays information output from the output unit 23. The display unit 50 may be configured to have a partial function of the input unit 40 as a touch panel. In addition to the display unit 50, the determination device 1 may further include a speaker that outputs information related to the determination result by the determination device 1 as audio, or a printing device that prints the information related to the determination result.

[0019] The aircraft 60 is, for example, an Unmanned Aerial Vehicle (UAV). Specifically, as the aircraft 60, a drone such as Phantom 4 Multispectral (trade name) manufactured by DJI JAPAN Co., Ltd. can be used.

[0020] The aircraft 60 flies over the farm field. The aircraft 60 flies over the farm field by, for example, an operation of a worker. The aircraft 60 may have a function of detecting its own position using a positioning system such as GPS (Global Positioning System). In this case, the aircraft 60 is capable of autonomous flight based on its own position information, map information of the farm field, a planned flight path, and the like.

[0021] Furthermore, the aircraft 60 is equipped with camera C. Camera C is, for example, a multispectral camera. Camera C takes multiple aerial images of the crop-planted field from above. The multiple aerial images only need to be images that can be combined to form an orthomosaic image that captures all the vegetation areas where the crops are planted. Camera C may also be an RGB camera.

[0022] (Configuration of the control unit) Next, the configuration of the control unit 10 will be described with reference to Figure 1. As shown in Figure 1, it includes an orthomosaic image generation unit 11, a vegetation index image generation unit 12, a first acquisition unit 13, a second acquisition unit 14, a third acquisition unit 15, a preprocessing unit 16, a determination unit 17, a threshold setting unit 18, a judgment unit 19, a missing plant calculation unit 20, a location identification unit 21, an estimation unit 22, and an output unit 23.

[0023] The third acquisition unit 15 acquires multiple aerial images taken by the camera C mounted on the aircraft 60 via the input unit 40. The multiple aerial images are images of the field taken by the camera C from different positions above the field.

[0024] The orthomosaic image generation unit 11 generates an orthomosaic image (orthomosaic image) from multiple aerial images acquired by the third acquisition unit 15. For example, the orthomosaic image generation unit 11 decomposes multiple aerial images into a large number of point clouds, calculates the positional information of each point cloud from the appearance of each point cloud in each aerial image, and corrects the amount of distortion that occurs as you move away from the center of the aerial image, thereby generating an orthomosaic image of the entire field as seen from directly above.

[0025] The vegetation index image generation unit 12 generates a vegetation index image representing the vegetation index of the field from the orthoimage generated by the orthoimage generation unit 11 (see Figure 5). The vegetation index is, for example, NDVI (Normalized Difference Vegetation Index). The vegetation index image generation unit 12 generates, for example, an NDVI image.

[0026] Vegetation indices are indicators used to understand the state of vegetation by utilizing the characteristics of light reflection by plants, including crops. For example, the NDVI value is calculated using the following formula (1). NDVI=(NIR-RED) / (NIR+RED) (Formula 1)

[0027] Here, NIR indicates the reflected light intensity in the near-infrared region, and RED indicates the reflected light intensity in the visible red region. The NDVI value ranges from "-1" to "+1". In an NDVI image, each pixel has an NDVI value assigned to it. Areas with a high NDVI value indicate a high amount and activity of plants, and are vegetated areas where crops are planted. On the other hand, areas with a low NDVI value indicate a low amount and activity of plants, and are soil or other areas outside of vegetation.

[0028] In addition to NDVI, other vegetation indices such as NDRE (Normalized Difference Red Edge), LCI (Leaf Chlorophyll Index), GNDVI (Green Normalized Difference Vegetation Index), and OSAVI (Optimized Soil-adjusted Vegetation Index) may also be used.

[0029] The first acquisition unit 13 acquires the vegetation index image generated by the vegetation index image generation unit 12. In this case, the first acquisition unit 13 acquires the NDVI image.

[0030] The second acquisition unit 14 acquires vegetation information via the input unit 40. Here, the vegetation information includes information about the vegetation area containing multiple rows in which crops are planted, the spacing between rows, the starting and ending positions of the crop plants in each row, and the spacing between plants in each row.

[0031] The preprocessing unit 16 performs processing on the NDVI image, which is a vegetation index image, to obtain the maximum value of the vegetation index within the plant area, which indicates the location where the plant is planted, based on the vegetation information obtained by the second acquisition unit 14. Alternatively, the preprocessing unit 16 may divide the vegetation index image into multiple rectangular sections corresponding to the spacing between plants, such that each rectangular section contains one plant area, and obtain the maximum value of the vegetation index for each rectangular section.

[0032] The determination unit 17 determines representative values ​​for the vegetation index of multiple plant areas based on the processing performed by the preprocessing unit 16. The threshold setting unit 18 sets the threshold used by the determination unit 19 to determine whether a plant is missing.

[0033] The determination unit 19 determines whether or not there are missing plants at the locations where plants included in each plant area are planted. Specifically, the determination unit 19 determines whether or not there are missing plants corresponding to each plant area by comparing the representative value of each plant area with a threshold set by the threshold setting unit 18. The determination unit 19 may use predetermined thresholds.

[0034] The missing plant calculation unit 20 calculates the number of missing plants in the field. Specifically, the missing plant calculation unit 20 calculates the number of missing plants in the field by counting the number of plant areas determined to be missing plants by the determination unit 19. The location identification unit 21 identifies the location of the missing plants in the field based on the determination result of the determination unit 19 (see Figure 10).

[0035] The estimation unit 22 estimates the amount of work and time required for replanting based on the number of missing plants calculated by the missing plant calculation unit 20 and the location information of the missing plants identified by the location identification unit 21. Replanting refers to planting new plants in the location of missing plants. Replanting also includes sowing new seeds of crops in the location of missing plants.

[0036] The output unit 23 outputs information such as the number of missing plants calculated by the missing plant calculation unit 20, the location of the missing plants identified by the location identification unit 21, and the amount of replanting work estimated by the estimation unit 22. The output unit 23 displays, for example, an image on the display unit 50 showing the number of missing plants, the location of the missing plants, the amount of replanting work, and the work time.

[0037] [Processing flow by the control unit] Next, the processing flow by the control unit 10 of the determination device 1 will be explained with reference to Figures 2 to 10. First, the operator operates the aircraft 60 to fly it over the field where the crops are planted, and the camera C mounted on the aircraft 60 takes an aerial photograph of the field.

[0038] Figure 2 is a flowchart showing an example of the processing flow by the control unit 10 of the determination device 1. In the flowchart shown in Figure 2, the third acquisition unit 15 of the control unit 10 acquires multiple aerial images of the field taken by the camera C via the input unit 40 (step S1).

[0039] After step S1, the orthomosaic image generation unit 11 generates an orthomosaic image from the multiple aerial images acquired by the third acquisition unit 15 as described above (step S2). Here, Figure 5 shows an example of an orthomosaic image G1 and a vegetation index image G2.

[0040] As shown in Figure 5, after step S2, the vegetation index image generation unit 12 generates a vegetation index image G2 from the orthoimage G1 generated by the orthoimage generation unit 11 (step S3). The vegetation index image G2 is, for example, an NDVI image.

[0041] After step S3, the first acquisition unit 13 acquires the vegetation index image G2 generated by the vegetation index image generation unit 12 (step S4: first acquisition step). In step S4, the first acquisition unit 13 may also acquire a vegetation index image generated by an external device other than the determination device 1.

[0042] After step S4, the second acquisition unit 14 acquires the vegetation information described above via the input unit 40 (step S5: second acquisition step). Note that the order of steps S4 and S5 may be reversed.

[0043] After step S5, the preprocessing unit 16 performs the process of determining the representative value of the vegetation index shown in Figure 3 (step S6: determination step). Here, Figure 3 is a flowchart showing an example of the flow of the process of determining the representative value of the vegetation index shown in Figure 2.

[0044] In the flowchart shown in Figure 3, the operator first creates vector data containing the positional information of all plants by plotting points at intervals between plants, from the starting position to the ending position of each plant, on the orthomosaic image generated by the orthomosaic image generation unit 11 via the input unit 40 (step S10). Note that the processing in step S10 may be performed automatically by the control unit 10 based on the orthomosaic image and vegetation information.

[0045] After step S10, the preprocessing unit 16 refers to the vegetation information acquired by the second acquisition unit 14 and expands the NDVI image, which is a vegetation index image, by the maximum value in order to obtain the maximum value of the vegetation index within the plant area where the plants are planted.

[0046] The preprocessing unit 16 may divide the vegetation index image into multiple rectangular sections corresponding to the spacing between plants, such that each rectangular section contains one plant area, and obtain the maximum value of the vegetation index for each rectangular section.

[0047] Subsequently, the preprocessing unit 16 performs a processing operation in which it replaces the vegetation index value within the plant area with the maximum value of the vegetation index within the plant area by using focal statistical processing (step S11).

[0048] Here, Figure 6 is an illustrative diagram illustrating the process by which the determination device 1 determines a representative value of the vegetation index within each plant area. In Figure 6, image P shows the initial state. Images P1A, P2A, P3A, and P4A show the vegetation index of each pixel before focal statistical processing of the framed area. On the other hand, images P1B, P2B, P3B, and P4B show the vegetation index of each pixel after focal statistical processing of the framed area.

[0049] In step S11, the preprocessing unit 16 expands the maximum value of the vegetation index by shifting the plant area, which consists of a rectangle with one side set to be less than or equal to the plant spacing, as shown in the upper diagram of Figure 6, and setting the maximum value within it to the center. The plant spacing differs depending on the type of crop. For example, in the case of corn, the plant spacing is about 30 cm.

[0050] After step S11, the determination unit 17 determines the maximum value of the vegetation index within each plant area as the representative value of the vegetation index for each plant area (step S12). Specifically, as shown in Figure 6, the determination unit 17 uses focal statistical processing to determine the maximum value of the vegetation index in each plant area as the representative value of the vegetation index included in that plant area.

[0051] The example shown in the upper part of Figure 6 represents the vegetation index of each pixel in the plant region P before focal statistical processing. Images P1A, P1B, P2A, P2B, P3A, P3B, P4A, and P4B in Figure 6 illustrate an example of the focal statistical processing flow. In this example, a 9-square grid is used, but other shapes such as circles can also be used.

[0052] Here, Image Q in Figure 6 shows the result of focal statistical processing of all pixels in the vegetation index image. As shown in Image Q in Figure 6, by using focal statistical processing, it is possible to simplify the calculations in the plant deficiency detection process described later and reduce the processing time required to detect plant deficiencies. Furthermore, by expanding the vegetation area, the ability of the detection device 1 to respond to changes in the shape of crops, such as when crops are tilted during imaging by camera C, can be improved.

[0053] After step S6, the control unit 10 executes the plant deficiency detection process shown in Figure 4 (step S7). Here, Figure 4 is a flowchart showing an example of the flow of the plant deficiency detection process shown in Figure 2.

[0054] The threshold setting unit 18 determines the density distribution of the representative value of the vegetation index determined by the determination unit 17 (step S13). Here, Figure 8 is a graph showing an example of the density distribution of the representative value of the vegetation index determined by the threshold setting unit 18.

[0055] As shown in Figure 8, the threshold setting unit 18 uses curve fitting (e.g., kernel density estimation, Gaussian mixture distribution, or other cluster analysis methods) on the density distribution of representative values ​​of the vegetation index NDVI to determine the threshold for dividing the curve obtained by fitting. This threshold is determined using methods such as mean squared error to minimize the error between classes, cluster classification using unsupervised learning, or the minimum value between two peaks to speed up processing (step S14). In the example shown in Figure 8, the threshold setting unit 18 sets the threshold to "0.353".

[0056] After step S14, the determination unit 19 compares the representative value of each stock area with the threshold set by the threshold setting unit 18. Based on this, the determination unit 19 determines for each stock area whether or not there are missing stocks in that area (step S15: determination step). Specifically, the determination unit 19 determines that there are missing stocks in a stock area if the representative value of that stock area is less than the threshold.

[0057] After step S15, the missing plant calculation unit 20 calculates the number of missing plants in the field that is the target of the determination device 1 by counting the number of plant regions that the determination unit 19 has determined to be missing plants (step S16).

[0058] Figure 9 shows an example of how the missing plant calculation unit 20 calculates the number of missing plants. In the example shown in Figure 9, the missing plant calculation unit 20 calculates that plants contained within the four plant areas enclosed by square frames are missing plants in a field divided into 12 plant areas.

[0059] After step S16, the location identification unit 21 identifies the location of the missing plant by comparing the location information of the plant area determined to be missing by the determination unit 19 with the vegetation information acquired by the second acquisition unit (step S17).

[0060] Here, Figure 10 shows an example of the location of a missing plant identified by the location identification unit 21. In the example shown in Figure 10, the location identification unit 21 has identified that the plant at the location of the black circle is missing, and that the plant at the location of the white circle exists. In this way, the location identification unit 21 can identify the location of a missing plant.

[0061] Next, returning to Figure 2, after step S8, the estimation unit 22 estimates the amount of replanting work to be done by the worker based on the number of missing plants calculated by the missing plant calculation unit 20 and the location information of the missing plants identified by the location identification unit 21 (step S8).

[0062] In step S8, the estimation unit 22 determines that the greater the number of missing plants calculated by the missing plant calculation unit 20, the greater the amount of replanting work and the longer the replanting work time. The estimation unit 22 also determines that the more scattered the locations of the missing plants identified by the location identification unit 21 are, the longer the amount of replanting work and the longer the work time will be.

[0063] After step S8, the output unit 23 outputs information regarding the number of missing plants calculated by the missing plant calculation unit 20, the location of the missing plants identified by the location identification unit 21, and the amount of replanting work and work time estimated by the estimation unit 22 to the display unit 50 as the determination result of the determination device 1 (step S9). The display unit 50 displays an image showing the number of missing plants, the location of the missing plants, the amount of replanting work, and the work time. In this way, the control processing by the control unit 10 of the determination device 1 is completed.

[0064] According to the determination device 1 in Embodiment 1 described above, it is not necessary to use a trained model to determine missing plants, so there is no need to create a trained model, and missing plants can be easily determined. Specifically, the determination unit 17 determines a representative value of the vegetation index for each of the multiple plant areas into which the field is divided, and the determination unit 19 can determine whether or not the plants included in each plant area are missing by comparing the representative value of the plant area with a threshold.

[0065] Furthermore, the missing plant calculation unit 20 can calculate the number of missing plants in the field by counting the number of plant areas determined to be missing plants by the determination unit 19. This allows the operator to grasp an approximate value of missing plants in the field and predict the harvest and shipment volume of the crop.

[0066] Furthermore, the location identification unit 21 can pinpoint the location of missing plants in the field. This allows workers to understand the location of missing plants and take appropriate measures such as replanting.

[0067] Furthermore, the estimation unit 22 can estimate the amount of replanting work based on the number of missing plants calculated by the missing plant calculation unit 20 and the location information of the missing plants identified by the location identification unit 21. This allows the worker to know in advance how long the replanting work will take.

[0068] Furthermore, the threshold setting unit 18 uses curve fitting on the density distribution of representative values ​​of the vegetation index for each plant area, and sets the minimum value of the curve fitting as the threshold. This makes it possible to set a good threshold for distinguishing between vegetation areas where crops are planted and areas with missing plants, thereby improving the accuracy of the determination unit 19 in determining missing plants.

[0069] Furthermore, by acquiring aerial images of a wide area of ​​the field using the camera C mounted on the aircraft 60 and determining missing plants, the labor required to identify the number and location of missing plants can be reduced.

[0070] Furthermore, by including crops such as corn, which have a lower ability to compensate for missing plants compared to rice, as targets for detection of missing plants in the detection device 1, a decrease in crop yield can be detected at an early stage. Moreover, unlike detection methods using pre-trained models, the target crops are not limited to corn, and the detection device 1 can be applied to detecting missing plants in other crops such as lettuce, thus increasing the versatility of the detection device 1.

[0071] [Embodiment 2] Next, the determination device 1 according to Embodiment 2 of the present invention will be described with reference to Figures 11 and 12. For the sake of convenience of explanation, components having the same function as those described in Embodiment 1 will be denoted by the same reference numerals, and their descriptions will not be repeated.

[0072] In Embodiment 2, the flow of the process for determining a representative value of the vegetation index by the control unit 10 differs from the process for determining a representative value of the vegetation index in Embodiment 1 (step S6) shown in Figure 3. Figure 11 is a flowchart showing an example of the flow of the process for determining a representative value of the vegetation index by the control unit 10 of the determination device 1 according to Embodiment 2 (step S6A). In the process for determining a representative value of the vegetation index (step S6A) shown in Figure 11, the maximum value within the squares forming the mesh is obtained after the mesh is created, and this is used as the representative value, which is different from the process for determining a representative value of the vegetation index in Embodiment 1.

[0073] In the flowchart shown in Figure 11, first, the operator specifies the starting and ending positions for planting each row of plants in the orthomosaic image generated by the orthomosaic image generation unit 11 via the input unit 40, and creates a mesh consisting of multiple squares representing the spacing between plants (step S18).

[0074] The processing in step S18 may be performed by the preprocessing unit 16 based on the orthomosaic image and vegetation information. Also, the length of one side of the squares forming the mesh does not have to be the same as the spacing between plants, and may differ by about ±20%.

[0075] In step S18, it is preferable to specify the starting and ending planting positions for each row of the orthomosaic image, then expand the pixels of the orthomosaic image using focal statistical processing or the like, and then create a mesh.

[0076] Here, Figure 12 is a diagram illustrating the process of creating a mesh in step S18. As shown in Figure 12, the operator creates an image G3a in which a mesh M consisting of multiple rectangles corresponding to the spacing between plants is created from the orthomosaic image G2a in which pixels have been expanded. Note that mesh M has mesh M1 and mesh M2. Mesh M1 and mesh M2 correspond to the positions of plants in different rows.

[0077] After step S18, the preprocessing unit 16 obtains the maximum value of the vegetation index within each mesh (step S19). Then, the determination unit 17 determines the maximum value of the vegetation index within each mesh as the representative value of the vegetation index in each mesh (step S20).

[0078] The determination device 1 in Embodiment 2 described above can also achieve the same effects as in Embodiment 1. In particular, the determination unit 19 can improve the accuracy of determining missing plants by creating a mesh M consisting of multiple squares corresponding to the spacing between plants and determining missing plants for each square. Furthermore, compared to the determination device 1 in Embodiment 1, it is possible to supplement the function of determining missing plants even when the crops are tilted when photographed by the camera C.

[0079] Furthermore, the aforementioned detection device 1 can determine missing plants in crops, thereby reducing the wasteful application of fertilizers and pesticides and mitigating biodiversity loss. Such effects contribute to achieving, for example, United Nations Sustainable Development Goal (SDG) 15, "Protect, restore and promote sustainable use of terrestrial ecosystems, sustainably manage forests, combat desertification, and halt and reverse land degradation and halt biodiversity loss."

[0080] [Other Embodiments] The determination device 1 in the embodiments 1 and 2 described above is used to determine missing plants in a field, but is not limited to this. For example, the determination device 1 can also determine poor plant growth or disease. In this case, the determination unit 19 should determine that the plants in each plant area are poorly growing or diseased if the representative value of the vegetation index in that plant area is below a predetermined threshold. It is preferable to appropriately change the threshold depending on the purpose of the determination device 1 and the type of crop being determined.

[0081] Furthermore, in the determination device 1 of the embodiments 1 and 2 described above, the threshold setting unit 18 sets the threshold by using curve fitting on the density distribution of representative values ​​of the NDVI. However, it is not limited to this. For example, the threshold setting unit 18 may set the threshold by performing clustering on the density distribution.

[0082] [Examples of implementation using software] The function of the determination device 1 (hereinafter referred to as "the device") is a program that causes the device to function as a computer, and can be realized by a program that causes the computer to function as each control block of the device (particularly each part included in the control unit 10).

[0083] In this case, the device includes a computer having at least one control device (e.g., a processor) and at least one storage device (e.g., memory) as hardware for executing the program. By executing the program using this control device and storage device, the functions described in each of the embodiments are realized.

[0084] The above program may be recorded on one or more computer-readable recording media, not temporary ones. These recording media may or may not be provided by the above device. In the latter case, the program may be supplied to the above device via any wired or wireless transmission medium.

[0085] Furthermore, some or all of the functions of each of the above control blocks can also be realized by logic circuits. For example, an integrated circuit in which logic circuits functioning as each of the above control blocks are formed is also included in the scope of the present invention. In addition, it is also possible to realize the functions of each of the above control blocks by, for example, a quantum computer.

[0086] The present invention is not limited to the embodiments described above, and various modifications are possible within the scope of the claims. Embodiments obtained by appropriately combining the technical means disclosed in different embodiments are also included in the technical scope of the present invention.

[0087] 〔summary〕 A determination device according to embodiment 1 of the present invention is a determination device for determining missing plants of crops in a field, comprising: a first acquisition unit that acquires a vegetation index image representing the vegetation index of the field; a second acquisition unit that acquires vegetation information indicating a vegetation area including a plurality of rows in which the crops are planted, the starting and ending positions of planting the crop plants for each row, and the spacing between the plants for each row; a pre-processing unit that performs processing to obtain the maximum value of the vegetation index in a plant area indicating the location where the plants are planted from the vegetation index image based on the vegetation information; a determination unit that determines representative values ​​for the vegetation index of a plurality of plant areas; and a determination unit that determines whether each plant area is missing plants by comparing each of the representative values ​​determined by the determination unit with a predetermined threshold.

[0088] In the determination device according to embodiment 2 of the present invention, in embodiment 1, the preprocessing unit performs a process in which, as the processing process, focal statistical processing is used to replace the value of the vegetation index within the plant area with the maximum value of the vegetation index within the plant area. The determination unit may determine the maximum value after processing by the preprocessing unit as the representative value of the plant area.

[0089] In the determination device according to embodiment 3 of the present invention, in embodiment 1 or 2, the preprocessing unit may, as the processing step, divide the vegetation index image into a plurality of rectangular sections corresponding to the spacing between plants, such that one rectangular section includes one of the plant regions, and obtain the maximum value of the vegetation index for each rectangular section.

[0090] The determination device according to embodiment 4 of the present invention may further include a missing plant calculation unit that calculates the number of missing plants in the field by counting the number of plant regions determined to be missing plants by the determination unit, in any of embodiments 1 to 3 above.

[0091] The determination device according to embodiment 5 of the present invention may further include a location identification unit that identifies the location of missing plants in the field based on the determination result of the determination unit, as described in embodiment 4 above.

[0092] The determination device according to embodiment 6 of the present invention may further include an estimation unit that estimates the amount of replanting work based on the number of missing plants calculated by the missing plant calculation unit and information regarding the location of the missing plants identified by the location identification unit, as described in embodiment 5 above.

[0093] In the determination device according to embodiment 7 of the present invention, in any of embodiments 1 to 6 above, the determination unit may use focal statistical processing to determine that the maximum value of the vegetation index in the plant area is the representative value of the vegetation index included in the plant area.

[0094] In the determination device according to embodiment 8 of the present invention, in any of embodiments 1 to 7 above, the determination unit may create a plurality of rectangular plots such that one rectangular plot includes the location where one of the plants is planted, and the maximum value within the rectangular plot may be used as the representative value of the vegetation index included in the rectangular plot.

[0095] The determination device according to embodiment 9 of the present invention may further include, in any of embodiments 1 to 8 above, a threshold setting unit which determines the density distribution of the representative value of the vegetation index determined by the determination unit, and uses curve fitting on the density distribution to determine the threshold for dividing the curve obtained by fitting using the mean squared error that minimizes the error between classes, or sets the minimum value between two peaks obtained by fitting as the threshold.

[0096] The determination device according to embodiment 10 of the present invention further comprises, in any of embodiments 1 to 9 above, a third acquisition unit that acquires a plurality of aerial images of the field taken from different positions in the sky using a multispectral camera mounted on an aircraft; an orthomosaic image generation unit that generates an orthomosaic image from the plurality of aerial images acquired by the third acquisition unit; and a vegetation index image generation unit that generates the vegetation index image from the orthomosaic image generated by the orthomosaic image generation unit. The first acquisition unit may acquire the vegetation index image generated by the vegetation index image generation unit.

[0097] In the determination device according to embodiment 11 of the present invention, in any of embodiments 1 to 10, the crop may include at least corn and lettuce, which have a lower ability to replace missing plants than rice.

[0098] A determination method according to embodiment 12 of the present invention is a determination method for determining missing plants of crops in a field using the determination device according to any of embodiments 1 to 11, and includes: a first acquisition step of acquiring the vegetation index image with the first acquisition unit; a second acquisition step of acquiring the vegetation information with the second acquisition unit; a preprocessing step of performing processing on the preprocessing unit to obtain the maximum value of the vegetation index in a plant area indicating the location where the plant was planted, based on the vegetation information, from the vegetation index image; a determination step of determining representative values ​​of the vegetation index in a plurality of plant areas with the determination unit; and a determination step of determining whether each plant area is missing plants by comparing each of the representative values ​​determined in the determination step with a predetermined threshold.

[0099] A program according to aspect 13 of the present invention is a program for causing a computer to function as the determination device according to any of aspects 1 to 11, wherein the computer functions as the first acquisition unit, the second acquisition unit, the preprocessing unit, the determination unit, and the determination unit. [Explanation of Symbols]

[0100] 1 Judgment device 10 Control Unit 11 Orthoimage Generation Unit 12 Vegetation Index Image Generation Unit 13 First acquisition part 14 Second acquisition part 15 Third acquisition part 16 Pre-processing section 17. Decision Section 18. Threshold setting section 19 Judgment section 20 Defective stock calculation department 21 Location identification part 22 Estimation part 23 Output section 30 Storage section 40 Input section 50 Display section 60 flying objects

Claims

1. A determination device for determining missing plants in a field, A first acquisition unit acquires a vegetation index image representing the vegetation index of the aforementioned field, A second acquisition unit acquires vegetation information indicating a vegetation area including multiple rows in which the crops are planted, the starting and ending positions of the crop plants in each row, and the spacing between the plants in each row. A preprocessing unit performs processing to obtain the maximum value of the vegetation index within the plant area indicating the location where the plant is planted, based on the vegetation information, A determination unit that determines representative values ​​for the vegetation index of multiple aforementioned plant areas, A determination unit determines whether each of the aforementioned stock regions is missing by comparing each of the aforementioned representative values ​​determined by the determination unit with a predetermined threshold, A determination device equipped with this device.

2. The preprocessing unit, as the processing step, performs a process in which it replaces the value of the vegetation index within the plant area with the maximum value of the vegetation index within the plant area by using focal statistical processing. The determination device according to claim 1, wherein the determination unit determines the maximum value after processing by the preprocessing unit as the representative value of the stock area.

3. The determination device according to claim 1, wherein the preprocessing unit divides the vegetation index image into a plurality of rectangular sections corresponding to the spacing between plants, such that each rectangular section includes one of the plant regions, and obtains the maximum value of the vegetation index for each of the rectangular sections.

4. The determination device according to claim 1, further comprising a plant missing calculation unit that calculates the number of plant missings in the field by counting the number of plant regions determined to be missing by the determination unit.

5. The determination device according to claim 4, further comprising a location identification unit that identifies the location of missing plants in the field based on the determination result of the determination unit.

6. The determination device according to claim 5, further comprising an estimation unit that estimates the amount of replanting work based on the number of missing plants calculated by the missing plant calculation unit and information regarding the location of the missing plants identified by the location identification unit.

7. The determination device according to claim 1, wherein the determination unit uses focal statistical processing to determine the maximum value of the vegetation index in the plant area as the representative value of the vegetation index included in the plant area.

8. The determination device according to claim 1, wherein the determination unit creates a plurality of rectangular plots such that each rectangular plot includes the location where one of the plants is planted, and the maximum value within the rectangular plot is used as the representative value of the vegetation index included in the rectangular plot.

9. The determination device according to claim 1, further comprising: a determination unit that obtains the density distribution of the representative value of the vegetation index determined by the determination unit; a threshold setting unit that uses curve fitting on the density distribution to determine the threshold for dividing the curve obtained by fitting using the mean squared error that minimizes the error between classes; or a threshold setting unit that sets the minimum value between two peaks obtained by fitting as the threshold.

10. A third acquisition unit, which uses a multispectral camera mounted on the aircraft to acquire multiple aerial images of the field taken from different positions in the sky, An orthophoto generation unit generates an orthophoto from a plurality of aerial images acquired by the third acquisition unit, A vegetation index image generation unit generates the vegetation index image from the orthoimage generated by the orthoimage generation unit, Furthermore, The determination device according to claim 1, wherein the first acquisition unit acquires the vegetation index image generated by the vegetation index image generation unit.

11. The determination device according to any one of claims 1 to 10, wherein the crops include at least corn and lettuce, which have a lower ability to replace missing plants than rice.

12. A determination method for determining the absence of crops in the field using the determination device described in claim 1, A first acquisition step involves acquiring the vegetation index image using the first acquisition unit, A second acquisition step involves acquiring the vegetation information using the second acquisition unit, The preprocessing unit performs a processing operation based on the vegetation information to obtain the maximum value of the vegetation index within the plant area indicating the location where the plant is planted from the vegetation index image, The determination unit performs a determination step of determining representative values ​​for the vegetation index in each of the multiple plant regions, The determination unit determines whether each of the strain regions is missing by comparing each of the representative values ​​determined in the determination step with a predetermined threshold, and A determination method that includes this.

13. A program for causing a computer to function as a determination device according to claim 1, wherein the computer functions as the first acquisition unit, the second acquisition unit, the preprocessing unit, the determination unit, and the determination unit.

Images