Information Processing Apparatus, Information Processing Method, Program, and Recording Medium

Abstract

To provide one of the techniques for selecting a strain with a specific trait in a predetermined plant from a photographed image of a field.SOLUTION: An information processing device (10) according to one aspect of the present invention comprises: a stock area identification unit (123) that identifies the area of each stock of a predetermined plant using a photographed image of the field in which the predetermined plant is planted; a circle area identification unit (124) that identifies the area of the minimum circumscribed circle that circumscribes a predetermined plant stock for each stock using the image; and a determination unit (125) that determines whether the stock has a specific trait using the ratio of the area of the stock and the area of the minimum circumscribed circle.SELECTED DRAWING: Figure 1

Description

Technical Field

[0001] The present invention relates to an information processing apparatus, an information processing method, a program, and a recording medium.

Background Art

[0002] Techniques for determining the growth or growth abnormalities of plants including agricultural crops by image analysis are known. For example, Non-Patent Document 1 describes a technique for extracting a node region and estimating an internode distance using an image from the side of a tomato stem. In recent years, techniques for determining the growth status of plants in a field by analyzing images taken by drone aerial photography have also been developed (for example, Non-Patent Document 2).

Prior Art Documents

Non-Patent Documents

[0003]

Non-Patent Document 1

Non-Patent Document 2

Summary of the Invention

Problems to be Solved by the Invention

[0004] Techniques for determining the growth status by image analysis have many problems yet to be solved for practical use because the determination methods and criteria differ depending on the type of plant or the object to be determined even for the same species of plant.

[0005] One aspect of the present invention aims to realize a technique for selecting strains having a specific trait in a predetermined plant from a photographed image of a field.

Means for Solving the Problems

[0006] To solve the above problems, an information processing apparatus according to one aspect of the present invention includes a strain area specifying unit that specifies the area of a strain of a predetermined plant for each strain using an image obtained by photographing a field in which the predetermined plant is planted, a circle area specifying unit that specifies the area of the minimum circumscribed circle circumscribing the strain of the predetermined plant for each strain using the image, and a determination unit that determines whether the strain is a strain having a specific trait using the ratio between the area of the strain and the area of the minimum circumscribed circle.

[0007] To solve the above problems, an information processing method according to one aspect of the present invention includes a strain area specifying step of specifying the area of a strain of a predetermined plant for each strain using an image obtained by photographing a field in which the predetermined plant is planted, a circle area specifying step of specifying the area of the minimum circumscribed circle circumscribing the strain of the predetermined plant for each strain using the image, and a determination step of determining whether the strain is a strain having a specific trait using the ratio between the area of the strain and the area of the minimum circumscribed circle.

[0008] The information processing apparatus according to one aspect of the present invention may be realized by a computer. In this case, a program for causing the computer to operate as each unit included in the information processing apparatus to realize the information processing apparatus by the computer, and a computer-readable recording medium recording the program also fall within the scope of the present invention.

Advantages of the Invention

[0009] According to one aspect of the present invention, it is possible to select a strain having a specific trait, such as a trait of short internodes, in a predetermined plant from a photographed image of a field.

Brief Description of the Drawings

[0010]

Figure 1

Figure 2

Figure 3

Figure 4

Figure 5

Figure 6

Figure 7

Embodiments for Carrying Out the Invention

[0011] 〔1. Configuration Example of Information Processing System〕 Hereinafter, an embodiment of the present invention will be described in detail. FIG. 1 is an example of a functional block diagram of an information processing system 1 according to this embodiment. The information processing system 1 is a system for determining whether plants planted in a field have specific traits. In this embodiment, one or more ridges are formed in the field, and a plurality of strains of a predetermined plant are planted in the ridges. The plants planted in the field are, for example, pumpkins or prostrate-growing plants. As prostrate-growing plants other than pumpkins, for example, plants having agricultural crops on the above-ground part such as watermelons and tomatoes (not limited to plants of the Cucurbitaceae family) are preferable, but plants having agricultural crops on the underground part such as sweet potatoes are also included.

[0012] In particular, an example of a system for determining whether a pumpkin plant has short internodes will be shown below. The short internodes of pumpkins are characterized by constricted internodes and few lateral branches during the early to mid-growth stages. Pumpkins with short internodes have the advantage of not requiring pinching, pruning, or tying, and are easy to harvest as the fruits tend to grow close to the base of the plant. That is, labor savings can be achieved from the early growth stage to harvest. Also, it has been reported that pumpkin varieties with short internodes can be cultivated in about half the field width compared to ordinary pumpkin varieties without short internodes, suggesting the potential to expand the planting area.

[0013] The selection of such pumpkins with short internodes needs to be found from pumpkin genetic resources and hybrid progenies growing in the field. Currently, the method adopted is for growers to walk around the field, visually inspect, and measure the vine length of each plant for selection. Therefore, it takes a great deal of time and labor, and the number of individuals that can be evaluated is limited, which is a factor reducing the breeding efficiency.

[0014] In view of this problem, the inventors have constructed a system that can significantly reduce the time and labor required for selection evaluation compared to the past and increase the number of individuals to be evaluated.

[0015] As shown in FIG. 1, the information processing system 1 of this embodiment includes an information processing device 10 and an aircraft 30. The information processing device 10 is a device for identifying (determining) a pumpkin plant with short internodes planted in the field, and is, for example, a personal computer. The information processing device 10 includes a control unit 12, a storage unit 22, an input unit 24, and a display unit 26. The control unit 12 is a control device that overall controls the entire information processing device 10, and also functions as an acquisition unit 121, a contour identification unit 122, a plant area identification unit 123, a circular area identification unit 124, a determination unit 125, and an output unit 126.

[0016] The acquisition unit 121 acquires an aerial image of the field taken by the aircraft 30 via the input unit 24. The input unit 24 and the aircraft 30 may be connected wirelessly or wired.

[0017] The contour specifying unit 122 analyzes the aerial image (hereinafter sometimes referred to as the captured image) acquired by the acquisition unit 121 to recognize (identify) the pumpkin plants in the image. As an example, the pumpkin plants in the image are recognized (identified) by selecting only the green pixels of the pumpkins from the image. When recognizing (identifying) the pumpkin plants, an image obtained by extracting a part of the aerial image can be used, or conversely, an image formed by connecting a plurality of aerial images can also be used.

[0018] Also, the contour specifying unit 122 specifies the contour of the image area of the identified pumpkin plants for each plant.

[0019] The plant area specifying unit 123 specifies the area of the pumpkin plants for each plant using the image. In one example, the plant area specifying unit 123 specifies the area of the plant for each plant based on the contour specified by the contour specifying unit 122.

[0020] The circumcircle area specifying unit 124 specifies the area of the minimum circumcircle circumscribing the pumpkin plants for each plant using the image. Specifically, the circumcircle area specifying unit 124 specifies the minimum circumcircle for the contour specified by the contour specifying unit 122 and specifies the area of the minimum circumcircle. The specification of the minimum circumcircle for the contour can be performed by adopting a well-known method.

[0021] The determination unit 125 determines whether the pumpkin plants have short internodes (whether they are plants with specific traits) using the ratio of the area of the pumpkin plants specified by the plant area specifying unit 123 to the area of the minimum circumcircle specified by the circumcircle area specifying unit 124.

[0022] Here, the ratio of the area of the pumpkin plant to the area of the minimum circumscribed circle is larger for pumpkin plants with short internodes than for so-called ordinary pumpkin varieties without short internodes. That is, in the case of plants with short internodes, since they do not spread widely on the ground and have a small and compact form, the minimum circumscribed circle is small and the plants are concentrated within that minimum circumscribed circle. On the other hand, for ordinary pumpkin varieties, the vines spread on the ground and extend away from the base of the plant. Therefore, in ordinary varieties, the area of the minimum circumscribed circle tends to be large, and the plants are not concentrated within the minimum circumscribed circle. From these facts, whether a pumpkin plant has short internodes can be specified (judged) by using the ratio of the area of the pumpkin plant to the area of the minimum circumscribed circle.

[0023] In short, the determination unit 125 has a threshold value, and by comparing this threshold value with the ratio of the area of the pumpkin plant to the area of the minimum circumscribed circle, it determines whether the pumpkin plant has short internodes. Note that the threshold value can be set and changed by the user.

[0024] The output unit 126 outputs information regarding the determination result of the determination unit 125. As an example, the information regarding the determination result is an image representing the location of the pumpkin plants with short internodes within the image. As an example, the output unit 126 outputs this information by performing a process of causing the display unit 26 to display an image representing the location of the pumpkin plants with short internodes.

[0025] The storage unit 22 is a recording device that stores various information, and stores, for example, the aerial image acquired by the acquisition unit 121, or information referred to when the determination unit 125 identifies pumpkin plants with short internodes.

[0026] The input unit 24 is an interface for performing operations or inputting information to the information processing device 10. For example, the input unit 24 supplies the acquisition unit 121 with the captured image of the flying object 30 input to itself. Also, a part of the input unit 24 can be realized as a device such as a keyboard or a mouse that receives operations to the information processing device 10.

[0027] The display unit 26 is a display panel that displays text, moving images, etc. based on the control of the control unit 12. Note that the display unit 26 may be configured to realize some functions of the input unit 24 as a touch panel.

[0028] The flying object 30 is an unmanned flying object 30 realized as a drone, UAV (Unmanned Aerial Vehicle), etc. The flying object 30 is equipped with a camera (not shown) and takes an aerial image of the field. Hereinafter, although the flying object 30 will be described as having a configuration for taking RGB images, it is not necessarily limited to the above configuration. Also, the flying object 30 does not need to fly autonomously along a pre-defined route and may be operated by the user in real time.

[0029] [2. Processing Example of Information Processing System] FIG. 2 is an example of a flowchart showing the flow of the information processing method according to the present embodiment. In step S101, the flying object 30 takes a stereoscopic photograph or a plurality of photographs that are aerial images of the field. In this processing example, a plurality of ridges are formed in the field to be photographed, and a plurality of pumpkin plants are planted in each ridge. An interval is provided between the ridges so that the pumpkin plants do not overlap. As the pumpkin plants grow, their vines grow and their leaves cover part of the ridge.

[0030] In step S102, the acquisition unit 121 acquires the aerial image taken by the flying object 30 via the input unit 24. When the acquisition unit 121 acquires a plurality of aerial images of a part of the field, an image formed by combining those plurality of aerial images may be generated. When combining, an image of the entire field (ortho image) may be formed. Note that the imaging range of each aerial image only needs to be such that the determination unit 125 can make a determination from the stock image within the imaging range, and there is no particular limitation. That is, for the aerial image, it is sufficient if at least one pumpkin plant is photographed. However, in order for the determination unit 125 to efficiently find pumpkin stocks having short internodes from the field or ridges, it is preferable that a plurality of stocks are shown in one aerial image.

[0031] FIG. 3 is a diagram illustrating an aerial image. Note that the image PIC in FIG. 3 may be an image obtained by stitching together a plurality of aerial images captured by the flying object 30 (FIG. 1). In the image PIC of FIG. 3, a single ridge 300 and a plurality of pumpkin plants 400 growing side by side along the longitudinal direction of the ridge are shown. Note that the image PIC of FIG. 3 can be displayed on the display unit 26 (FIG. 1). Note that the aerial image is actually a color image.

[0032] In steps S103 to S109, pumpkin plants having short internodes are selected from the pumpkin plants 400 in the image PIC. Specifically, in step S103, the image acquired in step S102 is analyzed, and the contour specifying unit 122 (FIG. 1) recognizes (identifies) the pumpkin plants in the image. As an example, the pumpkin plants in the image are recognized (identified) by selecting only the green pixels of the pumpkins from the image. Further, in step S103, the contour specifying unit 122 specifies the contour of the image area of each identified pumpkin plant.

[0033] Here, FIG. 4 is an image PIC1 in which the pumpkin plants in the aerial image PIC shown in FIG. 3 are recognized. The image PIC1 of FIG. 4 is a black-and-white image as an example, and the white area in the image PIC1 of FIG. 4 is the pumpkin plant. The image PIC1 of FIG. 4 can also be displayed on the display unit 26 in the same manner as the aerial image PIC of FIG. 3.

[0034] Note that the image PIC used when recognizing (identifying) the pumpkin plants and the image PIC2 used when specifying the contour of the image area of each identified pumpkin plant may be the same image data or may be copied image data. Further, the contour may be specified using only a part of the image of the image PIC used when recognizing (identifying) the pumpkin plants. As an example, the part of the image is an image obtained by extracting (cutting out) the area identified as the pumpkin plant and its peripheral area.

[0035] In step S104 (the step of identifying the plant area), the plant area identification unit 123 (Fig. 1) identifies the area of each pumpkin plant using an image. Specifically, in step S104, the plant area identification unit 123 identifies the area of each plant based on the contour identified by the contour identification unit 122. Note that the method for identifying the plant area is not limited to the method of calculating based on the contour. For example, an area where the pixel value representing the green shade of each pixel in the image is within a predetermined range is identified as the plant area. As an example, the plant area identification unit 123 can convert the entire image in the RGB color system to the L*a*b color system, identify the area of pixels where the value of channel a* is less than or equal to a predetermined value (for example, 120) as the plant area, and use the number of counted pixels in each area of the plant area as the (plant) area.

[0036] In step S105 (the step of identifying the circular area), the circular area identification unit 124 (Fig. 1) identifies the area of the minimum circumscribed circle circumscribing each pumpkin plant using an image. Specifically, in step S105, the minimum circumscribed circle for the contour identified in step S104 is identified, and the area of the minimum circumscribed circle is identified.

[0037] Note that in this exemplary embodiment, the mode where step S104 for identifying the plant area is performed prior to step S105 for identifying the area of the minimum circumscribed circle is described. However, the order of these two steps does not matter. Also, as described above, when identifying the plant area without using the contour in step S104, the order of step S104 for identifying the plant area and step S103 for identifying the contour also does not matter.

[0038] In step S106 (the determination step), the determination unit 125 (Fig. 1) uses the ratio of the plant area identified in step S104 and the area of the minimum circumscribed circle identified in step S105 to identify (determine) whether it is a pumpkin plant with short internodes.

[0039] Figures 5 and 6 show an image PIC2 in which, for a single pumpkin plant in an image, an image 500P showing the contour of the plant is combined with an image 500C showing the minimum circumscribed circle circumscribing the contour (image 500P). Figure 5 shows a common variety of pumpkin plant that does not have short internodes, and Figure 6 shows a pumpkin plant that has short internodes. The images PIC2 in Figures 5 and 6 are actually color images. In each of Figures 5 and 6, together with the image PIC2, as an example, the area of the plant, the area of the minimum circumscribed circle, and the ratio of these (area ratio) are shown. The area ratio in Figure 6 is larger than the area ratio in Figure 5. That is, it can be said that the plant in Figure 6 has a shorter vine length compared to the plant in Figure 5 and is a plant with so-called short internodes. This is obvious from looking at the images in Figures 5 and 6, and it can be seen that the plant in Figure 6 has a shorter vine length compared to the plant in Figure 5 and is a plant with so-called short internodes.

[0040] Thus, in step S106, the area ratio of the area of the plant to the area of the minimum circumscribed circle is calculated for each plant. If the calculated area ratio is greater than the threshold value, the process proceeds to step S107. On the other hand, if the calculated area ratio is less than or equal to the threshold value, the process proceeds to step S108.

[0041] In step S107, the determination unit 125 generates an output signal indicating that the plant to be determined has short internodes. Also, in step S108, the determination unit 125 generates an output signal indicating that the plant to be determined does not have short internodes.

[0042] In step S109, the output unit 126 (Figure 1) acquires the output signals generated in steps S107 and S108, and the output unit 126 outputs information corresponding to the output signals. As an example, the output unit 126 generates an image representing the location of the plant with short internodes and displays the generated image on the display unit 26.

[0043] FIG. 7 shows an example of the image displayed on the display unit 26. FIG. 7 shows two images. The upper image and the lower image show the difference in the determination results when the threshold value is changed using the same aerial image. Each image in FIG. 7 is actually a color image.

[0044] In the upper image of FIG. 7, with the threshold value set at 0.69 (69%), stocks with a ratio (area ratio) of 0.69 (69%) or more are displayed as stocks having short internodes, and stocks with a ratio less than 0.69 (69%) are displayed as stocks not having short internodes.

[0045] In the upper image of FIG. 7, with the threshold value set at 0.60 (60%), stocks with a ratio (area ratio) of 0.60 (60%) or more are displayed as stocks having short internodes, and stocks with a ratio less than 0.60 (60%) are displayed as stocks not having short internodes.

[0046] In each image of FIG. 7, for stocks having short internodes, the minimum circumscribed circle of the stock is displayed in white, and for stocks not having short internodes, the minimum circumscribed circle of the stock is displayed in black. As a result, the user can intuitively (visually) find stocks having short internodes. Note that as long as it is possible to distinguish whether a stock has short internodes or not, the display method is not limited to distinguishing by the color of the minimum circumscribed circle. As another example, there may be a mode in which only stocks having short internodes are displayed with a minimum circumscribed circle or an image of another shape. Also, when each stock is assigned an identification number, it may be a mode in which the identification number of stocks having short internodes is displayed.

[0047] When changing the threshold value as shown in FIG. 7, the threshold value may be easily changed by sliding the slider bar 500S displayed on the display unit 26. In this case, it is sufficient that the determination result is reflected on the display unit 26 almost simultaneously when the threshold value is changed. As a result, the user can also consider the threshold value while viewing the determination result displayed on the display unit 26.

[0048] (Effect of this embodiment) According to the present embodiment, it is possible to efficiently select a pumpkin strain having short internodes from a photographed image of a field.

[0049] 〔Supplementary matters〕 〔Supplementary matter 1〕 In the above description, a single ridge 300 shown in the image PIC of FIG. 3 is the imaging target, but it is not limited thereto, and a plurality of ridges can also be the imaging targets. Thereby, a wide area can be determined by the determination unit 125 at a time, and strains having short internodes can be efficiently searched. Further, it is not limited to a field in which ridges are formed. Further, it does not have to be a mode in which a plurality of strains are cultivated side by side, but any field in which the strains are planted at a predetermined interval so as not to overlap each other may be used.

[0050] 〔Supplementary matter 2〕 The functions of the information processing apparatus 10 according to the above-described embodiment may be realized by a single apparatus or may be realized by a system in which a plurality of apparatuses cooperate. For example, the information processing apparatus 10 may be realized by a first apparatus that implements the acquisition unit 121, a second apparatus that implements the contour identification unit 122, a third apparatus that implements the strain area identification unit 123, a fourth apparatus that implements the circular area identification unit 124, and a fifth apparatus that implements the determination unit 125 and the output unit 126.

[0051] 〔Supplementary matter 3〕 In the above-described embodiment, the acquisition unit 121 acquires the aerial image photographed by the flying object 30. However, it is not limited thereto, and as an example, a mode in which the acquisition unit 121 acquires a satellite image obtained by imaging the observation data of a sensor mounted on an artificial satellite may be used.

[0052] 〔Supplementary matter 4〕 In the above-described embodiment, the method is to select pumpkin plants with short internodes based on the ratio of the plant area specified based on the contour of the plant to the area of the minimum circumscribed rectangle. However, it is not limited to pumpkin plants with short internodes, and it is also possible to select individuals with specific traits based on the ratio of the plant area specified based on the contour of the plant to the area of the minimum circumscribed rectangle. For example, it is possible to compare whether the progress of the growth stage is fast or slow, and for example, it is possible to select plants with an early growth stage. As an example, based on the ratio of the plant area to the area of the minimum circumscribed rectangle, even for the same pumpkin variety, it is possible to distinguish between individuals that have started to grow vines and those that have not yet started to grow. In research focusing on cultivation techniques, the ratio of the plant area to the area of the minimum circumscribed rectangle can be adopted as a method for comparing individuals of the same variety. When observing the growth differences due to cultivation techniques, the target individuals may be individuals cultivated in different cultivation environments, and may not be individuals growing side by side in the same ridge as in the above-described embodiment.

[0053] 〔Supplementary Note 5〕 In the above-described embodiment, the target is a prostrate-growing plant (pumpkin), but it may also be a plant other than a prostrate-growing plant. For example, it may be a plant of the Poaceae family such as rice or wheat. When increasing tillers, plants of the Poaceae family such as rice and wheat have different angles (tiller opening angles) formed by the main stem and tillers depending on the genetic resources. When quantifying the appearance (plant type, plant form) of each genetic resource, by calculating the minimum circumscribed circle area and the ratio of both together with the plant area (coverage area), it can be shown that even genetic resources with the same plant area (coverage area) or the same minimum circumscribed circle area have different land coverage rates.

[0054] 〔Example of Realization by Software〕 The control blocks of the information processing system 1 (particularly the acquisition unit 121, the contour specification unit 122, the plant area specification unit 123, the circle area specification unit 124, the determination unit 125, and the output unit 126) may be realized by a logic circuit (hardware) formed in an integrated circuit (IC chip) or the like, or may be realized by software.

[0055] In the latter case, the information processing system 1 includes a computer that executes instructions of a program, which is software for realizing each function. This computer includes, for example, one or more processors and a computer-readable recording medium that records the above program. Then, in the above computer, when the above processor reads and executes the above program from the above recording medium, the object of the present invention is achieved. As the above processor, for example, a CPU (Central Processing Unit) can be used. As the above recording medium, in addition to "non-transitory tangible media" such as ROM (Read Only Memory), a tape, a disk, a card, a semiconductor memory, a programmable logic circuit, etc. can be used. Further, it may further include a RAM (Random Access Memory) for expanding the above program. Further, the above program may be supplied to the above computer via any transmission medium (such as a communication network or a broadcast wave) capable of transmitting the program. Note that one aspect of the present invention can also be realized in the form of a data signal embedded in a carrier wave, in which the above program is embodied by electronic transmission.

[0056] The present invention is not limited to the above-described embodiments, and various modifications are possible within the scope shown in the claims.

Explanation of Reference Numerals

[0057] 1 Information processing system 10 Information processing device 12 Control unit 22 Storage unit 24 Input unit 26 Display unit 30 Flying object 121 Acquisition unit 122 Contour identification unit 123 Stock area identification unit 124 Circular area identification unit 125 Judgment unit 126 Output unit 400 Pumpkin stock (stock of a predetermined plant)

Claims

1. A plant area specifying unit that specifies the area of each plant of a predetermined plant using an image of a field in which the predetermined plant is planted, A circle area specifying unit that specifies the area of the minimum circumscribed circle circumscribing each plant of a predetermined plant using the image, A determination unit that determines whether a plant has a specific trait using the ratio of the area of the plant and the area of the minimum circumscribed circle, An information processing apparatus comprising: an information processing apparatus.

2. further comprising a contour specifying unit that specifies the contour of each plant of a predetermined plant using the image, wherein the plant area specifying unit specifies the area of the plant based on the contour of the plant, and the circle area specifying unit specifies the minimum circumscribed circle with respect to the contour of the plant and specifies the area of the minimum circumscribed circle, The information processing apparatus according to claim 1.

3. wherein the predetermined plant is a creeping plant, The information processing apparatus according to claim 1 or 2.

4. wherein the predetermined plant is a pumpkin, The information processing apparatus according to any one of claims 1 to 3.

5. wherein the determination unit determines whether the plant has short internodes, The information processing apparatus according to claim 4.

6. wherein the determination unit compares the ratio with a threshold value to determine whether the plant has a specific trait, and the threshold value is variable, The information processing apparatus according to claim 4.

7. A plant area specifying step of specifying the area of each plant of a predetermined plant using an image of a field in which the predetermined plant is planted, A circle area specifying step of specifying the area of the minimum circumscribed circle circumscribing each plant of a predetermined plant using the image, A determination step of determining whether a plant has a specific trait using the ratio of the area of the plant and the area of the minimum circumscribed circle, An information processing method comprising: an information processing method.

8. A program for causing a computer to function as the information processing apparatus according to claim 1, the program for causing a computer to function as the plant area specifying unit, the circle area specifying unit, and the determination unit.

9. A computer-readable recording medium recording the program according to claim 8.

Images