Information Processing Apparatus, Information Processing Method, Program, and Recording Medium

Abstract

To realize a technology that can identify an area with poor plant growth even when it is difficult to distinguish between individual strains in an image of a field.SOLUTION: An information processing device (10) analyzes an image of a field in which a plurality of strains of a predetermined plant are planted in a ridge, and comprises an identification part (18) that identifies, in a first mode, a poor growth area, which is an area representing a poorly growing plant in the image, according to the area of each of the plurality of strains in the image, and in a second mode, a poor growth area according to the width of the plant area in the width direction of the ridge in the image.SELECTED DRAWING: Figure 1

Description

Technical Field

[0001] The present invention relates to an information processing apparatus, an information processing method, a program, and a recording medium.

Background Art

[0002] Techniques for determining the growth status of plants in a field are known. Non-Patent Document 1 describes that by performing drone aerial photography and image analysis, measuring the projected leaf area of cabbages planted in a field for each plant, identifying growth-delayed plants from the projected leaf area, and creating a field map, the quality of growth can be visualized.

Prior Art Documents

Non-Patent Documents

[0003]

Non-Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] However, in the conventional technology as described above, although it is possible to determine the quality of growth for a field where the plants do not overlap, there is a problem that it is difficult to identify the growth-deficient areas when the plants overlap. For example, some vine plants grow and their vines extend and leaves cover the ridges. In the case of such plants, with the technology described in Non-Patent Document 1 above, although it is possible to identify the growth-deficient areas because the plants do not overlap until a certain period, when growth occurs and it becomes difficult to distinguish between individual plants, there is a problem that the growth-deficient areas cannot be identified.

[0005] One aspect of the present invention has been made in view of the above problems, and an object thereof is to realize a technology capable of identifying growth-deficient areas of plants even when it is difficult to distinguish between individual plants in an image of a field taken.

Means for Solving the Problem

[0006] To solve the above problems, an information processing apparatus according to an aspect of the present invention analyzes an image obtained by photographing a field in which a plurality of plants of a predetermined plant are planted in a row, and in a first mode, according to the area of each of the plurality of plants in the image, identifies a growth-deficient portion that is a region representing the growth-deficient plant in the image, and in a second mode, identifies the growth-deficient portion according to the width of the plant region in the width direction of the row in the image, and includes a specifying unit.

[0007] To solve the above problems, an information processing method according to an aspect of the present invention analyzes an image obtained by photographing a field in which a plurality of plants of a predetermined plant are planted in a row, and in a first mode, according to the area of each of the plurality of plants in the image, identifies a growth-deficient portion that is a region representing the growth-deficient plant in the image, and in a second mode, according to the width of the plant region in the width direction of the row in the image, includes a step of identifying the growth-deficient portion.

Advantages of the Invention

[0008] According to an aspect of the present invention, even when it is difficult to distinguish each plant in an image obtained by photographing a field, it is possible to identify a growth-deficient portion of the plant.

Brief Description of the Drawings

[0009]

Figure 1

Figure 2

Figure 3

Figure 4

Figure 5

Figure 6

Figure 7

Figure 8

Figure 9

Figure 10

Figure 11

Figure 12

Figure 13

Figure 14

Figure 15

Mode for Carrying Out the Invention

[0010] 〔Embodiment 1〕 〔1. Configuration Example of Information Processing System〕 Hereinafter, an embodiment of the present invention will be described in detail. FIG. 1 is an example of a functional block diagram of the information processing system 1 according to this embodiment. The information processing system 1 is a system for determining the growth status of plants planted in a field. In this embodiment, one or more ridges are formed in the field, and a plurality of strains of a predetermined plant are planted in the ridges. The plants planted in the field are, as an example, pumpkins or vine plants. In particular, the plants planted in the field may be vine plants cultivated by ground cover. Also, in the case of a field planted with pumpkin plants, the pumpkin variety may be a vine variety or a short internode variety. Also, the vine plants are preferably plants other than pumpkins, such as plants that produce agricultural crops above the ground, such as watermelons and melons (not limited to cucurbitaceae plants), but also include plants that produce agricultural crops below the ground.

[0011] The information processing system 1 includes an information processing device 10 and an aircraft 30. The information processing device 10 is a device for determining the growth status of plants planted in a field, and is, for example, a personal computer. The information processing device 10 includes a control unit 12, a storage unit 22, an input unit 24, and a display unit 26. The control unit 12 is a control device that overall controls the entire information processing device 10, and also functions as an acquisition unit 14, a generation unit 16, a specification unit 18, and an output unit 20.

[0012] The acquisition unit 14 acquires, via the input unit 24, an aerial image of the field taken by the aircraft 30. The input unit 24 and the aircraft 30 may be connected wirelessly or may be connected by wire. The generation unit 16 generates an overall image that is an image representing the entire field by referring to the one or more aerial images acquired by the acquisition unit 14.

[0013] The specific part 18 analyzes the overall image and identifies the stunted parts. In the present embodiment, the stunted parts refer to the areas representing stunted plants in the overall image. The areas representing stunted plants include the areas representing stunted strains and the areas representing communities of stunted strains. A community of strains refers to a collection of multiple strains. The specific part 18 identifies the stunted parts in either the first mode or the second mode. The first mode is the mode corresponding to the overall image in which the occurrence frequency of the overlap of strains satisfies a predetermined condition. The second mode is the mode corresponding to the overall image in which the occurrence frequency of the overlap of strains does not satisfy the above condition.

[0014] In the case of a field where pumpkin strains are planted, immediately after transplantation, the strains do not overlap with each other. However, as the strains grow, the vines cover the ridges, and it becomes difficult to distinguish each strain because the strains overlap. In the present embodiment, as an example, the mode is selected according to whether adjacent strains overlap or not. The first mode is, for example, the mode corresponding to an image in which adjacent strains in the ridge do not overlap or the occurrence frequency of the overlap of adjacent strains is low (for example, the ratio of overlapping strains is less than the threshold). In this case, the above predetermined condition is the condition that the ratio of overlapping strains is less than the threshold. On the other hand, the second mode is the mode corresponding to an image in which all or most of the adjacent strains in the ridge overlap.

[0015] In the first mode, the specific part 18 identifies the stunted parts, which are the areas representing stunted strains in the image, according to the area of each of the multiple strains in the image. On the other hand, in the second mode, the specific part 18 identifies the stunted parts according to the width of the plant area in the width direction (short side direction) of the ridge in the image. In the present embodiment, the width of the plant area refers to the width of the area of the strain of the plant and / or the width of the area of the community in the image.

[0016] The output unit 20 outputs information regarding the growth-deficient location identified by the identification unit 18. As an example, the information regarding the growth-deficient location is an image representing the growth-deficient location. As an example, the output unit 20 outputs the above information by performing a process of causing the display unit 26 to display an image representing the growth-deficient location.

[0017] The storage unit 22 is a recording device that stores various types of information, and stores, for example, the aerial image acquired by the acquisition unit 14, or information referred to when the identification unit 18 identifies a growth-deficient location, etc.

[0018] The input unit 24 is an interface for performing operations or input of information to the information processing apparatus 10. For example, the input unit 24 supplies the acquisition unit 14 with a captured image of the aircraft 30 input to itself. Also, a part of the input unit 24 can be realized as a device such as a keyboard or a mouse that accepts operations to the information processing apparatus 10.

[0019] The display unit 26 is a display panel that displays text, moving images, etc. based on the control of the control unit 12. Note that the display unit 26 may be configured to realize a part of the functions of the input unit 24 as a touch panel.

[0020] The aircraft 30 is an unmanned aircraft 30 realized as a drone, UAV (Unmanned Aerial Vehicle), etc. The aircraft 30 is equipped with a camera (not shown) and takes an aerial image of the farmland. Hereinafter, the aircraft 30 will be described as having a configuration that takes an RGB image, but is not necessarily limited to the above configuration. Also, the aircraft 30 does not need to fly autonomously along a pre-specified route and may be one that is operated by the user in real time.

[0021] 〔2. Processing Example of Information Processing System〕 FIG. 2 is an example of a flowchart showing the flow of the information processing method according to the present embodiment. In step S101, the flying object 30 captures a stereoscopic photograph or a plurality of photographs that are aerial images of the farm field. In this processing example, a plurality of ridges are formed in the farm field that is the photographing target, and a plurality of pumpkin plants are planted in each ridge. An interval is provided between the ridges so that the pumpkin plants do not overlap. As the pumpkin plants grow, their vines extend and their leaves cover the ridges, and the plants may overlap each other in the longitudinal direction of the ridges.

[0022] In step S102, the acquisition unit 14 acquires the aerial image captured by the flying object 30 via the input unit 24. In step S103, the generation unit 16 generates an overall image representing the entire farm field based on the aerial image acquired by the acquisition unit 14. When the acquisition unit 14 acquires a plurality of aerial images of a part of the farm field, the generation unit 16 generates an overall image by stitching together those plurality of aerial images. The overall image is, for example, an orthoimage of the entire farm field.

[0023] FIGS. 3 and 4 are diagrams illustrating the overall image. The overall image 51 shown in FIG. 3 and the overall image 52 shown in FIG. 4 are overall images representing the entire farm field generated from the aerial images captured by the flying object 30. The overall image 51 is an overall image of the farm field in which adjacent plants in each ridge do not overlap. On the other hand, the overall image 52 is an overall image of the farm field in which adjacent plants in the longitudinal direction of the ridge overlap.

[0024] In steps S104 to S109, the specifying unit 18 analyzes the overall image and specifies a poorly growing portion in the overall image. First, in step S104, the specifying unit 18 performs mode determination based on a user operation. A user such as a farm field manager uses the input unit 24 to perform an operation of selecting either the first mode or the second mode. As an example, when the pumpkin plants can be distinguished in the image, such as immediately after transplanting the pumpkin plants, the user selects the first mode. On the other hand, when the vines cover the ridges as the pumpkin plants grow and it is difficult to distinguish each plant, the user selects the second mode.

[0025] The specific part 18 acquires information indicating the mode selected by the user via the input part 24. When the acquired information is information indicating the first mode, the specific part 18 proceeds to the process of step S105. On the other hand, when the acquired information is information indicating the second mode, the specific part 18 proceeds to the process of step S107.

[0026] (First mode) In the first mode, the specific part 18 specifies a growth-deficient area, which is an area representing a growth-deficient plant in the image, according to the area of each of a plurality of plants in the overall image. First, in step S105, the specific part 18 analyzes the overall image and calculates the area of each of the plants included in the overall image.

[0027] The specific part 18 calculates the area of the plant by the following method as an example. First, the specific part 18 specifies an area where the pixel value representing the green shade of each pixel in the overall image is included in a predetermined range as the area of the plant. As an example, the specific part 18 converts the overall image in the RGB color system to the L*a*b color system and specifies an area of pixels whose value of channel a* is equal to or less than a predetermined value (for example, 120) as the area of the plant. In addition, the specific part 18 counts the number of pixels in each of the specified plant areas. The specific part 18 uses the counted number of pixels as the area.

[0028] Next, in step S106, the specific part 18 specifies the growth-deficient area according to the area of each of a plurality of plants in the overall image. As an example, the specific part 18 classifies a plurality of plants in the image into a plurality of classes based on the area of each of the plurality of plants, and specifies an area representing the plants in the class that satisfies a predetermined condition among the plurality of classes as the growth-deficient area. The class that satisfies the predetermined condition is, for example, a class lower than the class with the highest appearance frequency among the classes excluding the lowest class and the highest class. In addition, the class that satisfies the predetermined condition may be, for example, a class equal to or lower than a predetermined threshold value.

[0029] The specific part 18 identifies the undergrown areas by the following method as an example. First, based on the calculated area (number of pixels) per strain, the specific part 18 classifies a plurality of strains into a plurality of classes and creates a frequency distribution table representing the frequency of occurrence for each class. Next, the specific part 18 identifies the class with the highest frequency of occurrence (hereinafter referred to as the "most frequently occurring class") from among the plurality of classes other than the lowest class and the highest class in the frequency distribution table. The reason for excluding the lowest class and the highest class from the selection of the most frequently occurring class is that the lowest class and the highest class contain unnecessary data (so-called garbage).

[0030] The specific part 18 identifies the area of the strain classified into a class smaller than the area of the identified most frequently occurring class as the undergrown area. In addition, the specific part 18 determines that the strains classified into a class larger than the area of the most frequently occurring class are in good growth condition.

[0031] The method for identifying the undergrown area is not limited to the method described above. As an example, the specific part 18 may identify the area of the strain whose area of the strain region satisfies a predetermined condition as the undergrown area. In this case, the predetermined condition may be, for example, that the order when the areas of the strain regions are sorted in descending order of area is equal to or less than a predetermined threshold (for example, included in the lower 20%), or the area is smaller than a predetermined threshold (for example, the average value, median value, etc.) of the area.

[0032] (Second mode) Next, the process executed by the information processing apparatus 10 in the second mode will be described with reference to the drawings. In the second mode, the specific part 18 identifies the undergrown area according to the width (the length in the short side direction) of the plant area in the width direction of the ridge. First, in step S107 of FIG. 2, the entire image is analyzed, and for each of a plurality of positions, the width of the plant area in the width direction of the ridge is calculated. The width of the plant area in the width direction of the ridge refers to the length in the width direction of the ridge of the plant area in the entire image.

[0033] The specific part 18 calculates the width of the plant area by the following method as an example. First, the specific part 18 identifies the area where the pixel values of each pixel included in the overall image satisfy a predetermined condition as the plant area. As an example, the specific part 18 converts the overall image in the RGB color system to the L*a*b color system, and identifies the area of the pixels whose value of channel a* is equal to or less than a predetermined value (for example, 120) as the plant area (the area of a plant or a community). Also, as an example, the specific part 18 rotates the overall image and identifies a predetermined direction (for example, the horizontal direction of the overall image) in the overall image as the longitudinal direction of the ridge. The specific part 18 may rotate the overall image according to a user operation, or may rotate the overall image by a predetermined angle. Further, as an example, the specific part 18 may analyze the shape of one or more areas identified as the plant area and identify the longitudinal direction of the area as the longitudinal direction of the ridge.

[0034] For each of a plurality of positions spaced apart at predetermined distances in the longitudinal direction of the ridge, the specific part 18 calculates the width of the plant area (the area of a plant or a community) in the width direction (the short side direction) of the ridge. As an example, the specific part 18 counts the pixel width of the plant area in the width direction of the ridge for each of the plurality of positions. The specific part 18 uses the counted pixel width as the width of the plant area.

[0035] In step S108, the specific part 18 identifies a poorly growing area according to the width of the plant area at a plurality of positions in the overall image. As an example, the specific part 18 classifies the plurality of positions into a plurality of classes based on the width calculated for each position, and identifies the area corresponding to the position of the class that satisfies a predetermined condition among the plurality of classes as the poorly growing area.

[0036] The specific part 18 identifies the growth-deficient locations by the following method as an example. First, based on the width for each calculated position, the specific part 18 classifies a plurality of positions into a plurality of classes and creates a frequency distribution table representing the frequency of occurrence for each class. Next, among the plurality of classes in the frequency distribution table, excluding the lowest and the highest classes, the specific part 18 identifies the modal class with the highest frequency of occurrence. The reason for excluding the lowest and the highest classes from the selection of the modal class is that the lowest and the highest classes contain unnecessary data (so-called garbage).

[0037] The specific part 18 identifies, as growth-deficient locations, the regions corresponding to the positions classified into classes with narrow widths as compared with the width of the identified modal class. Also, the specific part 18 determines that the regions corresponding to the positions classified into classes with widths wider than the width of the modal class are regions with good growth. The region corresponding to a position is, as an example, a region representing a location within the community including that position and its surroundings (within a predetermined range from that position).

[0038] The method for identifying growth-deficient locations is not limited to the method described above. As an example, the specific part 18 may identify, as growth-deficient locations, the regions corresponding to the positions where the calculated width satisfies a predetermined condition. In this case, the predetermined condition may be, as an example, that when a plurality of positions are sorted in descending order of width, the order is below a predetermined threshold (for example, included in the lower 20%), or the width is smaller than a predetermined threshold (for example, the average value, median, etc. of the width).

[0039] In step S109 of FIG. 2, the output unit 20 outputs information representing growth-deficient locations. As an example, the output unit 20 generates an image representing growth-deficient locations and displays the generated image on the display unit 26. At this time, the output unit 20 outputs an image representing, in different display modes, the regions identified by the specific part 18 as the growth-deficient locations and the other regions among the regions representing stocks or communities. Hereinafter, the image representing growth-deficient locations is also referred to as a "diagnostic image".

[0040] Figs. 5 and 6 are diagrams showing an example of diagnostic images. The diagnostic image 61 in Fig. 5 is a diagnostic image generated in the first mode and corresponds to the overall image 51 in Fig. 3. On the other hand, the diagnostic image 62 in Fig. 6 is a diagnostic image generated in the second mode and corresponds to the overall image 52 in Fig. 4. The diagnostic image 61 represents the determination result of the growth quality for each pumpkin plant. The diagnostic image 62 represents the determination result of the growth quality for each predetermined distance in the longitudinal direction of the ridge.

[0041] Specifically, the diagnostic image 61 represents the pumpkin plants determined to have poor growth in black, while representing the pumpkin plants determined to have no problem in growth in gray. That is, in the diagnostic image 61, the area of the plant with a small area is represented in black as a poor growth location, while the area of the plant with a large area is represented in gray as a location with no problem in growth. Also, the diagnostic image 62 represents the locations within the community determined to have poor growth in gray, while representing the locations of the community determined to have no problem in growth in white. That is, in the diagnostic image 62, the locations where the width of the plant or community is small in the width direction of the ridge are represented in gray as poor growth locations, while the locations where the width of the plant or community is large are represented in white as locations with no problem in growth.

[0042] A user such as a field manager can view the screen displayed on the display unit 26 and grasp which area requires topdressing. The user performs topdressing on the locations with poor growth in the field. Specifically, the user performs topdressing on the locations corresponding to the black areas in the diagnostic image 61 or the gray areas in the diagnostic image 62.

[0043] By the way, the appropriate timing for nitrogen application to pumpkins is after fruit set in vining varieties, and the male flower budding stage to the beginning of flowering stage in short internode varieties. By partially top-dressing the poorly growing areas at this time to compensate for nitrogen deficiency, an increase in yield can be expected. If the growth is poor, the yield will drastically decrease without nitrogen top-dressing. On the other hand, if nitrogen supply is excessive, due to too much nitrogen absorption, the vines will grow and the foliage will thrive, but there is a risk of problems such as so-called "vine lodging" where fruits cannot be set. Therefore, when top-dressing, it is necessary to conduct partial top-dressing by identifying the poorly growing areas at an appropriate time instead of overall top-dressing.

[0044] However, the appropriate time for partial top-dressing is when the vines grow onto the path and tractors cannot enter the field. Therefore, conventionally, when top-dressing, the only method was for skilled workers to walk in the field and visually confirm the growth status. For example, in large pumpkin fields in Hokkaido and other places, it is physically difficult to walk around while carrying fertilizers and perform top-dressing manually. Also, without grasping the growth of the entire field, it is impossible to determine whether the growth is delayed. Due to these factors, it was difficult to quantitatively evaluate the growth of pumpkins. Therefore, although top-dressing in the later stage of cultivation was expected to lead to an increase in yield, top-dressing was hardly ever actually carried out.

[0045] In contrast, according to this embodiment, the information processing device 10 determines the growth quality of pumpkins based on the growth stage of pumpkin plants, or the plant area, or the width of the plant or the width of the community, and identifies the poorly growing areas (areas where top-dressing is required). Thus, according to this embodiment, it is possible to identify the poorly growing areas regardless of whether the pumpkin plants overlap or not. The administrator of the pumpkin field or the like can perform top-dressing on the poorly growing areas identified by the information processing device 10. In other words, it is possible to effectively perform top-dressing that leads to an increase in the yield of pumpkins or the like without having skilled workers walk in the field and visually confirm the growth status in order to conduct partial top-dressing.

[0046] In particular, pumpkins are vine plants. In the case of vine plants, since it is necessary to widen the width of the ridge, it is easier to more preferably determine the growth quality of plants in the second mode. On the other hand, when cultivating non-vine plants, the space between plants and between ridges is often narrow. In this case, if the plants planted in different ridges overlap, the second mode may not be able to distinguish the ridges and may not be able to preferably determine the growth quality.

[0047] The lower limit of the cultivation area of the farmland according to the present embodiment is not particularly limited, but the larger the cultivation area, the more preferable. For example, it is 10 a or more, preferably 50 a or more, and more preferably 1 ha or more. The upper limit of the cultivation area of the farmland is not particularly limited, but for example, it is 3 ha or less. Further, the farmland according to the present embodiment preferably has a planting density with a ridge width of about 3 m to 4 m and a plant spacing of about 30 to 90 cm as an example.

[0048] The inventor of the present invention identified the poorly growing parts in the pumpkin field using the information processing method according to the present embodiment, and topdressed the identified parts. Due to this topdressing, a tendency for the yield to increase was recognized along with the tendency for the leaf area of the pumpkin plants in the field to increase.

[0049] 〔Embodiment 2〕 Other embodiments of the present invention will be described below. For convenience of explanation, members having the same functions as the members described in the above embodiment are given the same reference numerals, and the description thereof will not be repeated.

[0050] In the present embodiment, based on the analysis result of the overall image, the specific part 18 identifies the area representing the weeds included in the overall image, and identifies the poorly growing parts in the area other than the area representing the identified weeds. Hereinafter, the area representing the weeds is also referred to as the "weed area".

[0051] FIG. 7 is a flowchart showing the flow of the information processing method according to the present embodiment. The flowchart shown in FIG. 7 includes the process of step S121 after the process of step S103 in the flowchart of FIG. 2. In step S121, the specifying unit 18 analyzes the entire image and specifies a weed area based on the analysis result. When pumpkin plants are planted in the field, since the leaves of the pumpkin are darker green than the weeds, the pumpkin plants and the weeds can be distinguished by the shade of green. Therefore, as an example, the specifying unit 18 specifies, as a weed area, an area where the pixel values of the pixels constituting the entire image satisfy a predetermined condition. The area satisfying the predetermined condition may be, for example, an area of pixels whose pixel values representing the shade of green are included in a predetermined range. As an example, the specifying unit 18 converts the entire image in the RGB color system into the L*a*b color system, and specifies, as an area of plants or communities, an area of pixels whose value of channel a* is equal to or less than a predetermined value (for example, 100).

[0052] Note that the method for specifying the weed area in the entire image is not limited to the method described above. The specifying unit 18 may specify the weed area by another method. For example, the specifying unit 18 may specify the weed area by pattern matching that collates a predetermined pattern in the entire image. Also, for example, the generation unit 16 generates a three-dimensional model of the field based on the aerial image captured by the flying object 30, and the specifying unit 18 determines whether each plant is a weed based on the plant height of the plants included in the three-dimensional model.

[0053] In this case, in step S105, the specifying unit 18 specifies an area of plants from an area other than the weed area specified in step S121, and calculates the area of each plant. Also, in step S107, the specifying unit 18 specifies an area of plants (an area of plants or communities) from an area other than the weed area specified in step S121, and calculates the width of the area of plants in the width direction of the ridge.

[0054] According to the present embodiment, the specifying unit 18 specifies a weed area included in the overall image and specifies a poorly growing area in an area other than the weed area based on the analysis result of the overall image. Thereby, the information processing apparatus 10 can exclude the weed area from the target of determining the growth status of the plants, and can determine the growth status of the strains with higher accuracy.

[0055] In the present embodiment, after a field manager or the like removes weeds growing in the field manually or the like, the aircraft 30 may photograph the field. In this case, since the aerial image and the overall image of the field do not include weeds, the specifying unit 18 may not perform the process of specifying the weed area (the process of step S121 in FIG. 7).

[0056] 〔Embodiment 3〕 Other embodiments of the present invention will be described below. For convenience of explanation, members having the same functions as the members described in the above embodiments are given the same reference numerals, and the description thereof will not be repeated.

[0057] In the present embodiment, a plurality of ridges are formed in the field to be photographed, and a plurality of watermelon strains are planted in each ridge. An interval is provided between the ridges so that the watermelon strains do not overlap. As the watermelon strains grow, the vines grow and the leaves cover the ridges, and the strains may overlap each other in the longitudinal direction of the ridges.

[0058] FIG. 8 is an example of a flowchart showing the flow of the information processing method according to the present embodiment. The flowchart shown in FIG. 8 includes step S131 instead of step S107 as a process to be executed when the mode determination result is the "second mode".

[0059] (First mode) When the determination result in step S104 of FIG. 8 is in the first mode, the specific part 18 calculates the area of each stock in the overall image in step S105 in the same manner as in the first and second embodiments. As an example, the specific part 18 converts the overall image in the RGB color system into the L*a*b color system, and specifies the area of pixels whose value of channel a* is equal to or less than a predetermined value (for example, 120) as the area of the plant. As a result, the overall image is classified into the area of the plant and other areas (hereinafter, also referred to as "other areas"). The specific part 18 sets the area of the plant surrounded by the other areas as the area of the stock, and counts the number of pixels in each of the areas of the plurality of stocks. The specific part 18 uses the counted number of pixels as the area.

[0060] Also, in step S106, the specific part 18 specifies the growth-deficient parts according to the area of each of the plurality of stocks in the overall image in the same manner as in the first and second embodiments. As an example, the specific part 18 classifies the plurality of stocks in the image into a plurality of classes based on the area of each of the plurality of stocks, and specifies the area representing the stocks of the class that satisfies a predetermined condition among the plurality of classes as the growth-deficient part. The class that satisfies the predetermined condition is, for example, a class lower than the class with the highest appearance frequency among the classes excluding the lowest and highest classes. Also, the class that satisfies the predetermined condition may be, for example, a class equal to or lower than a predetermined threshold value.

[0061] FIG. 9 is a diagram showing an example of the overall image in the first mode. In FIG. 9, the overall image 53 is an overall image representing the entire field generated from the aerial image taken by the flying object 30, and is an overall image of the field in which the adjacent stocks in each ridge do not overlap.

[0062] FIG. 10 is a diagram showing an example of a diagnostic image corresponding to the overall image 53 in FIG. 9. The diagnostic image 63 in FIG. 10 is a diagnostic image generated by the information processing apparatus 10 in the first mode. The diagnostic image 63 represents the determination result of the growth quality for each watermelon plant. Specifically, the diagnostic image 63 represents the watermelon plants determined to have poor growth in black, while representing the pumpkin plants determined to have no problem in growth in gray. That is, in the diagnostic image 63, the area of the plant with a small area is represented in black as a poor growth area, while the area of the plant with a large area is represented in gray as an area with no problem in growth.

[0063] (Second mode) FIG. 11 is a diagram showing an example of the overall image in the second mode. In FIG. 11, the overall image 54 is an overall image representing the entire field generated from the aerial image taken by the flying object 30, and is an overall image of the field in which adjacent plants overlap in the longitudinal direction of the ridges.

[0064] In the present embodiment, in the second mode, the specific part 18 executes the process of step S131 in FIG. 8. In step S131, the specific part 18 calculates the width (width in the width direction of the ridge) of the area inside the contour of the plant area (hereinafter also referred to as "inner contour area") in the overall image. The inner contour area includes the plant area and other areas surrounded by the plant area.

[0065] First, as an example, the specific part 18 performs a process of specifying the area within the contour by the following method. First, the specific part 18 specifies, as the area of the plant, the area where the pixel values of each pixel included in the entire image satisfy a predetermined condition. As an example, the specific part 18 converts the entire image in the RGB color system to the L*a*b color system, and specifies, as the area of the plant, the area of the pixels whose value of the a* channel is equal to or less than a predetermined value (for example, 120). Further, the specific part 18 acquires the coordinates of the contour of the specified plant area (the area of the community). Based on the coordinates of the contour, the area within the contour is specified. Further, the specific part 18 writes out the contour of the community as an image separate from the entire image using the acquired coordinates, and executes a process of filling in the area within the contour. The specific part 18 calculates the width of the filled area, that is, the width in the width direction of the ridge, of the area within the contour.

[0066] In step S108, the specific part 18 specifies the growth-deficient part according to the width of the area within the contour calculated in step S131. The method of specifying the growth-deficient part in step S108 is the same as the method described in the above-described embodiment 1.

[0067] In other words, in the second mode, the specific part 18 specifies the growth-deficient part according to the width (the width in the width direction of the ridge) of the area within the contour including the area of the plant and the other areas surrounded by the area of the plant in the entire image (steps S131, S108).

[0068] Note that the process of specifying the area within the contour in step S131 is not limited to the above-described example. For example, the specific part 18 may perform a process of converting the pixel values of the other areas surrounded by the area of the plant in the entire image to a predetermined value (for example, 100). In this case, in the entire image, the area where the pixel value is equal to or less than a predetermined value (for example, 120) becomes the area within the contour.

[0069] FIG. 12 is a diagram showing an example of the area of the plant specified for the entire image 54 of FIG. 11. In the image 71 of FIG. 12, the white area is the area of the plant, and the black area is the area other than the area of the plant.

[0070] The leaves of the watermelon have shallow and coarse serrations. When such leaves overlap, in Image 71, there are many regions outside the plant region surrounded by the plant region. When such a large number of regions are included, the specific part 18 may not be able to appropriately calculate the width of the plant. One of the reasons is that, for example, the boundary between the plant region and another region surrounded by the plant region may be misjudged as the end of the plant region.

[0071] FIG. 13 is a diagram showing an example of the region inside the contour specified by the specific part 18 for the entire image 54 of FIG. 11. In the image 72 of FIG. 13, the white region is the region inside the contour, and the black region is the region outside the contour. By calculating the width of the region inside the contour for the image 72 of FIG. 13, the specific part 18 can more appropriately calculate the width of the plant region.

[0072] FIG. 14 is a diagram showing an example of a diagnostic image. The diagnostic image 64 of FIG. 14 is a diagnostic image generated in the second mode and is an image corresponding to the image 72 of FIG. 10. The diagnostic image 64 represents the determination result of the growth quality for each processing distance in the longitudinal direction of the ridge. More specifically, the diagnostic image 64 represents the locations within the community determined to have poor growth in gray, while representing the locations of the community determined to have no problem in growth in white. That is, in the diagnostic image 64, the locations where the width of the plant or community is small in the width direction of the ridge are represented in gray as poor growth locations, while the locations where the width of the plant or community is large are represented in white as locations with no growth problems.

[0073] A user such as a farm manager can visually recognize the screen displayed on the display unit 26 and grasp which areas need topdressing. The user performs topdressing on the locations with poor growth in the farm. Specifically, the user performs topdressing on the locations corresponding to the black regions of the diagnostic image 63 or the gray regions of the diagnostic image 64.

[0074] The inventor used the information processing method according to this embodiment to photograph a watermelon field with a drone and identify growth-deficient areas. As shown in FIG. 15, it was confirmed that the individuals determined to have poor growth in the information processing method according to this embodiment had shorter vine lengths measured in the field compared to the individuals determined to have good growth.

[0075] (Example) FIG. 15 is a graph showing the relationship between the diagnosis results of watermelons according to this embodiment and the measured values of vine lengths. In FIG. 15, graph 91 shows the average value of the vine lengths of watermelons diagnosed as having good growth in this embodiment (the watermelons in regions 911 to 918 in FIG. 14), and graph 92 shows the average value of the vine lengths of watermelons diagnosed as having poor growth (regions 921 to 928 in FIG. 14). As shown in FIG. 15, the average value of the vine lengths of watermelons diagnosed as having good growth in this embodiment is longer than the average value of the vine lengths of watermelons diagnosed as having poor growth. That is, from FIG. 15, it can be confirmed that the diagnosis of poor growth according to this embodiment is appropriately performed. Therefore, by performing a determination of poor growth using the information processing method according to this embodiment, it is possible to identify growth-deficient areas and perform partial top-dressing at an appropriate time. Thus, the information processing method according to this embodiment can also be used for the growth diagnosis of vining crops other than pumpkins such as watermelons.

[0076] (Effect of this embodiment) As described above, according to this embodiment, the information processing device 10 determines the quality of growth of watermelon plants based on the growth stage of the watermelon plants, or based on the plant area, or the width of the plant or the width of the community, and identifies growth-deficient areas (areas where top-dressing is required). Thus, according to this embodiment, it is possible to identify growth-deficient areas regardless of whether the watermelon plants overlap or not. The manager of the watermelon field or the like can perform top-dressing on the growth-deficient areas identified by the information processing device 10. In other words, without having a skilled person walk through the field to visually check the growth situation in order to perform partial top-dressing, it is possible to effectively perform top-dressing that leads to an increase in the yield of watermelons or the like.

[0077] In addition, as described above, since the watermelon leaves have shallow and coarse serrations, when the watermelon leaves overlap, there are many regions surrounded by the plant regions in the entire image of the watermelon field. In the present embodiment, in the second mode, the specific part 18 includes these regions in the plant region and executes the calculation process of the width of the plant. By performing the process of filling in the community (including other regions in the community in the plant region) in the second mode in this way, the calculation of the width of the plant can be performed more appropriately in the second mode.

[0078] In addition, in the present embodiment, the specific part 18 does not perform the filling process within the community in the first mode. Therefore, the accuracy of specifying the growth-deficient part in the second mode can be increased without increasing the processing load related to specifying the growth-deficient part in the first mode.

[0079] 〔Supplementary matters〕 〔Supplementary matter 1〕 In the above-described Embodiments 1 and 2, the operation when the information processing system 1 determines the growth status of the pumpkin plants was described. Also, in the above-described Embodiment 3, the operation when determining the growth status of the watermelon plants was described. The plant for which the growth status is to be determined is not limited to pumpkins or watermelons, and may be other plants. As an example, the plant for which the growth status is to be determined is a plant in which the plants overlap each other due to growth. In the above-described Embodiments 1 to 3, as an example, the plant for which the growth status is to be determined may be a vine plant other than pumpkins (for example, melons).

[0080] 〔Supplementary matter 2〕 In the above-described embodiment, the specific unit 18 switched between the first mode and the second mode based on a user operation. The mode switching is not limited to that shown in the above-described embodiment, and the specific unit 18 may switch the mode by other methods. As an example, the specific unit 18 may switch between the first mode and the second mode based on at least any one of a user operation, information attached to the entire image, and an analysis result of the image. As an example, the specific unit 18 may perform mode switching based on information (for example, information indicating the generation date and time of the image) attached to the header or the like of the entire image.

[0081] Also, as another example, when the specific unit 18 executes the process of specifying the stock area in the first mode (the process of step S106 in FIG. 2) and fails to appropriately specify the stock (the specific result does not satisfy a predetermined condition), the specific unit 18 may switch from the first mode to the second mode. In this case, as an example, the predetermined condition may be a condition such that the number of stocks per unit area in the field is equal to or greater than a threshold value, or the number of stocks per unit distance in the longitudinal direction of the ridge is equal to or greater than a threshold value. For example, when adjacent stocks overlap due to the growth of plants, the specific unit 18 may switch from the first mode to the second mode when the stock cannot be appropriately specified in the first mode.

[0082] Also, the specific unit 18 may be configured not to perform mode switching. As an example, the specific unit 18 may execute the process of specifying a growth-deficient portion in the second mode without performing mode switching. That is, the specific unit 18 may analyze an image of a field in which a plurality of stocks of a predetermined plant are planted in a ridge, and specify a growth-deficient portion according to the width of the plant area (the area of the stock or the community) in the width direction of the ridge.

[0083] [Supplementary Note 3] The functions of the information processing apparatus 10 according to the above-described embodiment may be realized by a single device or may be realized by a system in which a plurality of devices cooperate. For example, the information processing apparatus 10 may be realized by a first device that implements the acquisition unit 14 and the generation unit 16 and a second device that implements the specification unit 18 and the output unit 20.

[0084] 〔Supplementary Note 4〕 In the above-described embodiment, the specification unit 18 specified the growth-deficient portion by analyzing the overall image generated from the aerial image taken by the flying object 30. The overall image analyzed by the specification unit 18 is not limited to the image generated from the aerial image taken by the flying object 30. As an example, the overall image may be a satellite image obtained by imaging the observation data of a sensor mounted on an artificial satellite.

[0085] 〔Supplementary Note 5〕 In the above-described embodiment, the information processing apparatus 10 analyzed the image of the farm field in which stocks were planted in one or more ridges and specified the growth-deficient portion. The farm field to be subjected to the image analysis is not limited to the farm field in which ridges are formed. As an example, the farm field may be a farm field in which a plurality of plants are planted side by side (for example, linearly) without forming ridges. In this case, the information processing apparatus 10 may specify the region including the plurality of plants planted side by side and specify the growth-deficient portion according to the size (width) in the short side direction (width direction) of the specified region. In this case, as an example, the information processing apparatus 10 calculates the width of the plant region (stock or community region) in the width direction (short side direction) of the above region at each of a plurality of positions separated by a predetermined distance in the longitudinal direction of the region of the plants planted side by side, and specifies the growth-deficient portion based on the calculated width.

[0086] 〔Example of Realization by Software〕 The control blocks of the information processing system 1 (particularly, the acquisition unit 14, the generation unit 16, the specification unit 18, and the output unit 20) may be realized by a logic circuit (hardware) formed in an integrated circuit (IC chip) or the like, or may be realized by software.

[0087] In the latter case, the information processing system 1 includes a computer that executes instructions of a program, which is software for realizing each function. This computer includes, for example, one or more processors and a computer-readable recording medium storing the above program. Then, in the above computer, when the above processor reads and executes the above program from the above recording medium, the object of the present invention is achieved. As the above processor, for example, a CPU (Central Processing Unit) can be used. As the above recording medium, in addition to "non-transitory tangible media" such as a ROM (Read Only Memory), a tape, a disk, a card, a semiconductor memory, a programmable logic circuit, etc. can be used. Further, it may further include a RAM (Random Access Memory) for expanding the above program. Also, the above program may be supplied to the above computer via any transmission medium (such as a communication network or a broadcast wave) capable of transmitting the program. Note that one aspect of the present invention can also be realized in the form of a data signal embedded in a carrier wave, in which the above program is embodied by electronic transmission.

[0088] The present invention is not limited to the above-described embodiments, and various modifications are possible within the scope shown in the claims. Embodiments obtained by appropriately combining technical means disclosed in different embodiments are also included in the technical scope of the present invention.

Explanation of Reference Numerals

[0089] 1 Information processing system 10 Information processing device 12 Control unit 14 Acquisition unit 16 Generation unit 18 Identification unit 20 Output unit 22 Storage unit 24 Input unit 26 Display unit 30 Flying object

Claims

1. Analyze an image of a field in which a plurality of plants of a specified plant are planted in rows, In the first mode, according to the area of each of the plurality of plants in the image, identify a growth-deficient area, which is an area representing the plant with poor growth in the image, In the second mode, a specifying unit that specifies the growth-deficient area according to the width of the plant area in the width direction of the row in the image, An information processing apparatus comprising the same.

2. The first mode is a mode corresponding to an image in which the occurrence frequency of overlapping of the plants satisfies a predetermined condition, and in the second mode, it is a mode corresponding to an image in which the occurrence frequency does not satisfy the condition, The information processing apparatus according to Claim 1.

3. In the first mode, the specifying unit classifies the plurality of plants in the image into a plurality of classes based on the area of each of the plurality of plants, and identifies an area representing the plants in the class that satisfies a predetermined condition among the plurality of classes as the growth-deficient area, The information processing apparatus according to Claim 1 or 2.

4. In the second mode, the specifying unit calculates the width of the plant area in the width direction of the row for each of a plurality of positions spaced apart at predetermined intervals in the longitudinal direction of the row, Classify the plurality of positions into a plurality of classes based on the width, and identify an area corresponding to the position in the class that satisfies a predetermined condition among the plurality of classes as the growth-deficient area, The information processing apparatus according to any one of Claims 1 to 3.

5. An output unit that outputs an image representing the field, and in the area representing the plant, represents the area specified by the specifying unit as the growth-deficient area and the other area in different display modes, The information processing apparatus according to any one of Claims 1 to 4, further comprising the same.

6. The specifying unit switches between the first mode and the second mode based on at least any one of a user operation, information attached to the image, and an analysis result of the image, The information processing apparatus according to any one of Claims 1 to 5.

7. Based on the analysis result of the image, the specifying unit identifies an area representing weeds included in the image, and identifies the growth-deficient area in an area other than the area representing the identified weeds, The information processing apparatus according to any one of Claims 1 to 6.

8. The specified plant is a pumpkin or a vine plant, The information processing apparatus according to any one of Claims 1 to 7.

9. The predetermined plant is a watermelon or a melon, The information processing apparatus according to claim 8.

10. In the second mode, the specifying unit specifies the growth-deficient portion according to the width of a region including the region of the plant and another region surrounded by the region of the plant. The information processing apparatus according to any one of claims 1 to 9.

11. Analyzing an image of a field in which a plurality of strains of a predetermined plant are planted in a row, In a first mode, according to the area of each of the plurality of strains in the image, specifying a growth-deficient portion, which is a region representing the plant with growth deficiency in the image, In a second mode, a step of specifying the growth-deficient portion according to the width of the region of the plant in the width direction of the row in the image. An information processing method including the above.

12. Analyzing an image of a field in which a plurality of strains of a predetermined plant are planted in a row, A specifying unit that specifies a growth-deficient portion, which is a region representing the plant with growth deficiency in the image, according to the width of the region of the plant in the width direction of the row in the image. An information processing apparatus including the above.

13. Analyzing an image of a field in which a plurality of strains of a predetermined plant are planted in a row, A step of specifying a growth-deficient portion, which is a region representing the plant with growth deficiency in the image, according to the width of the region of the plant in the width direction of the row in the image. An information processing method including the above.

14. A program for causing a computer to function as the information processing apparatus according to claim 1 or 12, and a program for causing a computer to function as the specifying unit.

15. A computer-readable recording medium recording the program according to claim 14.

Images