Growth analysis system, growth analysis method, and computer program

Abstract

To provide a growth analysis system capable of detecting camera tilt, etc., without requiring a user to manually check images.SOLUTION: The growth analysis system for analyzing images of plants captured by a camera to analyze the growth status of the plants includes image analysis means for analyzing, on the basis of an image showing the shadow of a marker near the camera, the inclination of the shadow and the growth status of the plants, and determination means for determining whether the state of the camera is abnormal on the basis of at least the amount of change in the inclination of the shadow.SELECTED DRAWING: Figure 10

Description

[Technical Field]

[0001] The present invention relates to a growth analysis system, a growth analysis method, a computer program, and the like. [Background technology]

[0002] To make up for the labor shortage caused by the decline in the number of agricultural workers and the aging of the population, it is being considered to introduce the latest IT technologies such as AI and IoT to, for example, monitor the growth status of rice. This monitoring involves, for example, using cameras installed in the fields to automatically take pictures of the growth status at set times each day, and visualizing the results of analyzing the images by graphing them.

[0003] The field cameras are powered by solar panels or other devices, and connect to a network using radio waves such as LTE (Long Term Evolution).

[0004] In such a growth monitoring system, the camera is installed in a remote location, so it is not possible to visually check the camera's condition and to confirm that the camera is installed in the correct orientation. As a technology to solve this problem, Patent Document 1 discloses a camera equipped with a tilt detection device that correctly represents the up-down relationship of the image when it is captured. [Prior art documents] [Patent documents]

[0005] [Patent Document 1] Japanese Patent Application Publication No. 11-341324 Summary of the Invention [Problem to be solved by the invention]

[0006] However, in such monitoring systems, the equipment such as cameras and batteries installed outdoors is easily affected by weather, animals, insects, etc.

[0007] As a result, such effects may cause the camera to change direction, resulting in the rice not being captured during shooting or the rice being photographed in a different location, making it impossible to capture images necessary for monitoring the growth status.One way to address this issue is to install a tilt detection device in the camera, but this creates other issues, such as the device being expensive and being installed in a remote location, making it prone to theft.

[0008] Furthermore, while it is possible to understand the camera's tilt by visiting the site and looking at the captured images, for farmers who are short on labor, this means having to spend a lot of time on tasks other than their main farming business, which reduces the system's usefulness to users. In order to solve the above-mentioned problems, one of the objects of the present invention is to provide a growth analysis system that can detect when the camera is tilted, without the user having to take the time to check images, etc. [Means for solving the problem]

[0009] A growth analysis system that analyzes an image of a plant taken by a camera and analyzes the growth status of the plant, an image analysis means for analyzing the tilt of the shadow of the mark and the growth status of the plant based on an image in which the shadow of the mark near the camera is captured; a determining means for determining whether the state of the camera is abnormal based on at least the amount of change in the inclination of the shadow; The present invention is characterized by having the following. [Effects of the Invention]

[0010] According to the present invention, it is possible to provide a growth analysis system that can detect when a camera is tilted, without requiring a user to take the time to check images, etc. [Brief explanation of the drawings]

[0011] [Figure 1] 1 is a system configuration diagram showing an example of the overall configuration of a growth analysis system 100 for monitoring the growth status of rice in an embodiment. [Figure 2] 1 is a block diagram showing an example of the hardware configuration of a camera 101 according to an embodiment. [Figure 3] FIG. 2 is a functional block diagram illustrating a program configuration of a cloud 103 according to the embodiment. [Figure 4] 10 is a flowchart illustrating an example of processing during initial camera setup in the embodiment. [Figure 5] FIG. 10 is a diagram showing an example of how to add a mark in step S401. [Figure 6] FIG. 10 is a diagram showing an example of camera installation in step S402. [Figure 7] FIG. 10 is a conceptual diagram illustrating transmission of a captured image to a cloud in an embodiment. [Figure 8] FIG. 10 is a diagram showing the concept of analyzing the inclination of the shadow of the mark in the captured image in step S405. [Figure 9] 10 is a flowchart showing an example of a process for determining the state of a camera 101 during operation of the growth analysis system 100 in the embodiment. [Figure 10] 10A and 10B are conceptual diagrams showing examples of the amount of change in the tilt of a shadow cast on a captured image in the embodiment. [Figure 11] FIG. 10 is a diagram showing an example of a table for determining the state of the camera based on the amount of change in the shadow tilt and the growth state in the embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0012] Hereinafter, embodiments of the present invention will be described with reference to the drawings. However, the present invention is not limited to the following embodiments. In each drawing, the same members or elements are designated by the same reference numerals, and duplicate descriptions will be omitted or simplified.

[0013] 1 is a system configuration diagram showing an example of the overall configuration of a growth analysis system 100 for monitoring the growth status of rice in an embodiment. Note that in the following embodiment, rice is used as an example of a plant, but this embodiment can also be applied to other plants, such as ornamental plants, other than rice.

[0014] That is, the growth analysis system of this embodiment is a system that analyzes the growth status of plants such as rice based on images of the plants such as rice taken with a camera.

[0015] Note that some of the functional blocks shown in FIG. 1 are realized by causing a CPU or the like serving as a computer (not shown) included in the growth analysis system 100 to execute a computer program stored in a memory (not shown) serving as a storage medium.

[0016] However, some or all of these functions may be implemented by hardware, which may be a dedicated circuit (ASIC) or a processor (reconfigurable processor, DSP).

[0017] 1 may not be contained in the same housing, but may be configured as separate devices connected to each other via signal paths. The above explanation regarding FIG. 1 also applies to FIG. 3.

[0018] The growth analysis system 100 includes a camera 101, a network 102, a cloud 103, and a client PC 109. The cloud 103 includes a device management unit 104, an analysis unit 106, a data management unit 105, a display unit 107, a recording unit 108, and the like.

[0019] The cloud 103 has at least one CPU (not shown) as a computer, and executes computer programs stored in a memory (not shown) to perform processing of each part in the cloud. Although an example using a cloud will be described in this embodiment, a server or the like may be used instead of a cloud.

[0020] Camera 101 is a camera installed in a field, facing downwards, and takes pictures of rice plants in the field. Camera 101 connects to network 102 using a communication device such as LTE, and transmits data to cloud 103 via network 102.

[0021] The device management unit 104 sets various settings and operation schedules for the device. The data management unit 105 receives various data and records it in an appropriate location in the recording unit .

[0022] The analysis unit 106 analyzes the data read from the recording unit 108. The data management unit 105 records the analysis results of the data, etc., in the recording unit 108. The client PC 109 can view the analysis results recorded in the recording unit 108 via the display unit 107.

[0023] 2 is a block diagram showing an example of the hardware configuration of the camera 101 according to the embodiment. The CPU 201 is a central processing unit serving as a computer, and controls the operation of each part of the camera 101 based on a computer program stored in memory.

[0024] The ROM 202 is a read-only memory that stores basic programs, data used for basic processing, etc. The RAM 203 is a writable memory that functions as a work area for the CPU 201. The recording device 204 is a device that functions as a large-capacity memory such as an SSD (solid state drive).

[0025] Various computer programs and data are stored in the recording device 204. The operation unit 205 is a device that accepts instructions and command input from a user, and corresponds to a touch panel, etc. The display unit 206 includes a display device for displaying commands input from the operation unit 205 and various response outputs thereto.

[0026] The communication I / F 207 is a device that relays data exchange with external devices, and transmits, for example, images captured by a camera to an external cloud, etc. It also receives instructions from the cloud or other external sources.

[0027] The input unit 208 is a device for inputting data such as images captured by the camera. The system bus 209 is a data bus that controls the flow of data within the camera 101. Note that software that realizes the same functions as the above devices can also be configured as an alternative to hardware devices.

[0028] Fig. 3 is a functional block diagram showing an example of the configuration of the cloud 103 according to the embodiment. 301 to 305 in Fig. 3 correspond to 104 to 108 in Fig. 1. A device management unit 301 sets a method for deleting data stored in the recording device 204 of the camera 101 and a shooting schedule via the network 102.

[0029] The device management unit 301 also authenticates whether the camera 101 is a valid device that can access the cloud 103. The device management unit 301 functions as a control means that changes at least one of the method for deleting data stored in the camera and the camera's shooting schedule depending on the plant variety and growing season.

[0030] The data management unit 302 records the data transmitted from the camera 101 in an appropriate location in the recording unit 305, and transmits the data stored in the recording unit 305 to the camera 101. The data management unit 302 also receives the results of data analysis from the analysis unit 303 and records them in the recording unit 108.

[0031] The analysis unit 303 analyzes data such as images recorded in the recording unit 305 by performing image recognition or the like. The analysis results are stored as is in the recording unit 305. Furthermore, the analysis results are graphed and stored in the recording unit 305. The analysis unit 303 functions as an image analysis means that analyzes the slope of the shadow of the marker and the growth status of the plant based on an image that captures the shadow of the marker near the camera.

[0032] The display unit 304 performs display control for displaying the results of the analysis by the analysis unit 303 on the client PC 109. The display unit 304 displays the results of the analysis by the analysis unit 303 as numerical values ​​or graphs. The recording unit 305 stores data such as camera settings managed by the data management unit 302, the analysis results of the analysis unit 303, and the like.

[0033] FIG. 4 is a flowchart showing an example of processing during initial camera setup in an embodiment, and shows an example of processing during initial camera setup performed when installing a camera in a farm field when constructing the growth analysis system shown in FIG. 1.

[0034] Furthermore, as the installer works, the CPUs and other computers contained in the camera and cloud of the growth analysis system 100 execute computer programs stored in memory, thereby sequentially performing the operations of each step in the flowchart of Figure 4.

[0035] Step S401 is a process in which an installer places a mark near the camera 101 when the camera 101 is installed.

[0036] FIG. 5 is a diagram showing an example of how to add marks 501 and 502 in step S401, and shows an example of the processing in step S401.

[0037] The mark may be attached near the camera 101, as in mark 501, or attached to a support pole 503, as in mark 502. In this embodiment, an example is shown in which the camera 101 is installed by sticking the support pole 503 into the ground 504 and fixing it there. The support pole 503 may be at least a part of the mark.

[0038] In such installation, the marks 501 and 502 may be placed anywhere as long as the shadows of the marks appear in the captured image and the marks can be fixed. The type of marks 501 and 502 may be any shape that can be analyzed based on the image. For example, a part of the housing of the camera 101 may be used as the mark, and the number of marks may be one or more.

[0039] Step S402 is a process of adjusting the installation orientation and position of the camera 101 so that the shadow of the mark added in step S401 appears in the image captured at meridian time.

[0040] FIG. 6 is a diagram showing an example of camera installation in step S402, and shows an example of processing for installation in step S402 when, for example, a mark 501 is placed close to the camera 101 in step S401.

[0041] 6, 601 is the sun, and for example, a shadow 602 of the mark 501 is set to be reflected in the shooting range of an image 603 captured by the camera 101 at the time of meridian. Moreover, it is desirable to install the camera 101 so that the shadow 602 is not reflected in an analysis area 604 for monitoring the growth status by the camera 101.

[0042] In this embodiment, the shadow faces due north at meridian time, which makes it easier to install the camera 101 and adjust the tilt of the correct value, so meridian time is used as an example, but the shooting time does not have to be meridian time as long as it is the same time.

[0043] In step S403, for example, by issuing a shooting instruction from a timer (not shown) or the operation unit 205, the shadow 602 of the mark 501 is photographed at the time of meridian, and the photographed image is stored in the recording device 204.

[0044] In step S404, the CPU of the camera 101 transmits the captured image saved in step S403 to the cloud 103. That is, the image saved in the recording device 204 in step S403 is transmitted to the cloud 103 via the communication I / F 207. The image sent to the cloud 103 is saved in the recording unit 305 in accordance with instructions from the data management unit 302.

[0045] FIG. 7 is a conceptual diagram of transmitting a captured image to the cloud in this embodiment, showing the concept of the process of transmitting a captured image to the cloud 103 in step S404.

[0046] In step S405, the cloud CPU analyzes the captured image and stores the tilt of the shadow of the landmark. That is, the captured image transmitted in step S404 and stored in the recording unit 305 is read, and the analysis unit 303 analyzes the image and calculates the tilt of the shadow 602.

[0047] 8 is a diagram showing the concept of analyzing the slope of the shadow of the landmark in the captured image in step S405. For example, an intersection 800 between the shadow 602 of the landmark and the shadow of the support pillar 503, and an intersection 801 between an extension of the shadow of the support pillar 503 and one of the sides of the captured image 603 are detected, and the slope α of the line connecting these two intersections is calculated as the slope of the shadow 602.

[0048] The tilt α of this shadow 602 is stored in the recording unit 305 as a correct value indicating that the camera 101 is installed correctly, and is used when comparing with the tilt of the shadow calculated during operation as described below.

[0049] The method for calculating the tilt of the shadow 602 is not limited to this. For example, it may be the angle between the shadow of the support 503 and the horizontal scanning direction or vertical scanning direction of the image on the camera. In other words, the analysis unit 303 may calculate the tilt based on the angle between the shadow of the landmark and the horizontal scanning direction or vertical scanning direction of the image.

[0050] In that case, the shadow of the support 503 can be used as the shadow of the mark. That is, the mark near the camera can be the support 503. In this way, the mark in this embodiment can be anything that includes at least one of 501, 502, the support 503, and the camera body.

[0051] In step S406, the cloud CPU analyzes the captured image and saves the growth status. That is, it analyzes the growth status of the rice plants shown in the image transmitted in step S404. Specifically, the captured image saved in the recording unit 305 is read, and the analysis unit 303 analyzes the image to obtain the growth status by quantifying the plant height, number of stalks, leaf color, etc.

[0052] The analysis unit 303 as an analysis means may be any unit that quantifies at least one of the plant height, number of stems, and leaf color as the growth status of the plant.

[0053] The growth status is saved in the recording unit 305 and is used for comparison with the growth status during operation, which will be described later. After the processing of step S406, the flow of Fig. 4 ends. Note that steps S405 and S406 here function as image analysis steps that analyze the slope of the shadow of the marker and the growth status of the plant based on an image that captures the shadow of the marker near the camera.

[0054] 9 is a flowchart showing an example of a process for determining the state of the camera 101 during operation of the growth analysis system 100 according to the embodiment. Note that the operations of the steps in the flowchart in FIG. 9 are sequentially performed by a CPU or the like serving as a computer included in each of the camera and the cloud of the growth analysis system 100 executing a computer program stored in memory.

[0055] In step S901, an image is captured by the CPU in the camera at meridian time, and the image is automatically captured by a timer at meridian time, and the captured image is saved in the recording device 204 in the camera.

[0056] In step S902, the CPU in the camera transmits the captured images saved in step S901 to the cloud 103. That is, the captured images saved in the recording device 204 in the camera are transmitted to the cloud 103 via the communication I / F 207. The images sent to the cloud 103 are saved in the recording unit 305 in accordance with instructions from the data management unit 302.

[0057] In step S903, the cloud CPU analyzes the captured image transmitted in step S902 and calculates the change in tilt between the correct tilt and the captured image. That is, the captured image stored in the recording unit 305 is read, and the analysis unit 303 analyzes the captured image and calculates the amount of change in tilt of the shadow 602. In this way, in this embodiment, the amount of change in tilt at the current noon time relative to the tilt at the noon time when the camera 101 was initially set is used as the amount of tilt change.

[0058] For example, the system detects an intersection 1001 between the shadow 1000 of the landmark 501 and the shadow of the support pillar 503, and an intersection 1002 between an extension of the shadow of the support pillar 503 and one of the sides of the captured image 603, and calculates the slope β of the line connecting the intersections 1001 and 1002. The system compares this slope β with the slope α stored in the recording unit 305 when the camera is correctly installed, calculates the difference (amount of change) θ, and stores it in the data management unit 302.

[0059] FIG. 10 is a conceptual diagram showing an example of the amount of change in the tilt of a shadow cast on a captured image in this embodiment. The amount of change in tilt θ is calculated as tan θ = tan(α - β). This amount of change in tilt is calculated periodically at predetermined intervals. Here, the predetermined interval is, for example, the period from the previous execution of the flow in FIG. 9 to the current execution.

[0060] In step S904, the cloud CPU analyzes the captured image and calculates the change in the growth status from the previous time of capture. That is, it calculates the amount of change in the growth status of the rice plants shown in the captured image transmitted in step S902. Specifically, the captured image stored in the recording unit 305 is read, and the analysis unit 303 analyzes the image to obtain the growth status in which the plant height, number of stalks, leaf color, etc. are quantified.

[0061] That is, the growth conditions, such as plant height, number of stems, and leaf color, which are quantified using the analysis unit 303 as image analysis means, are compared with the growth conditions at the time of photographing a predetermined period ago, which is the amount of change in the growth conditions at the time of photographing this time. In this manner, in this embodiment, the judgment is made based on the amount of change in the growth conditions, which is the amount of change in the growth conditions at the time of photographing a predetermined period ago, and the amount of change in the growth conditions at the time of photographing this time.

[0062] The analysis result is compared with the previously acquired growth status stored in the recording unit 305, and the amount of change in the growth status is calculated and stored in the recording unit 305.

[0063] In step S905, the cloud CPU determines the state of the camera, for example, whether or not the camera 101 is tilted, based on the amount of change in tilt calculated in step S903 and the amount of change in growth condition determined in step S904. Here, step S905 functions as a determination step (determination means) that determines whether or not the state of the camera is abnormal based on at least the amount of change in tilt, which is the amount of change in tilt of the shadow.

[0064] The determination result in step S905 is saved as a log and is also notified to the user's PC terminal, smartphone, etc. by email, etc. Alternatively, the determination result in step S905 may be displayed on a predetermined website so that pre-registered users can check it. After the processing of step S905, the flow in FIG. 9 ends.

[0065] Fig. 11 shows an example of a table for determining the state of the camera based on the amount of change in the tilt of the shadow and the growth situation in an embodiment. In table 1100 in Fig. 11, the amount of change in the tilt of the shadow and the amount of change in the growth situation are each divided into three levels: "NA," "small," and "large." Based on each amount of change, the state of the camera, for example, whether or not the camera 101 is tilted, is determined. Note that "NA" indicates a case where the calculation result of the amount of change could not be obtained.

[0066] For example, if the tilt of the shadow is small and the amount of change in the growth condition is small, it is determined that the camera 101 is installed correctly, and if the tilt of the shadow is large and the amount of change in the growth condition is small, it is determined that the camera 101 is tilted. The criteria for the determination may be any number of levels, or the amount of change in the tilt of the shadow alone may be used as the criterion for the determination.

[0067] It should be noted that the notification to the user in step S905 may not be performed if the camera is determined to be "normal" in Fig. 11. Only if the camera is not determined to be "normal" (if the camera state is determined to be abnormal) may the user be notified of a predetermined determination result such as that shown in Fig. 11.

[0068] In this manner, in this embodiment, the determining means is configured to transmit a predetermined notification to the outside at least when it determines that the state of the camera is abnormal.

[0069] 9 is executed, for example, every day or every few days. However, it may not be executed on days when bad weather is predicted based on the weather forecast at noon or when the illuminance is detected to be below a predetermined value by a camera image or an illuminance sensor at noon. Furthermore, the processes of steps S901 and S902 may be executed based on instructions from a CPU in the cloud.

[0070] In this way, in this embodiment, it is possible to determine whether the camera has tilted or whether there is an abnormality in the camera without using a special tilt detection device or without the user having to check images or videos, and the user can quickly become aware of such abnormalities.

[0071] The present invention has been described in detail above based on its preferred embodiments, but the present invention is not limited to the above embodiments, and various modifications are possible based on the spirit of the present invention, and these modifications are not excluded from the scope of the present invention.

[0072] The present invention also includes those that realize the functions of the above embodiments using, for example, at least one processor such as a CPU, memory, or circuit (for example, ASIC). Also, multiple processors may be used to perform distributed processing.

[0073] In order to realize part or all of the control in the above embodiments, a computer program that realizes the functions of the above embodiments may be supplied to a growth analysis system or the like via a network or various storage media. Then, a computer (or a CPU, MPU, etc.) in the growth analysis system or the like may read and execute the program. In this case, the program and the storage medium storing the program constitute the present invention. The present invention also includes the following combinations.

[0074] (Configuration 1) A growth analysis system that analyzes images of plants captured by a camera and analyzes the growth status of the plants, characterized by having an image analysis means that analyzes the slope of the shadow of a landmark near the camera and the growth status of the plant based on an image that captures the shadow of the landmark, and a judgment means that judges whether the condition of the camera is abnormal or not based on at least the amount of change in the slope of the shadow.

[0075] (Configuration 2) The growth analysis system described in Configuration 1, characterized in that the amount of change in tilt is the amount of change in tilt at the current noon time relative to the tilt at the noon time when the camera was initially set.

[0076] (Configuration 3) The growth analysis system according to configuration 1 or 2, characterized in that the judgment means makes the judgment based on the amount of change in growth status, which is the amount of change between the growth status at the time of photographing a predetermined period ago and the growth status at the time of photographing this time.

[0077] (Configuration 4) The growth analysis system according to Configuration 3, wherein the image analysis means quantifies at least one of the plant height, number of stems, and leaf color of the plant as the growth condition.

[0078] (Configuration 5) The growth analysis system described in Configuration 4, characterized in that the determination means compares the growth conditions quantified using the image analysis means, which is the amount of change in growth condition between the time of photographing the predetermined period before and the time of photographing this time.

[0079] (Configuration 6) A growth analysis system described in any one of configurations 1 to 5, characterized in that the image analysis means calculates the tilt based on the angle between the shadow of the mark and the horizontal scanning direction or vertical scanning direction of the image in the camera.

[0080] (Configuration 7) The growth analysis system according to any one of configurations 1 to 6, wherein the determining means periodically calculates the amount of change in slope at predetermined intervals.

[0081] (Configuration 8) The growth analysis system according to any one of configurations 1 to 7, wherein the determining means transmits a predetermined notification to the outside when determining that at least the state of the camera is abnormal.

[0082] (Method) A growth analysis method that analyzes an image of a plant taken by a camera and analyzes the growth status of the plant, characterized by having an image analysis step that analyzes the slope of the shadow of a landmark near the camera and the growth status of the plant based on an image that captures the shadow of the landmark, and a judgment step that determines whether the condition of the camera is abnormal based on at least the slope change amount, which is the amount of change in the slope of the shadow.

[0083] (Program) A computer program for controlling each means of the growth analysis system according to any one of configurations 1 to 8 by a computer. [Explanation of symbols]

[0084] 100: Growth analysis system 101: Camera 102: Communication path 103: Cloud 109: Client PC

Claims

1. A growth analysis system that analyzes an image of a plant taken by a camera and analyzes the growth status of the plant, an image analysis means for analyzing the tilt of the shadow of the mark and the growth status of the plant based on an image in which the shadow of the mark near the camera is captured; a determining means for determining whether the state of the camera is abnormal based on at least the amount of change in the inclination of the shadow; A growth analysis system comprising:

2. 2. The growth analysis system according to claim 1, wherein the amount of change in tilt is the amount of change in tilt at the current noon time relative to the tilt at the noon time when the camera was initially set.

3. 2. The growth analysis system according to claim 1, wherein the determining means makes the determination based on a growth status change amount, which is a change amount between the growth status at the time of photographing a predetermined period before and the growth status at the time of photographing this time.

4. 4. The growth analysis system according to claim 3, wherein the image analysis means quantifies at least one of the plant height, the number of stems, and the leaf color of the plant as the growth condition.

5. The growth analysis system described in claim 4, characterized in that the judgment means compares the growth conditions quantified using the image analysis means, which is the amount of change in growth condition between the time of photographing the plant the specified period before and the time of photographing the plant this time.

6. 2. The growth analysis system according to claim 1, wherein the image analysis means calculates the tilt based on an angle between the shadow of the mark and a horizontal or vertical scanning direction of the image captured by the camera.

7. 2. The growth analysis system according to claim 1, wherein the determining means periodically calculates the amount of change in slope at predetermined intervals.

8. 2. The growth analysis system according to claim 1, wherein the determining means transmits a predetermined notification to an external device when it determines that at least the state of the camera is abnormal.

9. A growth analysis method for analyzing an image of a plant captured by a camera and analyzing the growth status of the plant, comprising: an image analysis step of analyzing the tilt of the shadow of the mark and the growth status of the plant based on an image in which the shadow of the mark near the camera is captured; a determining step of determining whether or not the state of the camera is abnormal based on at least an amount of change in tilt of the shadow; A growth analysis method comprising the steps of:

10. A computer program for controlling each means of the growth analysis system according to any one of claims 1 to 8 by a computer.

Images