Plant imaging system

The plant imaging system uses a reflective mirror and adjustable imaging device configuration to achieve a wider field of view within plant factories, addressing the challenge of monitoring plant growth in limited spaces and improving yield management by reducing the need for multiple cameras.

WO2024247590A1PCT designated stage expired Publication Date: 2024-12-05OMRON CORP

Patent Information

Application Number
PCT/JP2024/016653
Authority / Receiving Office
WO · WO
Patent Type
Applications
Current Assignee / Owner
Priority Date
2023-05-31
Filing Date
2024-04-30
Publication Date
2024-12-05

AI Technical Summary

Technical Problem

In plant factories, it is challenging to secure a wide viewing range for monitoring plant growth due to limited space, leading to delayed detection of defective plants and increased costs from using multiple cameras.

Method used

A plant imaging system utilizing a first mirror to reflect images and an imaging device that captures these reflections, either directly or indirectly, with mechanisms to adjust the reflection direction and placement to enhance the field of view while reducing the number of required imaging devices.

Benefits of technology

Enables a wider viewing range with fewer imaging devices, reducing costs and improving early detection of plant defects, thus enhancing yield management in plant factories.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure JP2024016653_05122024_PF_FP_ABST
    Figure JP2024016653_05122024_PF_FP_ABST
Patent Text Reader

Abstract

A plant imaging system including: a first mirror for reflecting an image of a plant; and an imaging device for directly or indirectly imaging the reflection image reflected by the first mirror.
Need to check novelty before this filing date? Find Prior Art

Description

Plant Imaging System

[0001] The present invention relates to a plant imaging system.

[0002] Patent Document 1 discloses a plant cultivation device having a plurality of sensors that monitor the growth state of the plants being cultivated, an environmental control means that controls the environment, which is at least one of the conditions of light, air, water, and space, within the plant cultivation device, and a process control means that controls the work process of cultivating the plants, with the aim of providing a mechanism for both environmental control within the device and process control for the work process of cultivating the plants.

[0003] Patent Publication No. 2021-166495

[0004] In this embodiment, a plant imaging system is provided that can ensure a wider field of view even in a location where it is difficult to ensure a wide field of view, even with a small number of imaging devices.

[0005] A plant imaging system according to one aspect of the present invention includes a first mirror that reflects an image of a plant, and an imaging device that directly or indirectly captures the image reflected by the first mirror.

[0006] According to the present invention, it is possible to provide a plant imaging system that can ensure a wider field of view even with a small number of imaging devices in a location where it is difficult to ensure a wide field of view.

[0007] 1B is a schematic perspective view showing one aspect of the plant imaging system of the present embodiment; FIG. 1C is a schematic cross-sectional view showing one aspect of the plant imaging system of the present embodiment; FIG. 1D is a flowchart showing one aspect of the processing of a conversion device; and FIG. 1E is a schematic diagram showing the configuration of the plant imaging system in the upper part of FIG.

[0008] Hereinafter, an embodiment of the present invention (hereinafter referred to as "the present embodiment") will be described in detail with reference to the drawings as necessary. However, the present invention is not limited to this embodiment, and various modifications are possible without departing from the spirit of the present invention. In the drawings, the same elements are given the same reference numerals, and redundant explanations will be omitted. Furthermore, positional relationships such as up, down, left, and right are based on the positional relationships shown in the drawings unless otherwise specified. Furthermore, the dimensional ratios of the drawings are not limited to those shown in the drawings.

[0009] 1. Plant Factory The plant imaging system 300 of this embodiment may be used in, for example, a plant factory. Before describing the plant imaging system 300 of this embodiment, an overview of a plant factory will be described. Note that the use of the plant imaging system 300 of this embodiment is not limited to a plant factory.

[0010] A plant factory is a type of greenhouse horticulture where plants are grown by controlling their growth environment (light, temperature, humidity, carbon dioxide concentration, nutrients, water, etc.) within the facility. This refers to a cultivation facility that allows for year-round, planned production of vegetables and other plants by controlling the environment and predicting growth based on environmental and growth monitoring. Plant factories can be broadly divided into two types: "fully artificial light type," which controls the environment without using sunlight in a closed environment and performs year-round, planned production, and "sunlight-utilizing type," which uses sunlight in a semi-closed environment such as a greenhouse and performs year-round, planned production by using supplemental lighting on rainy or cloudy days and techniques to suppress high temperatures in the summer.

[0011] The cultivation method carried out in the plant factory may be either hydroponic cultivation or soil cultivation. Here, hydroponic cultivation is a method of growing plants using water and liquid fertilizer (nutrient solution) without using soil, and is also called hydroculture. In hydroponic cultivation, the roots of the plants are immersed in water (solution) containing fertilizer, and the necessary water, nutrients, and oxygen are absorbed through the roots. In contrast, soil cultivation refers to cultivation using soil. Hereinafter, this embodiment will be described using an example of carrying out hydroponic cultivation in a plant factory, but this embodiment is not limited to hydroponic cultivation.

[0012] FIG. 1A shows a schematic perspective view of a plant factory. In the plant factory shown in FIG. 1A, plants 200 are grown on cultivation panels 120 installed on cultivation shelves 110. The cultivation shelves 110 are configured in a multi-tiered structure, which allows the plants 200 to be grown over a large area while saving space. However, the space between the tiers is narrow and the installation locations for the imaging device 410 and the like are limited, making it difficult to monitor the growth of the plants 200 on each tier.

[0013] A cultivation tank 130 is installed on each level of the cultivation shelf 110, and the cultivation panels 120 float in a pool of nutrient solution flowing through the cultivation tank 130. As an example, the cultivation tank 130 can store water used in hydroponic cultivation, and it is preferable that the water flows in a predetermined direction without stagnating. To create a water flow, each cultivation tank 130 may be provided with a water inlet and a drain outlet. Furthermore, the water flowing through the cultivation tank 130 may contain dissolved nutrients required for hydroponic cultivation.

[0014] The cultivation panel 120 may be a floating board that can float in the nutrient solution pool with the plant 200 placed on it. The cultivation panel 120 may have a through-hole for installing the plant 200 to be grown. In this case, the plant 200 is fixed to the cultivation panel 120 with its leaves exposed to the air and its roots immersed in the nutrient solution pool.

[0015] The cultivation panel 120 may be fixed to a predetermined position in the cultivation tank 130, or may be movable horizontally in the cultivation tank 130. As an example, in a cultivation shelf 110 configured to be long in the longitudinal direction Y, the cultivation panel 120 may be moved from one end to the other end of the cultivation tank 130 over a period of several hours to several days. Specifically, the cultivation panel 120 with the seedlings of the plants 200 fixed thereto may be introduced into one end of the cultivation shelf 110, and the plants 200 grown from the seedlings may be discharged from the other end of the cultivation shelf 110 after several days. This allows the plants 200 to be grown continuously.

[0016] In soil cultivation, plants are prone to disease due to various bacteria in the soil, necessitating the use of pesticides, so it is desirable to monitor the plants and check for any abnormalities. In contrast, hydroponic cultivation in plant factories is generally said to be less prone to insect infestation and disease. However, plants do not necessarily grow smoothly, and even in a completely enclosed environment, defective plants can occur when growth conditions tend to deviate from the adjusted range, such as during seasonal changes. Examples of defective plants here include, for leafy vegetables, defects in the head size or number of leaves, or various symptoms such as chip burn.

[0017] In outdoor cultivation, defective plants are easily identified by visual inspection, but as shown in Figure 1A, because the height of each shelf of the cultivation shelf 110 is low, it is difficult to visually check every corner of the shelf for defective plants. In other words, defective plants are often discovered late. This can lead to missed treatment times for plants that could have been improved, or defective plants being grown to the end, which can reduce factory yields.

[0018] To manage a plant factory more efficiently, it is desirable to detect defects early and strive to improve harvest yields. Therefore, it is conceivable to use a camera to observe the condition of the plants 200 on the cultivation shelves 110. However, because the height of each shelf of the cultivation shelves 110 is low, the field of view of the camera is also narrow. Furthermore, if an attempt is made to ensure a wide field of view, the number of cameras must be increased, which increases overall costs. Therefore, in this embodiment, as an example, a plant imaging system suitable for the environment of a plant factory is provided.

[0019] 2. Plant Imaging System The plant imaging system 300 of this embodiment includes a first mirror 310 that reflects an image of a plant, and an imaging device 410 that directly or indirectly captures the image reflected by the first mirror 310. Here, the first mirror 310 may reflect an image of a plant 200 grown on a cultivation panel 120 installed on a cultivation shelf 110.

[0020] Here, directly capturing a reflected image means that the imaging device 410 captures an image of the plant 200 reflected in the first mirror 310. Indirectly capturing a reflected image means that another mirror, such as the second mirror 320, is interposed in the optical path between the first mirror 310 and the lens of the imaging device 410, and the imaging device 410 captures an image of the plant 200 reflected in the other mirror, thereby capturing an image of the plant 200 reflected in the first mirror 310.

[0021] 1B shows an XZ cross-sectional view of the cultivation shelf 110 shown in FIG. 1A. As shown in FIG. 1B, for example, the first mirror 310 may be installed inside the cultivation shelf 110. This allows the first mirror 310 to be placed in a position that is easy to reflect the image of the plant 200 on the cultivation shelf 110, making it easier for the imaging device 410 to image a wider range within the cultivation shelf 110. In addition, in relation to this, the number of imaging devices 410 necessary to observe the entire cultivation shelf 110 can be reduced.

[0022] Alternatively, the first mirror 310 may be installed outside the cultivation shelf 110. This makes it less likely that the first mirror 310 will block the light from the light source 150, resulting in a decrease in the amount of light.

[0023] The plant imaging system 300 may include a first swing mechanism 311 that changes the reflection direction of the first mirror 310 and a second swing mechanism 321 that changes the reflection direction of the second mirror 320. Changing the reflection direction of the first mirror 310 using the first swing mechanism 311 can increase the range over which the first mirror 310 reflects the image of the plant 200 on the cultivation shelf 110. Changing the reflection direction of the second mirror 320 using the second swing mechanism 321 can appropriately reflect the image of the plant 200 reflected by the first mirror 310 to the imaging device 410. In other words, the inclusion of the first swing mechanism 311 and the second swing mechanism 321 tends to increase the range over which the imaging device 410 can capture an image. In addition, related to this, the number of imaging devices 410 required to observe the entire cultivation shelf 110 can be reduced.

[0024] The imaging device 410 may be installed outside the cultivation shelf 110, or may be installed in a location other than above the plants 200, regardless of its position relative to the cultivation shelf 110. The plants 200, the cultivation panel 120, the cultivation tank 130, and the areas around and above the plants 200 are humid spaces. Therefore, installing the imaging device 410 outside the cultivation shelf 110 can avoid evaporation from the plants 200 and prevent malfunctions due to humidity or poor imaging due to lens fogging. Furthermore, since the imaging device 410 can be spaced farther from the plants 200, the field of view of the imaging device 410 tends to be larger. This also reduces the number of imaging devices 410 required to observe the entire cultivation shelf 110. Furthermore, since there is no need to install the imaging device 410 in the narrow space between the cultivation tanks 130 of the cultivation shelf 110, the design flexibility of the system configuration is increased. Alternatively, the imaging device 410 may be installed inside the cultivation shelf 110. In this case, it is preferable to protect the imaging device 410 from moisture by enclosing it in a storage box or the like.

[0025] Furthermore, the first mirror 310 may have a heater for heating the first mirror 310 in order to prevent condensation due to humidity. This prevents the image reflected by the first mirror from becoming blurred due to condensation. From a similar perspective, the imaging device 410 may have a lens heater in order to prevent condensation due to humidity. This prevents the captured image from becoming blurred due to condensation on the lens.

[0026] The plant imaging system 300 may have a third swing mechanism 411 that changes the orientation of the lens. This tends to increase the range that can be imaged by the imaging device 410. In addition, the number of imaging devices 410 required to observe the entire cultivation shelf 110 can be reduced.

[0027] The plant imaging system 300 may have a first storage box 440 that stores the lens unit of the imaging device 410 when imaging is not being performed. This makes it possible to prevent transpiration from the plant 200 and avoid malfunctions due to moisture. For example, the third swing mechanism 411 may be used to move the lens unit out of the first storage box 440 when imaging is being performed, and store the lens unit in the first storage box 440 when imaging is not being performed.

[0028] The plant imaging system 300 may also have a second storage box 420 that stores the imaging device 410. This prevents evaporation from the plant 200 and prevents malfunctions due to moisture. For example, the imaging device 410 may capture a reflected image through a window 421 provided in the second storage box 420.

[0029] The first storage box 440 and the second storage box 420 may have a humidity adjustment mechanism 430 for reducing the humidity inside. The humidity adjustment mechanism 430 may be, for example, a blower such as a fan. This can prevent transpiration from the plants 200 and prevent malfunctions due to humidity.

[0030] The plant imaging system 300 may include a conversion device 500 that converts the reflected image captured by the imaging device 410 into a planar image of the cultivation panel 120 when viewed from above. The conversion device 500 may be, for example, a general-purpose computer, such as a desktop computer, a laptop computer, or other computer.

[0031] The size of the image captured by the imaging device 410 is determined by the distance from the mirror to the object and its position relative to the mirror surface. For example, even if the plants 200 are actually the same size, the images captured by the imaging device 410 will differ in size and other properties due to differences in the optical path length from the plant 200 to the first mirror 310 and the imaging device 410. Furthermore, if the first mirror 310 has a curved surface, the distortion rate and other properties will differ.

[0032] Therefore, the conversion device 500 may convert the reflected image acquired by the imaging device 410 into a planar image based on imaging conditions such as the optical path length and the curvature of the mirror, and control the display of the image on a display provided in the conversion device 500 as necessary. Furthermore, the conversion device 500 may acquire tilt information from the first swing mechanism 311 and the second swing mechanism 321 as an imaging condition, and accordingly convert the reflected image acquired by the imaging device 410 into a planar image of the cultivation panel 120 when viewed from above. Figure 2 shows a flowchart of the processing by the conversion device 500. This allows the user to more appropriately grasp the growth status of the plants 200 on the cultivation shelf 110.

[0033] The first swing mechanism 311, the second swing mechanism 321, the imaging device 410, the third swing mechanism 411, and the conversion device 500 may be configured to transmit and receive various data to and from each other via the network N. The communication may be performed either wired or wirelessly, and any communication protocol may be used as long as the communication between them is possible.

[0034] A specific installation mode of the plant imaging system 300 will be described below with reference to FIGS. 1B and 3. FIG.

[0035] 1B and 3, the plant imaging system 300 may further include, in addition to the first mirror 310, a second mirror 320 that reflects the reflected image reflected by the first mirror 310, and the imaging device 410 may capture the reflected image reflected by the second mirror 320. In this case, the first mirror 310 may be installed inside the cultivation shelf 110, and the imaging device 410 may be installed outside the cultivation shelf 110.

[0036] 3, the imaging device 410 is housed in a second housing box 420, and the imaging device 410 captures an image reflected by the second mirror 320 through a window 421 provided in the second housing box 420. This makes it possible to prevent transpiration from the plant 200 and avoid malfunctions due to moisture.

[0037] The plant imaging system 300 may have a first swing mechanism 311 that changes the reflection direction of the first mirror 310, or a second swing mechanism 321 that changes the reflection direction of the second mirror 320. By changing the inclinations of the first mirror 310 and the second mirror 320 using the first swing mechanism 311 and the second swing mechanism 321, the imaging device 410 can easily capture an image of a wider range within the cultivation shelf 110.

[0038] This allows for a wider field of view that can be captured by one imaging device 410. This also makes it possible to reduce the number of imaging devices 410 required to capture an image of the entire cultivation shelf 110. Furthermore, since the imaging device 410 can be installed outside the cultivation shelf 110, it is possible to prevent the imaging device 410 from being affected by moisture.

[0039] Furthermore, the conversion device 500 may acquire information regarding tilt from the first swing mechanism 311 and the second swing mechanism 321, and accordingly convert the reflected image captured by the imaging device 410 into a planar image when the cultivation panel 120 is viewed in a plane.

[0040] Furthermore, if the cultivation panel 120 is movable in the cultivation tank 130 in the longitudinal direction Y, the cultivation panel 120 may move in the longitudinal direction Y during the process of capturing images while changing the inclination of the first mirror 310 and the second mirror 320, which may result in a shift in the position of the plant 200 in the direction Y. In such a case, the conversion device 500 may acquire information regarding the amount of movement of the cultivation panel 120 in the direction Y, and accordingly correct the position of the plant 200 in the reflected image captured by the imaging device 410 so that it matches the position, and output a planar image.

[0041] As shown in the middle of FIG. 1B , the first mirror 310 may be a half mirror located between the cultivation panel 120 and the light source 150 located above the cultivation panel 120. As a result, light emitted from the light source 150 passes through the half mirror (first mirror 310) and is irradiated downward (toward the cultivation panel 120). The light passing through the half mirror (first mirror 310) is reflected by an object (plant 200) located below the half mirror (first mirror 310), then reflected again by the half mirror (first mirror 310), passes through the lens of the imaging device 410, and enters the imaging element. That is, the imaging device 410 can capture an image of an object present in the illumination area of ​​the light source 150 via the half mirror (first mirror 310).

[0042] With a conventional mirror, the installation position is limited so as not to block the light from the light source 150, thereby reducing the amount of light reaching the plants, and therefore the range in which the image can be reflected is also limited. Therefore, it is necessary to install a larger number of small mirrors and imaging devices 410. However, by utilizing a half mirror as described above, even if the first mirror 310 is placed between the light source 150 and the plant 200, it is possible to deliver light from the light source 150 to the plant 200, and the first mirror 310 can reflect and detect the image of the plant 200 below. This ensures the field of view of the imaging device 410, allowing for a plant imaging system to be installed at lower cost and with greater ease.

[0043] As shown in the lower part of FIG. 1B , the first mirror 310 may be a wide-angle curved mirror. For example, if a half mirror is used, the amount of light from the light source 150 may be attenuated as it passes through the half mirror. However, by using a wide-angle curved mirror instead of the half mirror, such attenuation of light can be avoided and a wider range of image can be reflected. Furthermore, if a regular flat mirror instead of a half mirror is installed, the installation area is limited to avoid blocking the light source 150, and the range of reflection is also limited. In contrast, by using a wide-angle curved mirror, a wider range of image can be reflected than with such a flat mirror. Therefore, a wider range of image can be reflected without installing as many flat mirrors or image capture devices 410.

[0044] When a wide-angle curved mirror is used as the first mirror 310, the conversion device 500 may acquire information regarding the curvature of the wide-angle curved mirror and, accordingly, convert the reflected distorted surface image captured by the imaging device 410 into a planar image when the cultivation panel 120 is viewed in a plane.

[0045] The present invention has industrial applicability as a component technology that can be used in smart agriculture systems.

[0046] REFERENCE SIGNS LIST 100...plant factory, 110...cultivation shelf, 120...cultivation panel, 130...cultivation tank, 150...light source, 200...plant, 300...plant imaging system, 310...first mirror, 311...first swing mechanism, 320...second mirror, 321...second swing mechanism, 410...imaging device, 411...third swing mechanism, 420...second storage box, 421...window, 430...humidity adjustment mechanism, 440...first storage box, 500...conversion device

Claims

1. A plant imaging system comprising: a first mirror that reflects an image of a plant; and an imaging device that directly or indirectly captures the image reflected by the first mirror.

2. The plant imaging system according to claim 1, wherein the first mirror reflects an image of the plant grown on a cultivation panel installed on a cultivation shelf, and the first mirror is installed on the inside of the cultivation shelf.

3. The plant imaging system of claim 1, wherein the first mirror reflects an image of the plant grown on a cultivation panel installed on a cultivation shelf, the first mirror is a half mirror, and the half mirror is positioned between the cultivation panel and a light source positioned above the cultivation panel.

4. The plant imaging system according to claim 1, wherein the first mirror reflects an image of the plant grown on a cultivation panel installed on a cultivation shelf, and the first mirror is a wide-angle curved mirror.

5. The plant imaging system according to claim 1, further comprising a first swing mechanism for changing the reflection direction of the first mirror.

6. A plant imaging system as described in claim 1, wherein the first mirror reflects an image of the plant grown on a cultivation panel installed on a cultivation shelf, and the system has a second mirror that reflects the reflected image reflected by the first mirror, and the imaging device captures the reflected image reflected by the second mirror.

7. The plant imaging system according to claim 6, further comprising a second swing mechanism for changing the reflection direction of the second mirror.

8. The plant imaging system according to claim 1, wherein the first mirror reflects an image of the plant grown on a cultivation panel installed on a cultivation shelf, and the imaging device is installed outside the cultivation shelf.

9. The plant imaging system according to claim 1, further comprising a third swing mechanism for changing the orientation of the lens of the imaging device.

10. The plant imaging system according to claim 1, further comprising a first storage box for storing a lens portion of the imaging device when imaging is not being performed.

11. A plant imaging system as described in claim 1, further comprising a second storage box for storing the imaging device, the imaging device capturing the reflected image through a window provided in the second storage box.

12. The plant imaging system of claim 1, wherein the first mirror reflects an image of the plant being grown on a cultivation panel installed on a cultivation shelf, and the system further comprises a conversion device that converts the reflected image captured by the imaging device into a planar image of the cultivation panel when viewed in a plane.

13. The plant imaging system according to claim 1, wherein the first mirror reflects an image of the plant grown on a cultivation panel installed on a cultivation shelf, and the cultivation shelf is of a multi-tiered type.

Citation Information

Patent Citations

  • Inspection device for appearance of farm and merine products

    JP1997062839A

  • Agricultural product evaluation system

    JP2004203514A

  • Screening device for screening plant specimens

    JP2014520524A

  • System and method for determining an attribute of a plant

    US20210019903A1

Cited By

  • Plant management system

    WO2026176831A1