Presents images stored with the planting plan along with the conditions for each planting stage

Abstract

A system is configured to obtain a plurality of images (83) of a plant and store a plurality of the plurality of images of the plant together with a planting plan (71) for planting the plant. Each image is associated with a different capture moment. The system is further configured to select the planting plan separately from the plant and present the plurality of images when the planting plan is selected. Each image is presented together with one or more desired and / or measured conditions (74‑76) for a planting stage (84). The planting stage corresponds to the capture moment of the corresponding image. The planting plan includes a plurality of planting stages, and the one or more desired and / or measured conditions are included in the planting plan.

Description

Technical Field

[0001] The present invention relates to a system for obtaining plant images.

[0002] The present invention further relates to a method for obtaining plant images.

[0003] The invention also relates to a computer program product enabling a computer system to perform such a method. Background Art

[0004] Horticultural systems, such as horticultural lighting systems, are becoming increasingly advanced. If cameras are integrated into horticultural systems, images can be captured frequently during the various growing stages of a plant or over the duration of a growing recipe application.

[0005] For example, US2018 / 0359931A1 discloses an image capture system for a planting pod, comprising a main controller having a processor, a memory, and a camera communicatively coupled to the main controller and positioned to capture images of plants or seeds. The planting recipe defines instructions for planting plants or seeds and expected properties corresponding to the instructions. The main controller receives an image of the plant or seed from the camera, determines properties of the plant or seed from the image, compares the properties of the plant or seed from the image with the expected properties defined by the planting recipe, and adjusts the instructions of the planting recipe for planting the plant or seed based on the comparison of the properties with the expected properties.

[0006] The gardening system allows growers to automatically control the planting of their plants. However, growers usually have to choose from many planting recipes for the same plant species, and the gardening system does not help growers choose a planting recipe. Summary of the Invention

[0007] A first object of the present invention is to provide a system for managing plant cultivation, which improves human-computer interaction to help growers select planting recipes.

[0008] A second object of the present invention is to provide a method that improves human-computer interaction in a system for managing plant cultivation to help growers select planting recipes.

[0009] In a first aspect of the present invention, a system for obtaining plant images includes at least one input interface, at least one output interface and at least one processor, wherein the at least one processor is configured to use the at least one input interface to obtain multiple images of the plant, each of the images being associated with a different capture moment, store multiple of the multiple images of the plant together with a planting plan for planting the plant, select the planting plan separately from the plant, and use the at least one output interface to present the multiple images when the planting plan is selected, each of the multiple images being presented together with one or more desired and / or measurement conditions of a planting stage corresponding to the capture moment of the corresponding image, and the planting plan including multiple planting stages, and the one or more desired and / or measurement conditions being included in the planting plan.

[0010] Representative images stored with a planting plan (also called a planting recipe) help growers see what results can be expected when the planting plan is applied to a given plant species or a given plant variety within a given plant species. By timing the presentation of plant images to coincide with corresponding desired and / or measured conditions, growers can better understand how the desired and / or measured conditions affect the planting of a given plant species. Images of individual plants are stored with the planting plan for that plant species to make this possible.

[0011] Thus, the captured images are used to augment a (pre-stored) planting plan with representative images of the plant at different planting stages or even at different moments within a planting stage. A graphical user interface shows a time-scale representation of the planting plan using plant images. For example, each planting stage can be one day. The one or more desired and / or measured conditions can include, for example, lighting conditions and / or climatic conditions and / or nutritional conditions. For example, the plant can be a flowering plant, i.e. a plant that reproduces by producing flowers, or a non-flowering plant.

[0012] The at least one processor may be configured to present the plurality of images as a video sequence, the images being in order of planting time elapsed in the video sequence. By creating such a time lapse, a grower may be able to obtain a visual overview of all planting stages of a planting plan without much effort.

[0013] The at least one processor can be configured to receive user input including user commands for navigating through the planting stages using the at least one input interface, and select one or more images to be presented next from the plurality of images based on the user commands. This can help a grower more easily explore and study a particular planting stage of a planting plan.

[0014] The at least one processor may be configured to select a representative subset of the plurality of acquired images as the plurality of images before storing the plurality of images of the plant with the planting plan. If a camera automatically provides images at a high rate, then storing all of the images is generally unhelpful, and selecting a representative subset may be beneficial.

[0015] The at least one processor may be configured to receive user input using the at least one input interface, the user input identifying an additional plant and the planting plan, obtain an image of the identified additional plant, determine a difference between the obtained image and a plurality of images stored with the identified planting plan, and provide an alert using the at least one output interface if the difference is determined to exceed a predetermined threshold. Thus, an image captured in real time may be compared with a representative image stored with the planting plan. This allows a grower to select an individual plant and be alerted if the planting of the individual plant is not as expected given the representative image stored with the planting plan.

[0016] The at least one processor can be configured to transmit, using the at least one output interface, a capture schedule to one or more cameras for capturing images of the identified additional plants. This can be used to prevent the transmission of unused images. For example, the capture schedule can specify that images be captured and transmitted each day.

[0017] The at least one processor may be configured to receive user input using the at least one input interface, the user input comprising a camera or location identifier and information identifying the planting plan, store the camera or location identifier with the planting plan, obtain a collection of images of a plurality of plants using the at least one input interface, the collection comprising the plurality of images, select the plurality of images from the collection of images based on the camera or location identifier, and store the plurality of images of the plants with the planting plan. Images of different individual plants may be received and then stored with different planting plans. By storing the camera or location identifier in the planting plan, it is easy to determine in which planting plan a received image should be stored, i.e., by finding a planting plan that is already associated with the camera or location identifier contained in or transmitted together with the image.

[0018] The at least one processor may be configured to transmit, using the at least one output interface, a capture schedule to the one or more cameras for capturing the plurality of images. This may be used to prevent the transmission of images that will not be stored with the planting recipe and not otherwise used. For example, the capture schedule may specify that images be captured and transmitted daily.

[0019] The at least one processor may be configured to use the at least one output interface to control one or more cameras to capture the plurality of images at the different capture times. This is beneficial, for example, if a camera is not capable of using a capture schedule but its capture function can be remotely controlled.

[0020] The at least one processor may be configured to use the at least one input interface to obtain a current position of the plant relative to the one or more cameras, and to use the at least one output interface to control the one or more cameras to capture one of the images at a time dependent on the current position. If individual plants (e.g. in a vertical farm) are moving on a conveyor belt or in a mobile water tank system, or as part of a collection of plants in mobile trays, and images of individual plants are captured in order to store them in a planting plan for a certain plant species, it is necessary to capture the image at the time when the individual plant is in front of the camera.

[0021] The one or more cameras may include a plurality of cameras, and the at least one processor may be configured to select one of the plurality of cameras based on the current location. This may be beneficial, for example, if a camera is not capable of using a capture schedule but its capture function can be remotely controlled. This may be useful, for example, for following an individual plant as it moves on a conveyor belt in a vertical farm.

[0022] In a second aspect of the invention, a method for obtaining an image of a plant comprises obtaining a plurality of images of a plant, each of the images being associated with a different moment of capture, storing a plurality of the plurality of images of the plant together with a planting plan for planting the plant, selecting the planting plan separately from the plant, and presenting the plurality of images when the planting plan is selected, each of the plurality of images being presented together with one or more desired and / or measured conditions for a planting stage corresponding to the moment of capture of the corresponding image, the planting plan comprising a plurality of planting stages, and the one or more desired and / or measured conditions being included in the planting plan. The method may be performed by software running on a programmable device. The software may be provided as a computer program product.

[0023] Furthermore, a computer program for carrying out the methods described herein and a non-transitory computer-readable storage medium storing the computer program are provided. For example, the computer program can be downloaded from or uploaded to an existing device, or stored when these systems are manufactured.

[0024] A non-transitory computer-readable storage medium stores at least one software code portion that, when executed or processed by a computer, is configured to perform executable operations for obtaining plant images.

[0025] The executable operations include obtaining multiple images of a plant, each of the images being associated with a different capture moment, storing multiple of the multiple images of the plant together with a planting plan for planting the plant, selecting the planting plan separately from the plant, and presenting the multiple images when the planting plan is selected, each of the multiple images being presented together with one or more desired and / or measured conditions for a planting stage corresponding to the capture moment of the corresponding image, and the planting plan including multiple planting stages, and the one or more desired and / or measured conditions being included in the planting plan.

[0026] The executable operations further include determining sensor coverage areas of the multiple presence sensor devices based on the sensor position, the sensor orientation, and the sensor field of view, including gaps in the sensor coverage areas, determining one or more parameters for presence detection based on the gaps in the sensor coverage areas, and outputting the one or more parameters or presence detection results that have been determined using the one or more parameters.

[0027] As will be appreciated by those skilled in the art, aspects of the present invention may be embodied as devices, methods or computer program products. Thus, aspects of the present invention may take the form of a complete hardware embodiment, a complete software embodiment (including firmware, resident software, microcode, etc.) or an embodiment combining software and hardware aspects, all of which may be collectively referred to herein as "circuits," "modules," or "systems." The functions described in this disclosure may be implemented as algorithms executed by a processor / microprocessor of a computer. In addition, aspects of the present invention may take the form of a computer program product embodied in one or more computer-readable media having computer-readable program code embodied thereon (e.g., stored thereon).

[0028] Any combination of one or more computer-readable media can be utilized.Computer-readable media can be a computer-readable signal medium or a computer-readable storage medium.Computer-readable storage media can be, for example, but not limited to, electronic, magnetic, optical, electromagnetic, infrared or semiconductor systems, devices or equipment, or any suitable combination of the foregoing.More specific examples of computer-readable storage media can include, but are not limited to, the following: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.In the context of the present invention, a computer-readable storage medium can be any tangible medium that can contain or store a program used by an instruction execution system, device or equipment or used in combination with an instruction execution system, device or equipment.

[0029] A computer-readable signal medium may include a propagated data signal in which computer-readable program code is embodied (e.g., in baseband or as part of a carrier wave). Such a propagated signal may take any of a variety of forms, including but not limited to electromagnetic, optical, or any suitable combination thereof. A computer-readable signal medium may be any computer-readable medium that is not a computer-readable storage medium and that can transmit, propagate, or transport a program for use by or in conjunction with an instruction execution system, apparatus, or device.

[0030] The program code embodied in the computer-readable medium can use any suitable medium (including but not limited to wireless, wired, optical fiber, cable, RF etc., or aforesaid any suitable combination) to transmit.The computer program code for realizing the operation of various aspects of the present invention can be written with any combination of one or more programming languages, and described one or more programming languages ​​comprise object-oriented programming languages ​​(such as Java (TM), Smalltalk, C++ etc.) and conventional process programming languages ​​(such as " C " programming language or similar programming languages).Program code can be executed on user's computer completely, as independent software package partly executed on user's computer, partly executed on user's computer and partly executed on remote computer or executed completely on remote computer or server.In latter scenario, remote computer can be connected to user's computer by the network of any type comprising local area network (LAN) or wide area network (WAN), or can be connected (for example, by using the Internet of Internet Service Provider) with external computer.

[0031] Aspects of the present invention are described below with reference to flowchart illustrations and / or block diagrams of methods, devices (systems) and computer program products according to embodiments of the present invention. It will be understood that each frame of the flowchart illustrations and / or block diagrams and the combination of frames in the flowchart illustrations and / or block diagrams can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer or other programmable data processing device, particularly a microprocessor or a central processing unit (CPU) to produce a machine so that instructions executed via the processor of the computer, other programmable data processing devices or other equipment create a component for implementing the function / action specified in one or more frames of the flowchart and / or block diagram.

[0032] These computer program instructions may also be stored in a computer-readable medium that can direct a computer, other programmable data processing apparatus, or other device to operate in a specific manner so that the instructions stored in the computer-readable medium produce an article of manufacture including instructions for implementing the functions / actions specified in one or more blocks of the flowchart and / or block diagram.

[0033] The computer program instructions may also be loaded onto a computer, other programmable data processing apparatus or other devices to cause a series of operating steps to be performed on the computer, other programmable apparatus or other devices to produce a computer-implemented process, so that the instructions executed on the computer or other programmable apparatus provide a process for implementing the functions / actions specified in one or more boxes of the flowchart and / or block diagram.

[0034] The flow chart and block diagram in each figure illustrate the possible architecture, functionality and operation of the device, method and computer program product according to various embodiments of the present invention.In this regard, each frame in the flow chart or block diagram can represent a module, code segment or code portion, which includes one or more executable instructions for realizing (one or more) specified logical functions.It should also be noted that in some alternative implementations, the function marked in the frame may not appear in the order marked in each figure.For example, the two frames shown in succession can actually be performed substantially simultaneously, or these frames can sometimes be performed in reverse order, depending on the functionality involved.It will also be noted that the combination of each frame in the block diagram and / or flow chart diagram and the frame in the block diagram and / or flow chart diagram can be realized by a system based on special-purpose hardware or a combination of special-purpose hardware and computer instructions that performs a specified function or action. BRIEF DESCRIPTION OF THE DRAWINGS

[0035] These and other aspects of the invention are apparent from and will be further elucidated, by way of example, with reference to the accompanying drawings, in which:

[0036] Figure 1 is a block diagram of a first embodiment of the system;

[0037] Figure 2 is a block diagram of a second embodiment of the system;

[0038] Figure 3 A first screen of an example user interface for managing plant cultivation is shown;

[0039] Figure 4 Shown Figure 3 A second screen of an example user interface;

[0040] Figure 5 is a flow chart of a first embodiment of the method;

[0041] Figure 6 is a flow chart of a second embodiment of the method;

[0042] Figure 7 is a flowchart of a third embodiment of the method;

[0043] Figure 8 is a flowchart of a fourth embodiment of the method;

[0044] Figure 9 shows plants moving slowly on a conveyor belt as the image is being captured; and

[0045] Figure 10 is a block diagram of an exemplary data processing system for executing the method of the present invention.

[0046] Corresponding elements in the drawings are marked with the same reference numerals. DETAILED DESCRIPTION

[0047] Figure 1 A first embodiment of a system for obtaining plant images is shown. Figure 1 In the example of FIG, plants are grown in a vertical farm 51 having three layers 53-55, and images of the plants are captured using cameras 41-43. Each of the layers 53-55 typically includes a plurality of segments (not shown). Figure 1 In the embodiment of FIG. 4 , the system is a mobile device 1. The mobile device 1 is connected to the Internet 11, for example, via a wireless LAN access point or a cellular communication network. An Internet server 13 is also connected to the Internet 11.

[0048] The mobile device 1 comprises a receiver 3, a transmitter 4, a processor 5, a memory 7 and a display 9. The processor 5 is configured to obtain a plurality of images of a plant using the receiver 3 and store a plurality of the plurality of images of the plant together with a planting plan (also called a planting recipe) for planting the plant on, for example, a storage member 27 or an internet server 13. Each image is associated with a different capture moment.

[0049] The cameras 41-43 can control the capture and storage of images. For example, they can retrieve a planting plan identifier and an identifier for the current planting stage and store the image together with these identifiers in the planting plan. Alternatively, for example, the cameras 41-43 can receive a planting plan activation command from the mobile device 1 or a gardening system (not shown). For example, the cameras 41-43 can detect a control command indicating activation of the planting plan. Based on the properties of the planting plan, the cameras 41-43 can determine or retrieve a corresponding image capture schedule (e.g., a predefined time interval for the duration of the plan). The gardening system can control one or more of lighting, climate, and nutrient distribution.

[0050] The processor 5 is further configured to select a planting plan separately from the plant and to present a plurality of images using the display 9 when the planting plan is selected. For example, the planting plan can be selected directly or via different plants to which the planting plan is applied. Each of the plurality of images is presented together with one or more desired and / or measured conditions for the planting stage. The planting stage corresponds to the moment of capture of the corresponding image. The planting plan (also called a planting recipe) includes a plurality of planting stages, and one or more desired and / or measured conditions are included in the planting plan.

[0051] One or more desired and / or measured conditions typically include lighting conditions and / or climatic conditions and / or nutritional conditions. Nutrition typically includes fertilization and water. Light recipes typically include thresholds, daylight measurements and / or control parameters, supplementary light levels, and supplementary spectrums. Plants typically require 5 to 10 hours of darkness / sleep. Planting is preferably done during the day because artificial light is relatively expensive. Each planting stage is typically a period in which the planting scheme / recipe remains unchanged (e.g., the same light schedule, irrigation schedule, plant density). For example, each planting stage can have a duration of one day, but alternatively, each planting stage can have a duration different from one day, and different planting stages may even have different durations.

[0052] exist Figure 1 In the embodiment of the mobile device 1 shown in FIG, the mobile device 1 includes a processor 5. In alternative embodiments, the mobile device 1 includes multiple processors. The processor 5 of the mobile device 1 can be a general-purpose processor, such as a general-purpose processor from ARM or Qualcomm, or a dedicated processor. The processor 5 of the mobile device 1 can run, for example, an Android or iOS operating system. The display 9 can include, for example, an LCD or OLED display panel. The memory 7 can include one or more memory units. For example, the memory 7 can include solid-state memory.

[0053] For example, the receiver 3 and transmitter 4 may use one or more wireless communication technologies, such as Wi-Fi (IEEE 802.11), to communicate with an access point to the Internet 11. In alternative embodiments, multiple receivers and / or multiple transmitters are used instead of a single receiver and a single transmitter. Figure 1 In the embodiment shown in , a separate receiver and a separate transmitter are used. In an alternative embodiment, the receiver 3 and the transmitter 4 are combined into a transceiver. The mobile device 1 may include other components typical for mobile devices, such as a battery and a power connector. The present invention may be implemented using a computer program running on one or more processors.

[0054] exist Figure 1 In an embodiment, the system is a mobile device. In an alternative embodiment, the system of the present invention is a different device, such as a computer. Figure 1 In an alternative embodiment, the system of the present invention comprises a plurality of devices.

[0055] Figure 2 A second embodiment of a system for obtaining plant images is shown. Figure 2 In the embodiment of FIG. 1 , the system is a computer 21. Computer 21 is connected to the Internet 11 and acts as a server. Computer 21 includes a receiver 23, a transmitter 24, a processor 25, and a storage component 27. Processor 25 is configured to use receiver 23 to obtain multiple images of a plant from cameras 41-43 and store multiple of the multiple images of the plant together with a planting plan for planting the plant, for example, on storage component 27. Each image is associated with a different capture moment.

[0056] The processor 25 is further configured to select a planting plan separately from the plants and, when selecting the planting plan, to present a plurality of images via the personal computer 17 and the monitor 19 connected thereto (e.g., via a web / html interface) using the transmitter 24. Each of the plurality of images is presented together with one or more desired and / or measured conditions for a planting stage. The planting stage corresponds to the moment of capture of the corresponding image. The planting plan includes a plurality of planting stages, and the one or more desired and / or measured conditions are included in the planting plan. Figure 2 In an embodiment of the present invention, the synchronization between the presentation of the desired and / or measured conditions and the presentation of the image is performed by the processor 25.

[0057] exist Figure 2In the embodiment of the computer 21 shown in FIG, the computer 21 includes a processor 25. In alternative embodiments, the computer 21 includes multiple processors. The processor 25 of the computer 21 can be a general-purpose processor, such as a processor from Intel or AMD, or a special-purpose processor. The processor 25 of the computer 21 can run an operating system based on, for example, Windows or Unix. The storage component 27 can include one or more memory units. For example, the storage component 27 can include one or more hard disks and / or solid-state memories. The storage component 27 can be used to store, for example, the operating system, applications, and application data.

[0058] For example, the receiver 23 and transmitter 24 may use one or more wired and / or wireless communication technologies, such as Ethernet and / or Wi-Fi (IEEE 802.11), to communicate with an access point to the Internet 11. In alternative embodiments, multiple receivers and / or multiple transmitters are used instead of a single receiver and a single transmitter. Figure 2 In the embodiment shown in , a separate receiver and a separate transmitter are used. In an alternative embodiment, the receiver 23 and the transmitter 24 are combined into a transceiver. The computer 21 may include other components typical for a computer, such as a power connector and a display. The present invention may be implemented using a computer program running on one or more processors.

[0059] The method of the present invention generally involves capturing multiple images of a plant over time, associating them with their corresponding planting stages, and combining them to present a planting plan representation on a display. The resulting representation is used to represent individual planting plans in a large database of planting plans. Thus, images captured during an active planting plan are used to represent individual planting plans in the database. Those representative images can help growers see what results can be expected when a planting plan is applied to a given plant species.

[0060] Possibly, intelligence or an algorithm is used to select the images that are most representative of the planting plan. A time-lapse recording of the planting of plants can be presented using a plurality of captured images to represent the planting plan, i.e. the plurality of images are presented as a video sequence in which the images are presented in order of the elapsed planting time. However, it is also possible to present a timeline representation representing the current planting plan stage, which is enriched with images representing the current planting plan state and enables the user to scroll through images representing earlier planting stages, e.g. Figure 3 As shown in .

[0061] exist Figure 3In the first screen 81 of the user interface shown in FIG, the planting plan timeline 72 is annotated with an image 83 representing the current state of the plant. The user interface is presented on the display 19. Desired and / or measured conditions 74-76 are shown on the right side of the user interface and include light (spectrum) conditions 74, nutrient conditions 75, and climate conditions 76. Figure 3 In the example of , the climate condition 76 includes CO2 and temperature conditions.

[0062] exist Figure 3 In the example shown, the user has selected an individual plant, or a group of individual plants at the same planting stage, and is now viewing a representation of the activated planting plan, as identified by label 71. Image 83 has been stored in the planting plan and is typically a photograph of a plant that was planted in the past. Image 83 is presented with label 84 ("Day 8") corresponding to the planting stage that the image 83 was captured at. Figure 3 The current growing stage of the individual plant selected in .

[0063] It is also possible for the user to scroll back in time to view earlier planting states. Figure 4 As shown in Figure 4 Depicts Figure 3 The second screen 91 of the user interface of the present invention presents the image 93 with a label 94 ("Day 3") of the planting stage corresponding to the moment of capture of the image 93. Figure 4 In the example, image 93 is Figure 3 The images 83 are photographs of the same plants, usually representative plants from past plantings. The desired and / or measured conditions 74-76 are updated accordingly, but this is not done in Figure 3 and Figure 4 Shown in.

[0064] Therefore, the presentation Figure 3 and Figure 4 The system of the user interface receives user input including user commands for navigating through the planting stages and selects one or more images to be presented next from the plurality of images based on the user commands. Alternatively, the user may be able to scroll back in time to check what the appearance of the selected individual plant was one or more days ago.

[0065] Not only can images of the current and previous planting stages be displayed, but it is also possible to move the slider into the future, since the images presented are usually images of plants planted in the past.

[0066] Figure 5A first embodiment of a method for obtaining plant images is shown in FIG. Step 101 comprises obtaining multiple images of a plant. Each image is associated with a different capture moment. Step 111 comprises selecting a representative subset of the multiple images obtained, for example by selecting images captured at noon each day or by using a more complex algorithm. Step 103 comprises storing the representative images selected in step 111 together with a planting plan for planting the plant.

[0067] Step 105 includes selecting a planting plan separately from the plant. Step 107 includes presenting a plurality of images when the planting plan is selected. Each image is presented along with one or more desired and / or measured conditions for the planting stage corresponding to the moment of capture of the respective image. The planting plan includes a plurality of planting stages, and the one or more desired and / or measured conditions are included in the planting plan.

[0068] Figure 6 A second embodiment of the method of obtaining plant images is shown in . Step 121 includes transmitting a capture schedule for capturing a plurality of plant images to one or more cameras. Next, step 101 includes receiving a plurality of images of plants from one or more cameras.

[0069] Since in this embodiment only requested images are received, there is no need to select a representative subset, and Figure 5 Step 111 has been omitted. In an alternative embodiment, step 111 may also be omitted, where no capture schedule is transmitted, but one or more cameras are controlled to capture multiple images at different capture times. After step 101, execution Figure 5 Steps 103, 105 and 107.

[0070] Figure 7 A third embodiment of the method for obtaining a plant image is shown in FIG. First, steps 101 to 107 are performed, see for example Figure 5 Step 101 includes obtaining a plurality of images of a plant. Step 103 includes storing the obtained images together with a planting plan for planting the plant. Step 105 includes selecting a planting plan separately from the plant by selecting a plant species associated with the planting plan. Figure 5 , step 107 includes presenting a plurality of images when selecting a planting plan.

[0071] Steps 131 to 139 are performed later. Step 131 comprises receiving user input identifying a further plant (ie an individual plant) and a planting plan for that plant. Figure 7In an embodiment, step 133 is then performed. Step 133 includes transmitting a capture schedule to one or more cameras for capturing images of the identified additional plants. In alternative embodiments, the images are selected from a collection of received images, or the one or more cameras are remotely controlled to capture images at certain times.

[0072] Step 135 is performed after step 133. Step 135 includes receiving the requested image of the identified additional plant from one or more cameras. Step 137 includes determining a difference between the obtained image and a plurality of images stored with the identified planting plan. Step 139 includes providing an alert if the difference is determined to exceed a predetermined threshold. For example, the grower may receive an alert on a mobile or fixed display indicating that a planting deviation has been detected, including one or more captured images showing the most recent or current status.

[0073] Steps 137 and 139 can be implemented, for example, using a trained deep learning network (e.g., a neural network). For example, the deep learning network can determine whether an alarm should be generated based on two input images (of a plant in the same planting stage and another plant).

[0074] Figure 8 A fourth embodiment of a method for obtaining plant images is shown in FIG. Step 151 includes receiving user input including a camera or location identifier and information identifying a planting plan. The planting plan includes a plurality of planting stages, and one or more desired and / or measured conditions are included in the planting plan. Figure 8 In some embodiments, the input is provided by the user. For example, when a grower activates a planting plan for a new individual plant, they may be able to indicate that they wish an image of that plant to be captured and stored with the planting plan. In alternative embodiments, the input is provided by the system, and this can be initiated, for example, based on a schedule or upon detecting the arrival of a particular type of plant or tray.

[0075] A planting plan typically includes at least a lighting recipe and optionally further includes a schedule of climate and nutrient conditions. A location identifier can indicate at which section (or device) of the horticultural system the planting plan is activated. Based on the location at which the planting plan is activated, a co-located camera device can be determined to be directed toward the plants planted under the planting plan. In a possible implementation, the camera device is associated with one or more lighting devices.

[0076] It is also possible to integrate the camera as part of the planting lighting system. Multiple camera identifiers and / or location identifiers can be associated with the planting plan. This can be beneficial because plants are often replanted after a certain period of time, such as after germination. A camera or location identifier can be associated with a particular planting stage or a sequence of planting stages. Step 153 includes storing the camera or location identifier with the planting plan.

[0077] The camera(s) determined based on the user input in step 151 are controlled to capture images of the plant(s) receiving the planting plan. In one implementation, the horticultural system sends regular control commands to the determined camera(s). Instead of sending multiple camera control commands, the horticultural system may (e.g., upon activating the planting plan) send a schedule to the camera device specifying the time points at which images should be captured. Alternatively, the camera may capture images continuously or frequently (e.g., daily).

[0078] Step 155 (which is somewhat similar to Figure 5-Figure 7 Step 101 of the present invention includes obtaining a collection of images of a plurality of plants. These images are captured using the camera(s) determined in step 151 based on user input. Step 157 includes selecting a plurality of images from the collection of images based on camera or location identifiers stored with the planting plan.

[0079] Step 159 (which is somewhat similar to Figure 5-Figure 7 Step 103) includes storing a plurality of images of the plant selected in step 157 along with the planting plan. The captured images may be stored, for example, at a camera device, a horticultural system, or a horticultural lighting system. The images are stored in such a manner that the corresponding planting plan and planting stage for each image can be determined.

[0080] For example, this can be achieved by storing location and timestamp information for each image. In an alternative implementation, each image can be annotated with data related to the active planting plan, such as its identifier or planting stage. In addition, it can be useful to store camera identifiers and / or camera locations. Camera identifiers help combine images from a single camera to generate a time-based representation of the plant's state over time, such as a time-lapse recording.

[0081] exist Figure 8 In the embodiment of the present invention, steps 157 and 159 are only performed for the planting plan identified by the information received in step 151. However, in general, for each acquired image associated with a camera or location identifier, a planting plan with that camera or location identifier will be searched for, and thus steps 157 and 159 will be performed for each matching planting plan.

[0082] Step 105 includes selecting a planting plan separately from the plants. For example, the user selects a plant species associated with the planting plan and then selects the planting plan from a list of planting plans. Step 107 includes presenting a plurality of images upon selecting the planting plan. Each image is presented along with one or more desired and / or measured conditions for the planting stage corresponding to the moment the respective image was captured.

[0083] Figure 9 A plant 203 is shown slowly moving on conveyor belt 201 while images are being captured. In this situation, the fifth embodiment of the method for obtaining plant images can be advantageously used. In this fifth embodiment, the current position of plant 203 relative to cameras 45-47 is determined, and cameras 45-47 are controlled to capture images at a time dependent on the current position. As plant 203 moves, an appropriate camera 45-47 is selected based on the current position so that the plant is within the camera's field of view.

[0084] When it is determined or expected that the plant 203 is at the appropriate distance, the appropriate camera is controlled to capture an image. For example, the current position of the plant can be determined using image recognition, or can be calculated based on the time the plant was placed on the conveyor and the speed of the conveyor. For example, there may be three or X times three predefined positions on the conveyor 201 (one or X for each camera) at which images are captured.

[0085] Figure 10 Depicted diagram can be executed as reference Figures 5 to 8 A block diagram of an exemplary data processing system for describing the method.

[0086] like Figure 10 As shown in , data processing system 300 may include at least one processor 302 coupled to memory element 304 via system bus 306. Thus, the data processing system may store program code within memory element 304. In addition, processor 302 may execute program code accessed from memory element 304 via system bus 306. In one aspect, the data processing system may be implemented as a computer suitable for storing and / or executing program code. However, it should be appreciated that data processing system 300 may be implemented in the form of any system including a processor and memory capable of performing the functions described in this specification.

[0087] Memory element 304 may include one or more physical memory devices, such as, for example, local memory 308 and one or more mass storage devices 310. Local memory may refer to random access memory or (one or more) other non-permanent memory devices generally used during the actual execution of program code. Mass storage devices may be implemented as hard drives or other permanent data storage devices. Processing system 300 may also include one or more cache memories (not shown), which provide temporary storage of at least some program codes, so as to reduce the number of times program codes must be retrieved from mass storage devices 310 during execution. For example, if processing system 300 is a part of a cloud computing platform, processing system 300 may also be able to use the memory element of another processing system.

[0088] Input / output (I / O) devices, depicted as input device 312 and output device 314, may optionally be coupled to the data processing system. Examples of input devices may include, but are not limited to, a keyboard, a pointing device such as a mouse, a microphone (e.g., for voice and / or speech recognition), etc. Examples of output devices may include, but are not limited to, a monitor or display, speakers, etc. Input and / or output devices may be coupled to the data processing system directly or through an intervening I / O controller.

[0089] In one embodiment, the input and output devices may be implemented as a combined input / output device (in Figure 10 314). An example of such a combination device is a touch-sensitive display, sometimes also referred to as a "touch screen display" or simply a "touch screen." In such an embodiment, input to the device can be provided by moving a physical object (such as, for example, a stylus or finger of a user) across or near the touch screen display.

[0090] A network adapter 316 may also be coupled to the data processing system to enable it to become coupled to other systems, computer systems, remote network devices, and / or remote storage devices through intervening private or public networks. The network adapter may include a data receiver for receiving data transmitted to the data processing system 300 by the system, device, and / or network, and a data transmitter for transmitting data from the data processing system 300 to the system, device, and / or network. Modems, cable modems, and Ethernet cards are examples of different types of network adapters that may be used with data processing system 300.

[0091] like Figure 10As depicted in FIG, memory element 304 may store applications 318. In various embodiments, applications 318 may be stored in local memory 308, one or more mass storage devices 310, or separate from local memory and mass storage devices. It should be appreciated that data processing system 300 may further execute an operating system ( Figure 10 318. Application 318, implemented in the form of executable program code, may be executed by data processing system 300, for example, by processor 302. In response to executing the application, data processing system 300 may be configured to perform one or more operations or method steps described herein.

[0092] Various embodiments of the present invention can be implemented as a program product for use with a computer system, wherein the program(s) of the program product define the functionality of the embodiments (including the methods described herein). In one embodiment, the program(s) may be contained on various non-transitory computer-readable storage media, wherein, as used herein, the expression "non-transitory computer-readable storage media" includes all computer-readable media with the sole exception of temporary propagation signals. In another embodiment, the program(s) may be contained on various temporary computer-readable storage media. Illustrative computer-readable storage media include, but are not limited to: (i) non-writable storage media on which information is permanently stored (e.g., a read-only memory device within a computer, such as a CD-ROM disk readable by a CD-ROM drive, a ROM chip, or any type of solid-state non-volatile semiconductor memory); and (ii) writable storage media on which variable information is stored (e.g., flash memory, a floppy disk within a disk drive or a hard disk drive, or any type of solid-state random access semiconductor memory). The computer program may be executed on the processor 302 described herein.

[0093] The terminology used herein is for the purpose of describing particular embodiments only and is not intended to limit the present invention. As used herein, the singular forms "a," "an," and "the" are intended to include the plural forms as well, unless the context clearly indicates otherwise. It will be further understood that the terms "comprises" and / or "comprising," when used in this specification, specify the presence of stated features, integers, steps, operations, elements, and / or components, but do not preclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and / or combinations thereof.

[0094] The corresponding structures, materials, acts, and equivalents of all means or step plus function elements in the claims below are intended to include any structure, material, or act for performing the function in combination with other claimed elements as specifically claimed. The description of the embodiments of the invention is presented for illustrative purposes but is not intended to be exhaustive or limited to implementation in the form disclosed. Many modifications and variations will be apparent to those of ordinary skill in the art without departing from the scope and spirit of the invention. The embodiments are selected and described in order to best explain the principles of the invention and some practical applications, and to enable others of ordinary skill in the art to understand the invention in various embodiments with various modifications as are suitable for the particular uses envisioned.

Claims

1. A system (1, 21) for obtaining plant images, the system (1, 21) comprising: at least one input interface (3, 23); at least one output interface (9, 24); and At least one processor (5, 25) configured to: using the at least one input interface (3, 23) to obtain a plurality of images of the plant, each of the images being associated with a different capture moment, storing a plurality of the plurality of images of the plant together with a planting plan for planting the plant, so as to augment the planting plan with images of the plant at different planting stages in the planting plan, the different planting stages corresponding to the different capture moments of the respective images, selecting the planting scheme separately from the plants, and When the implantation plan is selected, the plurality of images are presented using the at least one output interface (9, 24), each of the plurality of images being presented together with one or more desired and / or measured conditions for the implantation stage, the one or more desired and / or measured conditions being included in the implantation plan.

2. The system (1, 21) according to claim 1, wherein the at least one processor (5, 25) is configured to present the plurality of images as a video sequence, the images being in order of the planting time elapsed in the video sequence.

3. A system (1, 21) according to claim 1, wherein the at least one processor (5, 25) is configured to use the at least one input interface (3, 23) to receive user input including user commands for navigating through the implant stages, and select one or more images to be presented next from the multiple images based on the user commands.

4. System (1, 21) according to claim 1 or 2, wherein the one or more desired and / or measured conditions include lighting conditions and / or climate conditions and / or nutritional conditions.

5. The system (1, 21) according to claim 1 or 2, wherein the at least one processor (5, 25) is configured to select a representative subset of the plurality of obtained images as the plurality of images before storing the plurality of images of the plant with the planting plan.

6. The system (1, 21) according to claim 1 or 2, wherein the at least one processor (5, 25) is configured to: receiving user input using the at least one input interface (3, 23), the user input identifying additional plants and the planting plan, obtaining an image of the identified additional plant, determining a difference between the acquired image and a plurality of images stored with the identified planting plan, and If the difference is determined to exceed a predetermined threshold, an alert is provided using the at least one output interface (9, 24).

7. The system (1, 21) according to claim 6, wherein the at least one processor (5, 25) is configured to transmit a capture schedule for capturing the images of the identified additional plants to one or more cameras (41-43) using the at least one output interface (4, 24).

8. The system (1, 21) according to claim 1 or 2, wherein the at least one processor (5, 25) is configured to: receiving user input using the at least one input interface (3, 23), the user input comprising a camera or location identifier and information for identifying the planting plan, storing the camera or location identifier along with the planting plan, obtaining a set of images of a plurality of plants using the at least one input interface (3, 23), the set comprising the plurality of images, selecting the plurality of images from the image collection based on the camera or location identifier, and The plurality of images of the plant are stored along with the planting plan.

9. The system (1, 21) according to claim 1 or 2, wherein the at least one processor (5, 25) is configured to control one or more cameras (41-43) to capture the plurality of images at the different capture moments using the at least one output interface (4, 24).

10. The system (1, 21) of claim 9, wherein the at least one processor (5, 25) is configured to transmit a capture schedule for capturing the plurality of images to the one or more cameras (41-43) using the at least one output interface (4, 24).

11. The system (1, 21) according to claim 9, wherein the at least one processor (5, 25) is configured to: using the at least one input interface (3, 23) to obtain a current position of the plant relative to the one or more cameras (45-47), and The at least one output interface (4, 24) is used to control the one or more cameras (45-47) to capture one of the images at a time dependent on the current position.

12. The system (1, 21) of claim 11, wherein the one or more cameras (45-47) comprises a plurality of cameras, and the at least one processor (5, 25) is configured to select one of the plurality of cameras based on the current location.

13. A method for obtaining a plant image, the method comprising: obtaining (101) a plurality of images of a plant, each of said images being associated with a different moment of capture, storing (103) a plurality of the plurality of images of the plant together with a planting plan for planting the plant, so as to augment the planting plan with images of the plant at different planting stages in the planting plan, the different planting stages corresponding to the different capture moments of the respective images; selecting (105) the planting plan separately from the plants, and The plurality of images are presented (107) when the implantation plan is selected, each of the plurality of images being presented together with one or more desired and / or measured conditions for the implantation stage, and the one or more desired and / or measured conditions being included in the implantation plan.

14. A computer program product storing at least one software code portion configured to enable the method of claim 13 to be performed when the software code portion is run on a processor according to any one of claims 1 to 12.

Images