Method and system for capturing images for an automation task

Abstract

The disclosure relates to an automated method for capturing second images (51) for an automation task by a second camera (5) based on first images (41) from a first camera (4). The disclosure further relates to a corresponding system (1) arranged to execute the method, a computer program, and a computer-readable storage medium.

Description

[0001] METHOD AND SYSTEM FOR CAPTURING IMAGES FOR AN AUTOMATION

[0002] TASK

[0003] Independent of the grammatical term usage, individuals with male, female or other gender identities are included within the term.

[0004] DESCRIPTION

[0005] Technical Field

[0006] The embodiments disclosed herein relate an automated method for capturing second images for an automation task. The disclosure further relates to a corresponding system arranged to execute the method, a computer program, and a computer-readable storage medium.

[0007] Background

[0008] Automation in manufacturing and production refers to using technology and machines to perform specific tasks of the manufacturing and production process without the need for humans to intervene. The goal of process automation is to increase efficiency, productivity, and accuracy in the production process, reducing manual labour and minimizing the risk of human error.

[0009] An area where process automation is used is controlled environment agriculture (CEA). Controlled environment agriculture is a modern approach to agriculture that is based on creating closed environments with tightly controlled conditions that optimize plant growth. This approach allows farmers to produce more food with consistent yields throughout the year, but in turn requires greater control over the growing conditions. Therefore, farmers must monitor the plants throughout the grow cycle and adjust process parameters such as irrigation frequency and light schedules to ensure optimal yields.

[0010] One way in which CEA farmers monitor their plants is to look at objective metrics that are indicators of yield or plant health. For example, looking at the height and diameter of a lettuce plant is a good indicator of whether the plant is accumulating biomass properly, and can provide an early indication of whether the plant is entering a growth stage such as bolting that is counterproductive, e.g. because it makes the plant more bitter or otherwise stops biomass accumulation in favour of seed generation. In such cases, farmers need tools to identify these indicators and take corrective action where possible.

[0011] Systems and methods to support and automate this process, and manufacturing and production processes in general are desirable.

[0012] Summary

[0013] In one embodiment, a method for capturing images for an automation task is disclosed. The method includes the step of receiving first image data corresponding to at least one first image. The at least one first image being captured by at least one first camera, wherein the at least one first camera belongs to (= is part of) a first sensor group, the at least one first image comprising a visualization of at least one first object, wherein the at least one first object is involved in (= used in / related to) a process of the automation task.

[0014] The method further includes the step of extracting (= determining) state data = condition data, indicator data; state data may include a state label and / or state tag) by analysing the first image data, the state data being indicative for (= describing / being dependent on / defining) a state (= condition) of the at least one first object in the process.

[0015] The method further includes the step of defining at least one rule based on (= depending on) the state data.

[0016] The method further includes the step of determining at least one control command based on (= depending on) the at least one rule, the at least one control command comprising at least one capturing command to capture at least one second image of the at least one first object and / or of at least one second object, wherein the at least one second object is involved in (used in / related to) the process.

[0017] The method further includes the step of sending the at least one capturing command to a second sensor group, the second sensor group comprising at least one second camera, the at least one capturing command triggering (= steering) the at least one second camera to capture the at least one second image. According to the embodiment, a second image is captured depending on the state of the object captured on the first image using a second camera (different to the first camera, which took the first image). The second image is again taken from the first object, but from a different perspective and / or from a second object, while the second object is involved in the same process the first object is. The second camara and the second object are selected depending on the defined rule. The rule is then transferred into the control command (comprising the capturing command of the second image).

[0018] Unlike traditional agriculture, where drone or satellite-based imaging systems are integrated with geographic information systems (GIS), controlled environment agriculture (CEA) requires imaging systems that can operate in tight quarters and oversee much more densely packed plants. In CEA, large-scale plant monitoring is required, which requires farmers to integrate many cameras spread across their farm. The data from these cameras must then be processed and analysed to derive useful metrics and actionable information.

[0019] To reach this goal the described embodiment enables a closed CEA loop that produces automation and image acquisition actions based on the image analysis performed on the Industrial Edge Device or in the cloud. This requires a system that can communicate with and control the cameras (-> programmable logic controllers (PLCs)) and acquire images according to a ruleset (and optionally schedule) that can optionally be modified by an engineer / user.

[0020] The method is executable by an edge device. The edge device is connectable to programmable logic controller(s), which are connected to cameras.

[0021] The state data is to be understood as being indicative for a current state and / or a predicted future state of the at least one first object.

[0022] The method comprises the step of “determining at least one control command based on (= depending on) the at least one rule”. This is to be understood that the at least one control command is at least based / depending on the at least one rule, possibly depending on further factors.

[0023] The control command is arranged as a capturing command or a steering command: to take another image or to steer robot to water the earth of the plant or an automation action such as a change to a PLC setpoint and not necessarily robot steering, respectively. According to a further embodiment a system arranged to execute the disclosed method is provided.

[0024] According to a further embodiment a computer program product comprising instructions which, when the program is executed by the computational device, cause the computational device to carry out the steps of the method according to the disclosed method is provided.

[0025] According to a further embodiment a computer-readable storage medium comprising instructions which, when executed by the computational device, cause the computational device to carry out the steps of the method according to the disclosed method is provided.

[0026] Brief Description of the Drawings

[0027] Fig. 1 shows a general architecture and dataflow / workflow to enable an automated method for capturing images for an automation task on an embodiment of Edge-based analysis and collection of crop images, and

[0028] Fig. 2 shows a system arranged to execute a method for capturing images for an automation task according to this disclosure.

[0029] Detailed Description

[0030] Fig. 1. Shows a general architecture and data / workflow to enable an automated method for capturing images for an automation task on an embodiment of Edge-based analysis and collection of crop images.

[0031] Scheduling and triggering are managed on the “Industrial Edge device” for all production lines (lines 1-3, each with a “Programmable Logic Controller” (PLC)) and for all cameras (“Line 1-3 Camera (GigE, RTSP, etc.”) (cameras are controlled via the PLC between the edge device and the cameras) based on user input (received via the “Web-Based User Interface”) and via an image / Al pipeline (“Preprocessing Pipeline” and “AFModel” blocks). The results are then fed back into the loop to enable scheduling / triggering decisions and reach the “Programmable Logic Controller” of “Lines 1 -3” again. The “Programmable Logic Controller” of “Lines 1-3” are also connected to a “TLA Portal” (Totally Integrated Automation Portal, also an engineering and simulation framework).

[0032] The “Industrial Edge device” includes a “Cloud Connector” with a “Northbound” and a “Southbound” connection via which it is connected to a “Cloud” storage. The “Northbound” connection sends or provides a “Ingestion Layer” to the “Cloud”. The “Ingestion Layer” is processed further through an ”AI pipeline” in the “Cloud” or is stored by a “Storage Service”. Both lead to “Output / Access Layer” which reached the “Industrial Edge device” again via the “Southbound” connection.

[0033] The “Industrial Edge device” further comprises a “Preprocessing Pipeline” which receives the “Output / Access Layer” via the “Southbound” connection and / or a “Acquisition layer” from at least one of the cameras (“Line 1-3 Camera (GigE, RTSP, etc.”). The “Preprocessing Pipeline” processes the received layer and sends the processed layer to the “AEModel”, which is managed by a “A [ / Model Management”. The “AI / Model” again is connected to all production lines (lines 1-3, each with a “Programmable Logic Controller” (PLC)) via the ’’Scheduling / Triggering” block. The “Scheduling / Triggering” block further receives triggering from the “Acquisition layer” and the “Preprocessing Pipeline” blocks.

[0034] The “Acquisition layer” from the at least one camera (“Line 1-3 Camera (GigE, RTSP, etc.”) is further provided to the “Northbound” connection.

[0035] “Data flow” is shown by a solid line, “Upload / Management” is shown by a dashed line and “Triggering” is show by a dotted line in Fig. 1.

[0036] The architecture in Fig. 1 creates a closed loop that automates image acquisition and analysis based on calendar / clock-based scheduling (“Scheduling” block part in Fig. 1) or based on data-driven triggering (“Triggering” block part in Fig. 1), i.e. when a certain result is returned by the “preprocessing pipeline” or the “AI / Model” block.

[0037] In general, a system shown in Fig. 1 works as follows:

[0038] Camera triggering is controlled by the “scheduling / triggering” block, which writes tags to the “programmable logic controllers” (PLCs) that in turn trigger the cameras’ acquisition systems (e.g. by sending a HIGH pulse to an Input / Output pin on the camera). The user is able to tell the “scheduling / triggering” when it should perform this triggering operation (e.g. every two hours, or whenever a plant requires further analysis), so this “scheduling / triggering” block is both time-based and rule-based. Once the camera is triggered, the image is collected on an “Industrial Edge devices” by an application such as the Industrial Edge Vision Connector that acts as the “acquisition layer”. This application handles the generation of a packet that includes the image, any metadata such as the location of the camera or plant, and any other relevant information such as a timestamp. Once this packet is generated, the information is transmitted to the other blocks in a way that allows the image to be correlated to the metadata, etc.

[0039] The data is optionally sent to a “cloud” server, where it can be stored or analyzed. On the Edge device, the image and metadata are sent to a “pre-processing” script or application, where it can be split, modified or prepared for use by the “AEmodel” block. Optionally, the “pre-processing” block can also send the “triggering / scheduling” block signals or otherwise trigger alerts / warnings.

[0040] The “AEmodel”, which was uploaded and managed by the user / engineer / grower using the “A [ / model Management” block, is then used to analyze the images that have been pre- processed in the previous block. The “AEmodel” generates results that are then written back into the Databus (a data layer that allows fast communication between blocks inside the Industrial Edge Device), which allows any number of automation actions (e.g. anomaly detection, or writeback to a programmable logic controller, or integration with HMI, etc) based on the automation engineer’s needs. The main goal of this block is to generate actionable data and metrics that the grower can use to then feed back into the automation / acquisition loop.

[0041] This results in a flexible, user-defined workflow that can scale to the requirements of the use case. For example, a grower operating multiple grow cells in the same facility might opt to control acquisition for the cells through a single Edge device, which then routes trigger requests through programmable logic controllers installed in each cell.

[0042] A main feature and further benefit of the architecture of the embodiment in Fig. 1 is traceable data: Metadata with exact image timestamp and location is attached to the image, and is carried throughout the processing pipeline. Being able to correlate processing / analysis results to certain plants or zones is an important part of then being able to take appropriate automation / control actions to address any issues. Generating metadata and attaching / correlating this metadata to the images acquired by the “Industrial Edge Devices” is therefore a necessary step to ensure traceable, actionable data and analysis. Furthermore, recording and storing this data for future use allows engineers and data scientists to investigate trends, root causes, and potential prevention measures. Traceable data is also beneficial to growers as it creates a verifiable paper trail for the growth cycle of each harvested plant. Further, data management is improved. By this, images that are collected from each camera can be accurately mapped to individual plants using camera and processing metadata, and this metadata can be carried through to storage and automation. On the storage side, this allows data scientists to pinpoint issues and correlate them to environmental variables such as temperature and humidity, e.g. data scientists can look at historical data stored in the cloud for a specific plant and track the effects of irrigation, ventilation, etc. on its growth and biomass accumulation. On the automation side, better data management allows engineers to devise automation rules that affect only the relevant plants, e.g., if a certain image indicates that the plant is not getting enough water, the engineer can design automation rules to increase irrigation to the affected region based on the metadata without affecting other regions which may be irrigated optimally already.

[0043] Further features and further benefits of the architecture of the embodiment in Fig. 1 are scalability and modularity: The architecture can be implemented once, and that implementation can be copied over to new farms and facilities, or extended to new zones within the same farm. Industrial Edge is based on the concept of app-based analysis and automation. This means that the end user (in this case, engineers, or integrators) should be able to pick which applications (i.e. which blocks in the architecture) are used in their farms. With the proposed architecture, this modularity is accomplished by splitting the architectural blocks into separate apps that can be enabled or disabled, or configured and modified, based on the needs of the farm. For example, Al-based analysis might not be necessary for newly sprouted lettuce, whereas further growth stages might require cloud- based analysis. The proposed architecture allows control over what modules are used where and when. The network-based architecture allows engineers and integrators to add more zones and cameras to the system as the farm grows. This can be done by keeping the Edge hardware fixed but adding cameras and programmable logic controllers to cover the new zones, or by also adding more Edge devices to increase parallel processing capabilities for the system. This also works for multiple facilities, where the hardware and network infrastructure can be replicated to provide scalability benefits to CEA companies that want to expand to new regions or markets. Additionally, since all devices are networked, the engineers can employ traditional IT tools such as switch / router management to partition the zones (e.g. to manage and direct traffic) or established cybersecurity practices to secure the data across the whole farm. Further, the area that is monitored can be scaled easily by installing additional cameras and replicating the existing programmable logic controller and Industrial Edge Device setup. This also gives the farmer flexibility in how they want to manage their grow zones; multiple zones can be monitored from a single device, or individual zones can be split up across multiple devices. This also allows better control over who has access to data from which zones, since users can be locked out of some devices but given access to others as needed.

[0044] Further features and further benefits of the architecture of the embodiment in Fig. 1 are automated scheduling and triggering: Images are captured by event-based triggering (according to the rules, e.g. when some process parameter is above a threshold), and optionally according to a set schedule. The system accesses and controls cameras based on automation needs, and the user has full control over what triggers image acquisition and analysis. The user can select to perform pre-processing / simple image processing only on certain images, and can then prescribe conditions that trigger Al-based analysis. The results from these steps can in turn feed back into rule-based acquisition. In many CEA applications, imaging data benefits from being spatially dense (covers many plants at high resolution) but temporally sparse (data points can be taken every couple of hours instead of every couple of seconds), since plant growth is typically only noticeable over longer periods (i.e., images of the same plant taken a couple of seconds apart will not be appreciably different). As such, growers may want to schedule image acquisition based on their crop varietal (e.g. synced with the daily cycle of the plant / based on metabolic / circadian rhythms). This has the additional benefit of allowing the Industrial Edge Device to put cameras into standby mode when acquisition is not required. This has immediate energy-saving benefits as well, since energy-intensive operations like camera acquisition and GPU-based analysis only happen when needed. For most CEA applications, this reduces total energy consumption for the imaging system (since cameras are switched from “low-power / standby” mode to “on” mode for less than a second during acquisition). Compared to other systems where the camera might continually produce images, this system is therefore more energy-efficient.

[0045] Further features and further benefits of the architecture of the embodiment in Fig. 1 are in the pre-processing and filtering: Images are pre-processed on the Edge device to reduce the required computation resources for more complex analysis (e.g. Al-based disease detection). Image pre-processing and filtering allows the growers to prepare, filter or sort the raw camera images into pre-processed images that can be passed on to the Al-based analysis pipeline or used to directly trigger further image acquisition. For example, growers can choose to only send “interesting” images to the Al pipeline, or select certain image properties (e.g. based on basic image processing results) for triggering. Additionally, basic image processing can trigger automation actions by itself without needing to go through the Al pipeline. For example, if basic image processing is enough to determine whether the plants are not as dense (i.e. the leaf area index is lower than expected) for a certain region, this can immediately trigger automation actions or alarms without waiting for the results of the Al pipeline.

[0046] A further feature and a further benefit of the architecture of the embodiment in Fig. 1 is a seamless Edge-based Al-enabled analysis: Apps such as the Al Inference Server allow users to manage their Al models and write the inference results from image streams back into the Databus (a data layer that allows fast communication between blocks inside the Industrial Edge Device) on their Industrial Edge device. In the proposed architecture, the inference would start automatically when the pre-processing and filtering step is completed, allowing users to create Al pipelines that fire automatically upon image acquisition and pre-processing. The user is then able to track and visualize the results from the pipeline, or install other offerings to tie the results into some other automation process (e.g. feed it back into the control loop).

[0047] A further feature and a further benefit of the architecture of the embodiment in Fig. 1 is found in the cloud analysis: Seamless integration with cloud-based solutions based on customer needs. Many CEA growers have additionally requested cloud integration for the Edge devices to then move the images and Al inference results into long-term storage, e.g. in their cloud storage, or to then send the data to their cloud-based analytics systems. It therefore makes sense to build cloud connectivity into the Edge architecture to enable more powerful and flexible image analysis. Cloud integration also enables long-term storage options and retrospective analysis when new models or equations become available to the grower / data scientist.

[0048] Fig. 2 shows a system 1 (e.g. an edge device 1 for a production line) arranged to execute a method for capturing images for an automation task according to this disclosure together with entities (e.g. a PLC 2, a cloud storage 3, cameras 4, 5, sensors 6, 7, a user interface 8) with which the system 1 is interacting.

[0049] The system 1 comprises a receiving unit 11, arranged to receive first image data 41 corresponding to at least one first image 41, the at least one first image 41 being captured by at least one first camera 4, wherein the at least one first camera 4 belongs to a first sensor group 4, 6, the at least one first image 41 comprising a visualization of at least one first object 91, wherein the at least one first object 91 is involved in a process 9 of the automation task. The system 1 further comprises an extracting unit 12, arranged to extract state data 121 by analysing the first image data 41, the state data 121 being indicative for a state of the at least one first object 91 in the process 9.

[0050] The system 1 further comprises a definition unit 13, arranged to define at least one rule 131 based on the state data 121.

[0051] The system 1 further comprises a determining unit 14, arranged to determine at least one control command 141 based on the at least one rule 131, (e.g. transforming the rule 131 into a executable command 141 for the PLC 2,) the at least one control command 141 comprising at least one capturing command to capture at least one second image 51 of the at least one first object 91 and / or of at least one second object 92, wherein the at least one second object 92 is involved in the process 9.

[0052] The system 1 further comprises a sending unit 15, arranged to send the at least one capturing command to a second sensor group 5, 7 (the sending is especially done indirect by sending via the PLCs 2), the second sensor group 5, 7 comprising at least one second camera 5, the at least one capturing command triggering the at least one second camera 5 to capture the at least one second image 51, e.g. an image of the same object 91, but from a different angle 51 and / or from another object 92, which is part of the same process 9.

[0053] According to the shown embodiment the process is a controlled environment agriculture process. According to an alternative embodiment the process is arranged as a manufacturing process and / or an automated bakery process and / or a food industry process and / or a beverage industry process and / or an industrial process.

[0054] According to a further embodiment method executed by the system 1 comprises the further step of sending the at least one capturing command to the first sensor group 4, 6 (via the PLCs 2), the at least one capturing command triggering (= steering) the at least one first camera 4 to capture (a further instance of) the at least one second image (further instance of the second image not shown in Fig. 2).

[0055] According to a further embodiment the at least one control command 141 further comprises at least one steering command to execute at least one steering action 10, the at least one steering action steering the process 9 of the automation task, and / or taking a corrective action 10 of the process 9 of the automation task (e.g. aiming an adjustment), e.g. watering a plant. E.g. if the state data 121 shows that the plant is wilting, a rule 131 the in this case the plant needs water, could lead to a steering command 141 for irrigating the soil.

[0056] According to a further embodiment the at least one control command further comprises at least one sensing command to execute at least one sensing. The method comprising the further steps of sending the at least one sensing command to the first sensor group 4, 6 (via the PLCs 2), the first sensor group 4, 6 comprising at least one first sensor 6, wherein the at least one first sensor 6 is of a different type of sensor as the at least one first camera 4, (e.g. a vibration sensor and / or, a temperature sensor 6 and / or, a humidity sensor and / or, a sound sensor, . . . but no camera), the at least one sensing command triggering (= causing) the at least one first sensor 6 to execute at least one sensing,

[0057] In addition or as alternative the method comprising the further steps of sending the at least one sensing command to the second sensor group 5, 7 (via the PLCs 2), the second sensor group 5, 7 comprising at least one second sensor 7, wherein the at least one second sensor 7 is of a different type of sensor as the at least one second camera 5, the at least one sensing command triggering (= causing) the at least one second sensor 7 to execute at least one sensing.

[0058] According to a further embodiment the at least one sensing is performed on the at least one first object 91 and / or the at least one second object 92.

[0059] According to a further embodiment the first image data 41 being processed image data and / or raw image data and / or meta data of the at least one first image, especially a location of the camera, data of a plant, and any other relevant information such as a timestamp.

[0060] According to a further embodiment the at least one control command 141 is further determined based on (= depending on) a schedule 132, the schedule 132 defining at least one time step when the at least one control command 141 is to be determined.

[0061] According to a further embodiment a schedule 132 is used in addition to rule-based determination of 141 control command. The schedule 132 can be also defined by an Al, a model and / or a user input. According to this embodiment a calendar / clock-based scheduling defines when the at least one control command 141 is to be determined.

[0062] According to a further embodiment the state data 121 is extracted (= determined) and / or the at least one rule 131 is defined based on (= depending on) the state data 121. For this an image analysis algorithm 12, 13, and / or an artificial intelligence algorithm 12, 13, and / or a modelling algorithm 12, 13, and / or a user input via the user interface 8 is used.

[0063] This embodiment results in a data-driven triggering, i.e. a triggering when a certain result is returned by an image preprocessing pipeline, by an Artificial intelligence (Al) and / or by Modelling which analyses the at least one first image data 41 to extract the state data 121 and / or to define the rule 131.

[0064] According to the shown embodiment the steps comprised by the method are executed by an edge device 1 , wherein the at least one capturing command is sent to the second sensor 5, 7 group via a programmable logic controller 2 (PLC 2).

[0065] According to this embodiment the method is based on the combination of the programmable logic controllers 2 to talk to the cameras 4, 5 and automation system, an Industrial Edge Devices 1 to control acquiring images 41, 51 from the cameras 4, 5 in the grow zone and process 9, analyze and / or transmit the images 41, 51 to a cloud server 3 that can ingest (take in) the images 41, 51 and store or analyze them.

[0066] This combination implements a feedback loop between the entities and an integration of plant monitoring with automation: The loop from the programmable logic controller 2 to Edge- 1 and cloud-based 3 analytics provides a unique opportunity to tie in automation based on image analysis. With the architecture outlined in Fig. 1, Al / model results can be used to trigger automation actions and the results of the automation action can be monitored using the cameras 4, 5. This also means that CEA integrators and engineers can build a single automation system without having to patch together systems from multiple vendors and OEMs. This is a common pain point for current CEA farmers, since most have to work with different systems and interfaces to control image acquisition, automation, and recipe / environment management.

[0067] Wherein there is a feedback loop between the edge device 1 and the programmable logic controller 2 (PLC 2). The at least one second image 51 is used to repeat the method. Instead of the first image data 41 of the at least one first image 41, image data from the at least one second image 51 is extracted and used to perform the method. In the repetition of the method, the method has the step (not shown in Fig. 2) of receiving second image data corresponding to at least one second image, the at least one second image being captured according to method described herein by the at least one second camera. The method further includes the step of extracting (= determining) second state data (= condition data, indicator data; state data may include a state label and / or state tag) by analysing the second image data, the second state data being indicative for (= describing / being dependent on / defining) a second state = condition) of the at least one first object and / or the at least one second object in the process, which (first / second image) are visualized on the at least one second image.

[0068] The method further includes the step of defining at least one second rule based on (= depending on) the second state data.

[0069] The method further includes the step of determining at least one second control command based on (= depending on) the at least one second rule, the at least one second control command comprising at least one second capturing command to capture at least one third image of the at least one first object and / or of the at least one second object and / or of the at least one third object, wherein the at least one third object is involved in (used in / related to) the process.

[0070] The method further includes the step of sending the at least one second capturing command to a third sensor group, the third sensor group comprising at least one third camera, the at least one second capturing command triggering = steering) the at least one third camera to capture the at least one third image.

[0071] According to a further embodiment the at least one second camera 5 is located at a different position than the at least one first camera 4 and / or wherein the second sensor group 5, 7 is located at different positions than the first sensor 4, 6 group.

[0072] It is to be understood that the elements and features recited in the appended claims may be combined in different ways to produce new claims that likewise fall within the scope of the present invention. Thus, whereby the dependent claims appended below depend from only a single independent or dependent claim, it is to be understood that these dependent claims can, alternatively, be made to depend in the alternative from any preceding or following claim, whether independent or dependent, and that such new combinations are to be understood as forming a part of the present specification.

[0073] While the present invention has been described above by reference to various embodiments, it should be understood that many changes and modifications can be made to the described embodiments. It is therefore intended that the foregoing description be regarded as illustrative rather than limiting, and that it be understood that all equivalents and / or combinations of embodiments are intended to be included in this description.

[0074] Although the invention has been explained in relation to its advantageous embodiments as mentioned above, it is to be understood that many other possible modifications and variations can be made without departing from the scope of the present invention. It is, therefore, contemplated that the appended claim or claims will cover such modifications and variations that fall within the true scope of the invention.

Claims

Claims1. An automated method for capturing images for an automation task, comprising the steps:- receiving first image data (41) corresponding to at least one first image (41), the at least one first image (41) being captured by at least one first camera (4), wherein the at least one first camera (4) belongs to a first sensor group (4, 6), the at least one first image (41) comprising a visualization of at least one first object (91), wherein the at least one first object (91) is involved in a process (9) of the automation task,- extracting state data by analysing the first image data, the state data being (121) indicative for a state of the at least one first object (91) in the process (9),- defining at least one rule (131) based on the state data (121),- determining at least one control command (141) based on the at least one rule (131), the at least one control command (141) comprising at least one capturing command to capture at least one second image (51) of the at least one first object (91) and / or of at least one second object (92), wherein the at least one second object (92) is involved in the process (9), and- sending the at least one capturing command to a second sensor group (5, 7), the second sensor group (5, 7) comprising at least one second camera (5), the at least one capturing command triggering the at least one second camera (5) to capture the at least one second image (51).

2. Method according to claim 1, wherein the process is arranged as:- a controlled environment agriculture process and / or- a manufacturing process and / or- an automated bakery process and / or- a food industry process and / or- a beverage industry process and / or- an industrial process.

3. Method according to one of the previous claims, the method comprising the further step of:- sending the at least one capturing command to the first sensor group (4, 6),the at least one capturing command triggering the at least one first camera (4) to capture the at least one second image (51).

4. Method according to one of the previous claims, wherein the at least one control command (141) further comprises at least one steering command to execute at least one steering action, the at least one steering action:- steering the process of the automation task, and / or- taking a corrective action (10) of the process of the automation task.

5. Method according to one of the previous claims, wherein the at least one control command (141) further comprises at least one sensing command to execute at least one sensing, the method comprising the further steps of:- sending the at least one sensing command to the first sensor group (4, 6), the first sensor group (4, 6) comprising at least one first sensor (6), wherein the at least one first sensor (6) is of a different type of sensor as the at least one first camera (4), the at least one sensing command triggering the at least one first sensor (6) to execute at least one sensing, and / or- sending the at least one sensing command to the second sensor group (5, 7), the second sensor group (5, 7) comprising at least one second sensor (7), wherein the at least one second sensor (7) is of a different type of sensor as the at least one second camera (5), the at least one sensing command triggering the at least one second sensor (7) to execute at least one sensing.

6. Method according claim 5, wherein the at least one sensing is performed on the at least one first object (91) and / or the at least one second object (92).

7. Method according to one of the previous claims, wherein the first image data (41) being:- processed image data and / or- raw image data and / or- meta data of the at least one first image (41).

8. Method according to one of the previous claims,wherein the at least one control command (141) is further determined based on a schedule (132), the schedule (132) defining at least one time step when the at least one control command (141) is to be determined.

9. Method according to one of the previous claims, wherein:- the state data ( 121 ) is extracted and / or- the at least one rule (131) is defined based on the state data (121), using:- an image analysis algorithm, and / or- an artificial intelligence algorithm, and / or- a modelling algorithm, and / or- a user input (8).

10. Method according to one of the previous claims, wherein the steps comprised by the method are executed by an edge device (1), wherein the at least one capturing command is sent to the second sensor group (5, 7) via a programmable logic controller (2).

11. Method according to one of the previous claims, wherein the at least one second camera (5) is located at a different position than the at least one first camera (4).

12. System (1) arranged to execute a method according to one of the previous claims.

13. System (1) according to claim 12, comprising:- a receiving unit (11), arranged to receive first image data (41) corresponding to at least one first image (41), the at least one first image (41) being captured by at least one first camera (4), wherein the at least one first camera (4) belongs to a first sensor group (4, 6), the at least one first image (41) comprising a visualization of at least one first object (91), wherein the at least one first object (91) is involved in a process (9) of the automation task,- an extracting unit (12), arranged to extract state data (121) by analysing the first image data (41),the state data (121) being indicative for a state of the at least one first object (91) in the process (9),- a definition unit (13), arranged to define at least one rule (131) based on the state data (121),- a determining unit (14), arranged to determine at least one control command (141) based on the at least one rule, the at least one control command (141) comprising at least one capturing command to capture at least one second image (51) of the at least one first object (91) and / or of at least one second object (92), wherein the at least one second object (92) is involved in the process (9), and- a sending unit (15), arranged to send the at least one capturing command to a second sensor group (5, 7), the second sensor group (5, 7) comprising at least one second camera (5), the at least one capturing command triggering the at least one second camera (5) to capture the at least one second image (51).

14. A computer program product comprising instructions which, when the program is executed by the computational device, cause the computational device to carry out the steps of the method according to one of the claims 1 to 11.

15. A computer-readable storage medium comprising instructions which, when executed by the computational device, cause the computational device to carry out the steps of the method according to one of the claims 1 to 11.

Images