Monitoring of material storage objects

Abstract

A method for monitoring and / or controlling one or more material storage object(s) in a material storage area associated with a production network, the method comprising: - providing image data of the material storage area including one or more material storage object(s), wherein the image data is related to geographic metadata indicating a position of the area where the image data was captured; - providing reference data of the material storage area, wherein the reference data is related to geographic metadata and including one or more static characteristic(s) associated with the infrastructure of the material storage area, wherein the one or more static characteristics are indicative of a possible position of the one or more material storage objects according to the infrastructure; - matching an object position of the one or more material storage object(s) in the image as captured with the one or more static characteristic(s) linked to the position; - detecting one or more object identifier(s) associated with a material stored in the material storage object; - providing the one or more object identifier(s) linked to the one or more static characteristic(s) for monitoring and / or controlling one or more material storage object(s).

Description

[0001] MONITORING OF MATERIAL STORAGE OBJECTS

[0002] TECHNICAL FIELD

[0003] The disclosure relates to the tracking of materials stored in material storage objects located in a material storage area of a production network, in particular a chemical production network. The disclosure relates to methods, apparatuses for monitoring and / or controlling at least one object in an area associated with a production network.

[0004] TECHNICAL BACKGROUND

[0005] Tracking of objects storing material located in an area associated with a production network, in particular a chemical production network, is a complex process.

[0006] EP3851359A1 discloses a method and system for resetting a track section occupancy state in the field of guided vehicles (subways, trains or train subunits, etc.) and concerns the detection of a presence or an absence of a guided vehicle within a track section.

[0007] SUMMARY

[0008] In an aspect the disclosure relates to a method for monitoring and / or controlling one or more material storage object in a material storage area associated with a production network, the method comprising: providing image data of the material storage area including one or more material storage object(s), wherein the image data is related to geographic metadata indicating a position of the area where the image data was captured; providing reference data of the material storage area, wherein the reference data is related to geographic metadata and including one or more static characteristic® associated with the infrastructure of the material storage area, wherein the one or more static characteristics are indicative of a possible position of the one or more material storage objects according to the infrastructure; matching an object position of the one or more material storage object(s) in the image as captured with the one or more static characteristic(s) linked to the position; detecting one or more object identifier(s) associated with a material stored in the material storage object; providing the one or more object identifier(s) linked to the one or more static characteristic(s) for monitoring and / or controlling one or more material storage object(s).

[0009] In another aspect the disclosure relates to an apparatus for monitoring and / or controlling one or more material storage object(s) in a material storage area associated with a production network, the apparatus comprising: an image data providing interface configured to provide image data of the material storage area including one or more material storage object(s), wherein the image data is related to geographic metadata indicating a position of the area where the image data was captured; a reference data providing interface configured provide reference data of the material storage area, wherein the reference data is related to geographic metadata and including one or more static characteristic® associated with the infrastructure of the material storage area, wherein the one or more static characteristic® are indicative of a possible position of the one or more material storage object(s) according to the infrastructure; a matching unit configured to match an object position of the one or more material storage object(s) in the image as captured with the one or more static characteristic® linked to the position; a detector configured to detect one or more object identifier(s) associated with a material stored in the material storage object; an identifier providing interface configured provide the one or more object identifier(s) linked to the one or more static characteristic(s) for monitoring and / or controlling one or more material storage object(s).

[0010] In another aspect the disclosure relates to a method for monitoring and / or controlling one or more material storage object(s) in a material storage area for use as one or more input material (s) for a production of one or more produces), the method comprising the steps: providing input material data associated with the production of one or more product(s) using one or more material input(s); providing one or more object identifier(s) associated with one or more material(s) stored in the one or more material storage object(s); gathering, based on the one or more object identifier(s) associated with the one or more material (s) stored in the one or more material storage object(s), material data associated with the one or more material(s) stored in the one or more material storage object(s) and the linked one or more static characteristic(s) indicative of the position of the one or more material storage object(s) as generated according to any of the methods disclosed herein or by the apparatus disclosed herein; matching the input material data associated with the production of one or more product(s) using one or more input material(s) with the material data associated with the one or more material(s) stored in the one or more material storage object(s); determining, based on the matching, one or more material(s) stored in the one or more material storage object associated with one or more object identifier(s) as one or more input material (s) for the production of the one or more product(s) using one or more input material(s); providing the one or more object identifiers associated with the one or more material(s) stored in the one or more material storage object(s) linked to the one or more static characteristic(s) for monitoring and / or controlling the one or more material(s) stored in the one or more material storage object(s) for use as one or more input material(s) for the production of the one or more product(s).

[0011] In another aspect the disclosure relates to an apparatus for monitoring and / or controlling one or more material storage object(s) in a material storage area for use as one or more input material(s) for a production of one or more product®, the apparatus comprising: an input material data providing interface configured to provide input material data associated with the production of one or more product(s) using one or more material input(s); an identifier providing interface configured to provide one or more object identifier(s) associated with one or more material(s) stored in the one or more material storage object(s); a gathering unit configured to gather, based on the one or more object identifier(s) associated with the one or more material(s) stored in the one or more material storage object(s), material data associated with the one or more material(s) stored in the one or more material storage object(s) and the linked one or more static characteristics) indicative of the position of the one or more material storage object(s) as generated according to any of the methods disclosed herein or by the apparatus disclosed herein; matching the input material data associated with the production of one or more product(s) using one or more input material(s) with the material data associated with the one or more material(s) stored in the one or more material storage object(s); a determining unit configured to determine, based on the matching, one or more material(s) stored in the one or more material storage object(s) associated with one or more object identifier(s) as one or more input materials) for the production of the one or more product(s) using one or more input material(s); a data providing interface configured to provide the one or more object identifiers associated with the one or more material(s) stored in the one or more material storage object(s) linked to the one or more static characteristic^) for monitoring and / or controlling the one or more material(s) stored in the one or more material storage object(s) for use as one or more input material(s) for the production of the one or more product(s).

[0012] In another aspect the disclosure relates to a method for producing one or more product(s) by using the one or more object identifier(s) as determined according to claim 10 for monitoring and / or controlling the one or more material(s) stored in the one or more material storage object(s) and providing the one or more material (s) stored to the production system for the production of the one or more product(s).

[0013] In another aspect the disclosure relates to the use of the one or more object identifier(s) associated with the respective one or more material(s) stored in the one or more material storage object(s) linked to the one or more static characteristics), wherein the linking is generated according to any of the methods disclosed herein or by the apparatus disclosed herein, for generating monitoring and / or control data for the one or more material(s) stored in the one or more material storage object(s) based on the linking.

[0014] In yet another aspect the present disclosure relates to a computer element with instructions, which when executed on one or more computing node(s) is configured to carry out the steps of the method(s) of the present disclosure or configured to be carried out by the apparatus(es) of the present disclosure.

[0015] Any disclosure, embodiments and examples described herein relate to the methods, the systems, apparatuses, chemical products and computer elements lined out above and below. Advantageously, the benefits provided by any of the embodiments and examples equally apply to all other embodiments and examples. EMBODIMENTS

[0016] In the following, embodiments of the present disclosure will be outlined by ways of examples. It is to be understood that the present disclosure is not limited to said embodiments and / or examples.

[0017] The invention relates to the field of inventory tracking of at least a material storage area related to a large production network. Inventory tracking is a common issue in complex production networks that are moving large amounts of multiple materials on a frequent basis. Whether that material be in containers, railcars, ships, or any other sort of as- set / equipment, tracking down inventory is often a physically intensive task which logistics personnel are involved in. Especially with storage sites that expand over a wide area with multiple storage positions and multiple input materials, it can take personnel hours or even days to find the desired piece of inventory. Thus, not only are personnel using their time inefficiently, but are potentially being placed in unsafe conditions due to the strenuous nature of this task. The objects, where the material is stored, are positioned with a high possibility on dedicated storage positions in the material storage area referred to as static characteristics such as rails, roads, rivers. The static characteristics are often positioned close to each other. Therefore, to differ the positions of the material storage objects and track the accurate position of the objects depending on the static characteristics, a sophisticated detection method has to be provided.

[0018] By matching an object position of the one or more material storage objects in the image as captured with the one or more static characteristics linked to the position, the material storage object(s) may be monitored and / or controlled. The matching enhances the accuracy of the monitoring of the position of the object. Further a reliable linking of the one or more material storage objects to the one or more static characteristic is ensured by the matching. As a result, the one or more material storage objects are uniquely linked to the one or more static characteristic(s). This limits the possibilities of the location of material storage objects and reduces the time for tracking the materials stored in the objects. Advantageously, this enables a stable production process in a complex production network by more efficient tracking of the input material(s).

[0019] In order to provide image data of the area including material storage area including one or more material storage objects, wherein the image data is related to geographic metadata indicating a position of the area where the image data was captured, data of the infrastructure of the material storage area is collected. The one or more static characteristics associated with the infrastructure of the material storage area are obtained from reference data of the material storage area, wherein the one or more static characteristics are indicative of a possible position of the one or more material storage object according to the infrastructure. Based on the result of the matching, one or more object identifier associated with a material stored in the material storage object are detected and linked to the one or more static characteristics. The result of the linking is provided for monitoring and / or controlling one or more material storage objects. Image data may be associated with the material storage area. Image data may be associated with at least a part of the material storage area associated with the production network. Image data may relate to the data of the infrastructure of the material storage area. Image data may include one or more material storage objects being positioned in the material storage area. Image data may relate to aerial image data, image data may relate to one or more images. Image data may be captured by dynamic systems. Dynamic systems may relate to moving systems capturing the image data such as input image(s). The dynamic system may include at least a camera system configured for acquiring images. The input images may be aerial images. The input images may be obtained during a flight over the material storage area associated with the production network such as a chemical production network. The flight may be operated by the dynamic system without any operator input. Dynamic system may relate to an aerial moving system such as unmanned arial vehicle e.g. a drone. Image data of the material storage area may be collected. Image data may include a set of image data. Image data may include a set of images. The set of images may include two or more fixed images such as pictures. The set of images may include two or more moving images such as videos. The image data may relate to geographic metadata indicating a position of the area where the image data was captured. The image data may relate to geographic metadata and including one or more static characteristics associated with the infrastructure of the material storage area. The material storage objects included in the image data may be positioned in relation to the one or more static characteristics. The object(s) may be located at the one or more static characteristics such as dedicated storage places. Image data may not be aligned to a spatial reference system or coordinate reference system.

[0020] Material storage object may relate to any asset being present in the material storage area associated with a production network such as a chemical production network. Material storage objects may relate to objects storing materials associated with the production network. Material storage objects may relate to objects storing inventory associated with the production network. Inventory may include equipment and / or materials and / or material types associated with the production network. Objects may relate to railcars, ships, cars, containers, trucks, and / or other storage means. There may be different types of material storage objects. Types of material storage objects may relate to the goods stored in the material storage objects. Types of material storage objects may relate to input material(s), intermediates and / or output material(s) for and / or produced by the production network. Types of material storage objects may relate to the classification of the stored material(s). Classification may relate to the content of the stored material(s). Classification may relate to the production system, for which the stored material(s) may be used. Material(s) related to the production system may relate to physical substances that products may be made from, including metal, wood, stone, crude oil or the like. Material(s) related to the chemical production system may include various types of materials) relating to chemical processes like Chemical process steps include for example oxidation, reduction, hydrogenation, dehydrogenation, hydrolysis, hydration, dehydration, halogenation, nitrification, sulfonation, amination, alkylation, dealkylation, esterification, polymerization, polycondensation, catalysis, fermentation, mixing, separation, purification or the like.

[0021] Material storage object(s) may be associated with one or more object identifiers. Object identifier(s) may comprise any identifier being uniquely associated with a material stored in the material storage object. The object identifier may be associated with the physical entity of the material. Object identifier may comprise any identifier being uniquely associated with a material storage object. Object identifier(s) may be uniquely linked to the material storage object and the material stored in the material storage object. The object identifier may be a digital identifier. For digital linking different identifiers associated with the physical material stored in the material storage objects may be linked. For example, an order number, a batch number, LOT number or a combination thereof may be linked. The object identi- fier(s) may relate to a string including characters, numbers or symbols being used to uniquely identify the material(s) stored in the respective material storage object. Object identifier(s) may relate to a unique sequence of characters and / or numbers. Object identifier(s) may relate to a code such as a QR-Code. Object identifier(s) may be positioned on the material storage objects such as a painted-on label. The object identifier(s) captured in image data may be read using an optical character recognition model.

[0022] Material storage area may be an area associated with a production network. Material storage area may be a particular part of an area associated with a production network. The material storage area may be located inside or outside a system boundary of a production network. The material storage area may represent an area where material storage objects are present. The material storage area may be used as an area for positioning or locating material storage object(s) associated with the production network such as a chemical production network. The material storage area may be a storage area for material storage objects conveying materials associated with the production network. The material storage area may include static characteristics associated with the infrastructure of the material storage area. The location of the material storage area may be described using geographical metadata.

[0023] The production network may relate to a chemical production network. The chemical production network may include one or more production process(es) with multiple production steps. The production steps included in the chemical network may be defined by the physical system boundary of the chemical production network. The system boundary may be defined by location and / or control over production processes or steps. The system boundary may be defined by a site of the chemical production network. The system boundary may be defined by production process(es) or step(s) controlled by one entity or multiple entities jointly. The system boundary may be defined by the value chain with staggered production process(es) or step(s) to the chemical end product, which may be controlled by multiple entities jointly or separately. The chemical production network may include a waste collection, a sorting step, a recycling step, a cracking step, a separation step to separate intermediates of one process step and further processing steps to convert such intermediates to output material(s) leaving the system boundary of the chemical production network. The input material(s) may enter the physical system boundary of the chemical production network. The entry point(s) of the chemical production network may be marked by the entry of input material(s) to the chemical production network or the system boundary of the chemical network. The output material(s) may leave the physical system boundary of the chemical production network. The exit point(s) of the chemical production network may be marked by the exit of output material(s) from the chemical production network or the system boundary of the chemical network. The chemical production network may include material flows including one or more connected or interconnected multi-input-multi output processes. Multi input-multi output processes may include chemical processes that use multiple chemical educts to produce more than one chemical product. The chemical production network may include one or more production chain(s) for the production of output material(s). The production chain(s) for the production of output material(s) may be inter-connected. The production chain(s) for the production of output material(s) may be interconnected with production chain(s) for the production of other output material(s). The production chain(s) for the production of output material(s) may include production chain(s) for the production of intermediates used to produce output material(s). The production chain(s) for the production of output material(s) may use input material(s) provided by chemical network(s) for the production of intermediates usable to produce output material(s). The chemical production may use multiple different types of material (s) for producing multiple different types of output materials.

[0024] Reference data may be associated with the material storage area. The reference data may include data of all parts of the material storage area associated with the production network. The reference data may include at least part of the material storage area associated with the production network. The reference data of at least part of the area may be assembled to cover relevant parts of the area. Reference data may relate to data indicating the area. Reference data may relate to data characterizing the area. Reference data may relate to data indicating and / or characterizing the area. Data indicating and / or characterizing the area may relate to static characteristics. Reference data may include one or more static characteristics associated with the infrastructure of the material storage area. The one or more static characteristics included in the reference data are indicative of a possible position of the one or more material storage object according to the infrastructure.

[0025] Reference data may relate to aerial data. The reference data may be generated by taking a map of the location. The reference data may be generated by using satellite data. The reference data may be generated by taking aerial images. The reference data such as reference images may be captured by a dynamic system. The dynamic system may include cameras, sensors, and / or other components that enable the dynamic system to capture and / or transmit the data. Dynamic systems may relate to moving systems. The dynamic system may relate to moving systems capturing the dynamic image. The dynamic system may include at least a camera system configured for acquiring images. The images may be aerial images. The images may be obtained during a flight over the chemical production network. The flight may be operated by the dynamic system without any operator input. Dynamic systems may relate to aerial moving systems such as an unmanned arial vehicle e.g. a drone. Dynamic systems may relate to aerial moving systems such as remotely piloted aircrafts (RPA).

[0026] The reference data may include a set of reference data. A set may include one or more data points. Reference data may include a set of images. The set of images may include two or more moving images such as a video. The set of images may include two or more fixed images. The set of image data may show material storage areas being located next to each other. The set of images may include image data having an overlay. The overlay may help to ensure the set of image data is aligned. The overlay may help to maintain the perspective. The overlay may support ensuring that the set of image data is aligned and / or to maintain the perspective. The reference data may relate to geographic metadata. The reference data may be aligned to a spatial reference system or coordinate reference system. The reference data may be transformed to a spatial reference system or coordinate reference system such as a geographic coordinate system before using it as reference data. The reference data may be validated. Validation may be relevant for using the refence data as a baseline for linking one or more objects one or more static characteristics. Geographic metadata may be associated with a position of the area where the image data and / or reference data was captured. Geographic metadata may relate to data having an implicit or explicit association with a location relative to earth. Geographic metadata may relate to data indicating a position. Geographic metadata may relate to geographic coordinate systems. Geographic coordinate systems may include geographic coordinates such as latitude and longitude. Alternative geographic coordinate systems may include Maidenhead Locator System, Global area reference system (GARS), Open Location Code, Geohash, Mapcode, What3words or the like. Geographic metadata may relate to a geolocation. Geographic metadata may relate to global positioning system (gps) data.

[0027] Static characteristic(s) may be associated with the infrastructure of the material storage area. Static characteristic(s) may be indicative of a possible position of the one or more material storage objects according to the infrastructure. Static characteristic(s) may relate to a structural framework indicating a possibility for the position of material storage objects. The arrangement of the possible positioning of material storage objects may form the static characteristic(s). Static characteristic(s) may limit the area, where the material storage objects may be able to position and / or move. The limitation may be obtained from the infrastructure of the material storage area. The limitation may be obtained from the probability for the positioning of objects and / or their movement. The limitation may be obtained from dedicated areas for the positioning and / or movement of material storage objects in such a way that they are suitable for the positioning and / or movement of material storage objects such as dedicated places and / or roads. Static characteristics may relate to at least part of the area with dedicated places for the one or more material storage objects such as storage places. Static characteristic(s) may be included in the reference data. Static characteristic(s) may be derived from reference data. Static characteristic(s) may be included in and / or derived from reference area. Static characteristics) may relate to a line structure representing the essence of the shape of the region such as a skeleton. Skeletonization may be used for generating static characteristics. Skeletonization may relate to reducing binary objects to a one-pixel wide representation. Skeletonization may involve shrinking the reference image and / or the set of reference images until the area of interest is 1 pixel wide. Static characteristics may be constant over a period of time. Period of time may relate to 3 months, preferably more than half a year, more preferable more than a year. Examples of static characteristics may be tracks such as rail-tracks, roads, rivers, storage places, parking lots or the like.

[0028] Static characteristic(s) may be associated with one or more static characteristic identifiers. The static characteristic identifier may comprise any identifier uniquely associated with the static characteristic(s). The static characteristic identifier may be a digital identifier. For digital linking different identifiers associated with the physical static characteristic may be linked. The static characteristic identifier may be associated with the possible positions of the one or more material storage objects according to the infrastructure. Static characteristic(s) may be associated with at least one static characteristic identifier. The static characteristic identifier may be a unique identifier for the static characteristics). Identifier(s) may relate to a name, series of numbers or symbols being used to represent a static characteristic and / or a part of the static characteristic(s). Identifier(s) may relate to a unique sequence of characters and / or numbers. Identifier(s) may relate to a code such as a QR-Code. Identifiers may relate to a numeration of the static ch aracteristi c(s) such as a numeration of tracks of a railway system.

[0029] Input material data and / or material data may be associated with the material(s) stored in one r more material storage object(s). Input material data and / or material data may be associated with the input material for producing a product using input material(s). An object identifier may relate to the (input) material data. Object identifier(s) may be uniquely linked to the material data. (Input) material data may relate to the content of the material(s). The content of the materials) may relate to the ingredients. Input material data may relate to the type of material(s). Type of material(s) may relate to raw material(s), semi-finished material(s) and / or finished product(s). Raw material(s) may include fossil and non-fossil raw material(s). Type of material(s) may relate to the state of aggregation such as gas, liquid or solid. (Input) material data may relate to one or more characteristics or properties of the input material. Input material data may relate to characteristic(s) of the input material(s). Type of material(s) may include metals, ceramics, polymers, composites or semiconductors. Input material data may relate to chemical and / or physical and / or mechanical properties of the input material(s). Input material data may relate to chemical properties. Chemical properties may describe the change of a substance's chemical composition given a specific set of conditions. Chemical properties may describe the change of a substance's chemical composition given a specific set of conditions. Chemical properties may include flammability, toxicity, acidity, reactivity, combustibility, or the like. Input material data may relate to physical properties. Physical properties may be used to describe the physical characteristics of a substance. Physical properties may be observed or measured without changing the identity of the substance. Physical properties may include color, density, viscosity, hardness, a boiling point, flashpoint, melting point, and / or pour point or the like. Mechanical properties may relate to the behavior of materials under the action of external forces. Mechanical properties may include modulus, strength, elasticity, or the like. Chemical and / or physical properties and / or mechanical properties may relate to any properties of input material(s) that may be relevant and / or critical for the production process(es) using material inputs. Input material data may include any one or more of values being indicative of properties such as quantity of the input material. Alternatively, or in addition, the value being indicative of the quantity may be fill degree and / or mass flow of the input material. Input material data may include data relating to the production processes using input material(s). Input material data may relate to the place of the production processes using input material(s).

[0030] In an embodiment the static characteristics may indicate an arrangement of the possible position of material storage objects. Static characteristics may be represented by a line structure indicating the probability for objects in the storage area. The line structure may be generated based on skeletonization.

[0031] In another embodiment the reference data may comprise a set of images of the storage area.

[0032] In another embodiment the provided at least one object identifier linked to the at least one static characteristic identifier for monitoring and / or controlling one or more material storage object may be used for reordering of the one or more material storage object. Reordering of material storage objects based on the data may assist to secure a time efficient processing of the one or more material (s) stored in the one or more material storage object. In another embodiment the reference data of the material storage area is selected from a set of reference data of the material storage area by matching the related geographic metadata of the reference data with related geographic metadata of the of the one or more input images.

[0033] In another embodiment the geographic metadata may relate to a geographic coordinate system, wherein the geographic coordinate system includes latitude and longitude. Such geographic coordinates may enable an exact and unique description of the position of the area and / or included objects.

[0034] In another embodiment the image data of the material storage area may be collected by a dynamic system. The dynamic system may relate to an aerial moving system. Dynamic system may relate to an arial moving system such as unmanned arial vehicle e.g. a drone.

[0035] In another embodiment the at least one static characteristic associated with the at least one static characteristic identifier may be derived from the reference data of the area. Static characteristic(s) may relate to a line structure representing the essence of the shape of the region such as a skeleton. Skeletonization may be used for generating static characteristics. Skeletonization may relate to reducing binary objects to a one-pixel wide representation. Skeletonization may involve shrinking the reference image and / or the set of reference images until the area of interest is 1 pixel wide. Static characteristics may be constant over a period of time. Period of time may relate to 3 months, preferably more than half a year, more preferable more than a year. Examples of static characteristics may be tracks such as rail-tracks, roads, rivers, storage places, parking lots or the like.

[0036] In another embodiment a perspective transformation may be applied to the one or more image data based on the reference data by selecting one or more feature points, extracting one or more feature descriptors and matching the one or more features. Perspective transformation may include scaling, translation, rotation, affine transformation and / or geometric transformation. Perspective transformation of the image data may include feature detection on the input image(s) and the respective reference image(s), feature matching and constructing a transformation matrix such as a homography matrix. Perspective transformation may include constructing a transformation matrix. The transformation matrix may be constructed by using feature matches such as matching the static characteristics that may be included in input data and reference data. The feature matches may be generated by a feature matching algorithm. The feature matching algorithm may use feature detected by a feature detection algorithm, which may detect feature on the input image and the reference image. The feature may include at least points such as pixels of the input image and of the reference image.

[0037] In another embodiment the one or more object identifiers associated with a material stored in the material storage object are detected employing an optical character recognition model. The object identifier(s) may be located on the one or more storage objects. The object identifier may be a number located on the detected material storage objects such as a number located at the top of a railcar. The object identifier of the detected material storage object associated with the geolocation of the one or more input images may be stored. The identifier of the detected object associated with the geolocation of the one or more input image(s) and / or the one or more detected objects may be the output. Optical character recognition may relate to the conversion of images of strings into machine-encoded text.

[0038] In another embodiment matching the object position of the one or more material storage objects in the image as captured with the one or more static characteristics linked to the position includes determining a probability of the position of the one or more material storage objects in relation to the one or more static characteristics and / or assigning the one or more material storage object with a highest probability to the static characteristics. Highest probability may relate to a minimum distance of the center point(s) of the material storage object(s) to the static characteristic(s). Highest probability of the position of the at least one material storage object at the static characteristic may relate to the material storage object having a larger distance of the center point to other static characteristic(s).

[0039] In another embodiment matching the object position of the one or more material storage objects in the image as captured with the one or more static characteristics linked to the position includes assigning the one or more material storage objects to the closest static characteristic. Matching may include calculating a distance of the of the material storage object(s) in relation to the static characteristic(s). Matching may include calculating a rotation of the bounding box and matching this to a static characteristic(s) in order to determining the probability for linking of the material storage object to the respective static characteristic(s).

[0040] In another embodiment matching the position of the at least one object with the at least one static characteristic in such a way that the at least one object is linked to the at least one static characteristic may include providing a bounding box of the objects. The bounding box may be the smallest bounding box. The smallest bounding box may be a box with the smallest measure such as area within all points e.g. pixels of the object lie. The bounding box may be the output of the deep learning model may be a Region-Based Convolutional Neural Network such as R-CNN, Fast R-CNN and Faster-RCNN.

[0041] In another embodiment matching the object position of the one or more material storage object in the image as captured with the one or more static characteristics linked to the position includes determining one or more center points of the one or more material storage objects and calculating a distance of the one or more center points of the one or more material storage objects in relation to the one static characteristics. The center point of the object may be the midpoint of the object. The midpoint of the object may be a point that may be equidistant to at least two points of the circumference or surface of the object. The circumference of the object may be defined by the bounding box. Matching may include superposing the center points of the material storage objects and the static characteristics. Matching may include calculating a distance of the at least one center point of the at least one material storage object in relation to the at least one static characteristic. Matching may include determining the shortest distance based on the calculation.

[0042] In another embodiment determining, based on the matching, one or more material(s) stored in the one or more material storage object(s) associated with one or more object identifier(s) as one or more input material(s) for the production of the one or more product(s) using one or more input material(s) includes selecting the one or more materials) stored in one or more material storage object(s) relating to a minimum number of additional one or more material storage object(s) to be moved. The additional material storage object(s) may relate to the same static characteristics). The additional material storage object(s) may relate to adjacent static characteristic(s). The additional material storage object(s) may have to be relocated in order to move the material storage object(s) storing material used as input material(s) for producing a product.

[0043] In another embodiment determining, based on the matching, one or more material(s) stored in the one or more material storage object(s) associated with one or more object identifier(s) as one or more input material(s) for the production of the one or more product(s) using one or more input material(s) includes selecting the one or more material(s) stored in the one or more material storage object(s) having a minimum distance to the production site. The minimum distance to the production site may be determined based on the infrastructure of the material storage area. The minimum distance to the production site may be determined based on the infrastructure from the material storage area to the production site.

[0044] In another embodiment providing the one or more materials stored in the material storage objects associated with one or more object identifiers linked to the one or more static characteristics for monitoring and / or controlling the material stored in a material storage object includes generating control data for providing the one or more materials stored in the one or more material storage object(s) as input materials for the production of a product using material inputs. Monitoring and / or controlling data may be generated based on the linking of the material storage objects to the one or more static characteristics. Monitoring and / or controlling data may be generated based on the infrastructure from the material storage area to the production site. Monitoring and / or control data may relate to the position of the material storage object storing the input material. The material storage object may be relocated based on the position. The relocation may influence further material storage objects linked to the same static characteristic or adjacent static characteristic(s). The monitoring and / or controlling data may be provided via a communication interface. The monitoring and / or controlling data may be displayed to the user. A user interface may display the monitoring and / or controlling data.

[0045] BRIEF DESCRIPTION OF THE DRAWINGS

[0046] In the following, the present disclosure is further described with reference to the enclosed figures. The same reference numbers in the drawings and this disclosure are intended to refer to the same or like elements, components, and / or parts.

[0047] Fig. 1 illustrates an example of a production network, in particular a chemical production network comprising multiple materials as inputs and outputs. Fig. 2a, Fig. 2b and Fig.2c illustrate examples of the generation of reference data such as data of the area associated with the infrastructure of the material storage area.

[0048] Fig. 3a, Fig. 3b, Fig. 3c, Fig. 3d, Fig ,3e illustrate examples or parts of a workflow for monitoring inventory located at a material storage area associated with a production network.

[0049] Fig. 4 illustrates an example of a method for monitoring and / or controlling one or more material storage objects in a material storage area associated with a production network.

[0050] Fig. 5 illustrates an example for a method for monitoring and / or controlling one or more material storage object(s) in a material storage area for use as one or more input material (s) for a production of one or more product(s).

[0051] DETAILED DESCRIPTION

[0052] The following embodiments are mere examples for implementing the method, the system or application device disclosed herein and shall not be considered limiting.

[0053] Fig. 1 illustrates an example of a production network, in particular a chemical production network comprising multiple materials as inputs and outputs.

[0054] For producing one or more output product(s) and / or intermediates different materials may be provided as physical inputs from material providers or suppliers.

[0055] The chemical production network may include multiple interlinked or interrelated processing steps. Chemical production networks may include different process steps for producing one or more output material(s) from multiple materials as input material(s). Chemical processes may include at least one process step involving at least one chemical reaction. The chemical processes may produce from multiple materials as input materials multiple output materials. Chemical process steps include for example oxidation, reduction, hydrogenation, dehydrogenation, hydrolysis, hydration, dehydration, halogenation, nitrification, sulfonation, amination, alkylation, dealkylation, esterification, polymerization, polycondensation, catalysis, fermentation, mixing, separation, purification or the like. The chemical process steps may be distributed over the chemical production network. The chemical production steps may be connected by means of dedicated transportation systems such as railcars, pipelines, supply chain vehicles, like trucks, supply chain ships or other cargo transportation means.

[0056] The input materials may be fed into the chemical production network at any entry point. The input materials may be fed into the chemical production network at the start of the chemical production network. The output materials may leave the chemical production network at any exit point.

[0057] The chemical production network may include multiple production steps. The production steps included in the chemical network may be defined by the system boundary of the chemical production network. The system boundary may be defined by location or control over production processes. The system boundary may be defined by the site of the chemical production network. The system boundary may be defined by production processes controlled by one entity or multiple entities jointly. The system boundary may be defined by value chain with staggered production processes to an end product, which may be controlled by multiple entities separately.

[0058] At least some of the multiple material(s) as input material(s), intermediates, output material(s) and / or waste stream(s) may be located at the chemical production network. At least some of the multiple material(s) may be located at a material storage area 100. The chemical production network may be associated with a material storage area 100. The material storage area 100 may be a railcar system including railcars and railways. Due to the size of the chemical production area, the storage area may be a large area. The storage area may include multiple railcars. The process steps and / or production steps of production chains as described before, may be performed sequentially in time and / or space to chemically transform the input materials to output materials and / or intermediates. Chemical processes may be time critical. For starting a chemical process, one or more input material(s), such as intermediates and / or feedstock, may be transported to a respective plant. The material(s) may be located at different material storage areas. The material(s) may be located at one material storage area with many other types of material(s). The material storage area may be a large storage area having a complex environment. Complex environment may relate to multiple tracks and / or a tangled arrangement of tracks. Complex environment may relate to multiple railcars being arranged at the multiple tracks storing multiple types of material(s). Types of material(s) stored in the railcars may relate to the classification of the material(s). Despite the varying type of material(s) stored in the object such as the railcars, the object cannot be distinguished based on the material(s) stored. Each object may be associated with an object identifier having a unique link or connection the material(s) stored in the object. Each object may be associated with an object identifier pointing to information about the material(s) stored in the object.

[0059] There may be multiple production points such as plants and / or production areas in the production network. Different material(s) may be required based on the production process. There may be one or more entry point(s) to the production processes. The material (s) may be matched to the production process based on a request. The request may include one or more material type(s), quantity and / or the object identifier(s). The request may be matched to one or more material(s) and the object storing the material(s) may be located. The process of locating the one or more materials) stored in one or more object(s) may be time critical. The material(s) may be requested for production process at a predefined time. If the location of the one or more material(s) stored in one or more object(s) and / or their transportation exceeds a specified time limit, the production process may have to be stopped. Exceeding the time limit may not be possible based on the production process leading to the production of out-of-spec material(s). Exceeding the time limit may lead to critical process states of the production process.

[0060] Multiple material(s) stored in the objects may be shipped to one or more costumer(s) to match the request of the one or more costumer(s).

[0061] For a chemical process, railcars 102, 104, 106 and / or 108 may provide at least two or more materials. Railcar 104 may be used to store and / or transport the material(s) to a chemical plant. Railcar 104 may be used to store and / or transport the material(s) to one or more costumers. Before moving the railcar to an entrance point of the chemical production network, the location of railcar 104 may be tracked. For the transportation of railcar 104 to the chemical plant, railcar 102 may have to be moved. For the transportation of railcar 104 to at least another chemical plant or another entrance point of the chemical production network, railcar 106 may have to be moved. The material(s) stored in the material storage object(s) may be transported to the production system of the chemical plant for producing the product.

[0062] If railcar 108 has to be tracked in order to deliver the material(s) stored to a chemical production process or a costumer, it is essential to determine whether the railcar is located on track 112 or 114. Based on the assignment to a track multiple different railcars may have to be moved in order to move railcar 108.

[0063] This is a simplified example of a material storage area. The dependences of objects such as railcars in a production network such as a chemical production network may be more complex with large areas and multiple objects storing multiple material(s), multiple railways, for multiple processes and / or multiple customers. Providing a system for monitoring and / or controlling the material storage area associated with the chemical production network may enable a tracking of the material storage area 100.

[0064] Fig. 2a, Fig. 2b and Fig.2c illustrate examples of the generation of reference data such as data of the area associated with the infrastructure of the material storage area.

[0065] For generating a reference data of the area to be monitored, one or more reference image(s), one or more reference image set(s) and / or static characteristics associated with the infrastructure of the material storage area may be obtained.

[0066] Fig. 2a shows a reference information of the area to be monitored that may be obtained. The respective area associated with the chemical network may relate to a material storage area. The material storage area may expand over a large area. The storage area associated with the chemical network may be located at the site of chemical network. The storage area may be distributed throughout the site of the chemical network. The material storage area associated with the chemical network may be located inside the boundary of the chemical network as described in detail in Fig. 1. The material storage area may relate to a material storage area with dedicated storage places for one or more objects storing material(s). The storage places may limit the area, where the objects may be able to be located. The storage places may limit the area, where the objects may be able to move. The limitation may be obtained by the environment of the material storage area and / or the movement of the objects such as rails or rivers. The limitation may be obtained by dedicated areas for the location and / or movement of the object(s) such as dedicated places and / or roads. The material storage places may relate to static characteristics associated with the infrastructure of the material storage area. The arrangement of the material storage places may form the static characteristics. The static characteristics may relate to a structural framework such as the arrangement of rails. Static characteristics may be indicative of a possible position of the one or more material storage objects according to the infrastructure. Static characteristics may be constant over time. The time may expand over at least half a year preferably a year, more preferably 3 years. The reference data may be generated by taking a map of the location. The reference data may be generated by using satellite data. The reference data may be generated by taking aerial images. The reference data may be captured by a dynamic system. The dynamic system may include cameras, sensors, and / or other components that enable the dynamic system to capture and / or transmit the data. Dynamic systems may relate to moving systems. The dynamic system may relate to moving systems capturing a dynamic image. The dynamic system may include at least a camera system configured for acquiring images. The dynamic images may be aerial images. The dynamic images may be obtained during a flight over the chemical production network. The flight may be operated by the dynamic system without any operator input. Dynamic systems may relate to arial moving systems such as an unmanned arial vehicle e.g. a drone. Dynamic systems may relate to an arial moving system such as a remotely piloted aircraft (RPA). Dynamic systems may be configured for autonomously starting the flight from a predefined position. The dynamic system may be configured for flying according to a trajectory. The trajectory may be defined by a predefined route. The predefined route may be based on the material storage area to be monitored. The trajectory may relate to the material storage objects located in the storage area as described later in detail. The trajectory may be defined by flying from the start of the material storage area to the end such as shown in Fig. 2a. In the example of Fig. 2a, the trajectory starts on the left-hand side and ends on the right-hand side as signified by progression of time (t, t+1, t+2). The dynamic system may be configured for obtaining images of the material storage area of the chemical production network. The dynamic system may be configured to fly to a predefined position. The predefined position may be another position depending on the trajectory of the area of the chemical production network to be checked. The reference data of the location may be collected during a flight. The dynamic system may include at least one processing unit capable of image processing tasks. The dynamic system may communicate with an interface of a processing unit. The reference data may be transmitted via a wired and / or wireless network such as Ethernet, USB, LAN, WLAN, WiFi, cellular networks and / or Bluetooth. The data may be provided to a computing device or system that includes at least one physical and tangible processor and a physical and tangible memory capable of having thereon computerexecutable instructions that are executed by a processor. The data may be provided to an operating system of the chemical production network.

[0067] The reference data may include a set of reference data. A set may include two or more data points. Reference data may include a set of images. The set of images may include two or more moving images such as a video. The set of images may include two or more fixed images such as pictures as shown in the context of Fig. 2a. The set of images may show areas being located next to each other. The set of images may include images having an overlay. The overlay may help to ensure the set of images is aligned. The overlay may help to maintain the perspective. The overlay may support to ensure that the set of images is aligned and / or to maintain the perspective. The reference data set may be associated with a digital identifier (ID). The ID may relate to a unique ID. The ID may have a unique link or connection to the reference image set. The ID may have a unique link or connection to the reference image. The reference image and / or the reference image set may be associated with the time of collecting the image. The reference image set may relate to geographic metadata. The geographic metadata may include a geolocation. The geographic metadata may include geographic coordinate such as latitude and longitude. The reference image set may be assembled from the reference images based on the time associated with the reference images. The reference image set may be assembled by using reference images based on the geolocation associated with the reference images. The reference image set may be assembled based on counting numbers associated with the reference images. The reference image set may be assembled based on time and / or geolocation and / or counting numbers. The reference images and / or the reference image set may be stored in a folder. The assembled reference images may show relevant details of the respective area such as the material storage area. The assembled reference images may relate to geographic metadata and may include one or more static characteristics associated with the infrastructure of the material storage area. The reference data may be transformed to a spatial reference system or coordinate reference system such as a geographic coordinate system before using it as reference data.

[0068] Reference images may include static characteristics (Fig. 2c). Static characteristics may relate to tracks of a railcar. For each reference image the tracks may be annotated. Annotation of reference images may relate to a manually marking of the static characteristics. Annotation may relate to form a line with segments formed by points such as a polyline. The points may relate to coordinates of the reference images. The points may relate to pixels of the reference image. Static characteristic(s) may relate to a line structure representing the essence of the shape of the region such as a skeleton. A skeleton of a shape may relate to a thin version of that shape, which is equidistant to its boundaries. Skeletonization may be used in digital image processing. Skeletonization may refer to the application of an algorithm that produces a skeleton-like texture from the outline of prominent features in an image. Skeletonization may be used for generating static characteristics. Skeletonization may relate to reducing binary objects to a one-pixel wide representation. Skeletonization may involve shrinking the reference image and / or the set of reference images until the area of interest is 1 pixel wide. There may be different algorithms possible for computing skeletons for shapes in images, e.g. Zhang-Suen Thinning algorithm, morphological thinning, or medial axis skeletonization.

[0069] The static characteristics may be stored. Static characteristics may be stored employing a markup language for the representation of hierarchically structured data in the format of a text file such as XML. Static characteristics may be associated with the image ID and / or the reference image such as an overlay of the tracks. Static characteristics may be associated with the respective static characteristic identifier. The assembled static characteristics may form the arrangement of the storage places of at least a part of the storage area. The assembled static characteristics may be associated with the infrastructure of the material storage area.

[0070] The reference data such as the reference image, the reference image set and / or the static characteristics may be stored. The reference data may be generated as a preparatory step. The reference data may form the baseline reference data. The reference data may be used for matching the input images as described in more detail in the context of Fig. 3 to 5.

[0071] Fig. 3a, Fig. 3b, Fig. 3c, Fig. 3d, Fig ,3e illustrate examples or parts of a workflow for monitoring inventory located at a material storage area associated with a production network. The workflow described in this figure may include the collection of data of the area in association with a chemical production network, the matching with reference data as described in detail in Fig. 2a-c and the identification of objects to track the location of at least an object.

[0072] Input data of at least a part of the area such as a material storage area may be generated. Input data may relate to one or more input images. Image data may be generated by dynamic systems. Dynamic systems may relate to moving systems capturing the input image(s) such as dynamic image(s). The dynamic system may include at least a camera system being configured to acquire images. The images may be aerial images. The images may be obtained during a flight over the storage area associated with the production network. The flight may be operated by the dynamic system without any operator input. Dynamic system may relate to an arial moving system such as unmanned arial vehicle e.g. a drone.

[0073] Dynamic data of the location may be collected. This dynamic data may include a set of dynamic data. A set may include one or more data points. Dynamic data may include a set of images. The set of images may include two or more fixed images such as pictures. The set of images may include two or more moving images such as a video.

[0074] The input image(s) may be an image of the material storage area including one or more material storage objects. The input image(s) may be an image of the material storage area including one or more static characteristics associated with the infrastructure of the material storage area. The input image(s) may include at least one material storage object in the material storage area. The one or more static characteristic(s) may be indicative of a possible position of the one or more material storage object(s) according to the infrastructure. The material storage object(s) may be located in relation to the static characteristics. The material storage object(s) may be located at the static characteristics such as storage places. The material storage object(s) may include equipment for storing material (s) such as input material(s), intermediates, output material(s) or the like. The material storage object(s) may relate to railcars being located at the static characteristics such as tracks. The material storage object(s) may relate to shipping containers and / or barges being located at static characteristics such as rivers. The material storage object(s) may relate to van and / or bulk trailers being located at static characteristics such as roads.

[0075] The image(s) data may be associated with geographic metadata such as a data from a geographic coordinate system. Geographic coordinate system may include geographic coordinates such as latitude and longitude.

[0076] The reference data such as the reference image, the reference image set and / or the static characteristics may be generated as described in detail in Fig. 2a-2c. The reference data may be provided. The reference data may be provided in association with the metadata such as the geographic coordinates, static characteristic identifier, further IDs and / or a time stamp indicating the time when the image was captured.

[0077] For each input image(s) the geographic metadata may be matched to the geographic metadata of the reference data such as the reference image set. Matching may include the identification of the reference image by comparing the geolocation such as the latitude and longitude of the image data and the reference image set. The result of the matching may include the extraction of the respective reference image set and / or reference image, which is associated with the same geographic metadata as the input image(s).

[0078] Next, a perspective transformation may be applied to the input image. The transformation is shown in detail in Fig. 3b. The transformation may include a geometric transformation such as rotation of the input image and / or a perspective transformation. Perspective transformation may include image registration for transforming different sets of data into one coordinate system. The perspective transformation of the input image may include feature detection on the input image(s) and the respective reference image(s), feature matching and constructing a transformation matrix such as a homography matrix.

[0079] A feature of an image may be a specific structure in the images such as points, edges and / or objects. Features may be classified into two categories. The first category may include features that are in a specific location of the image such as mountain peaks, building corners, and / or interestingly shapes patches of grass. The second category may include features that can be matched based on their orientation and local appearance (edges). Features of the second category may be an indicator of object boundaries.

[0080] A feature detection algorithm may detect relevant features in each input image. Static characteristics may be features. The feature(s) detected in the input image may be the tracks. The feature detection algorithm may detect relevant features in each reference image. The feature detection may first extract feature points of a reference image and the input image to be applied to an image matching operation. Feature detection may include Scale Invariant Feature Transform (SIFT), Speeded Up Robust Features (SURF), and / or Maximum Stable Extremal Regions (MSER).

[0081] First, scale-based filtering may be used. LaPlace of Gaussian or Difference of Gaussian may act as a blob detector which detects blobs in various sizes due to change in standard deviation. Standard deviation may act as a scaling parameter. The features may be extracted by detecting local extrema. For example, one pixel in an image may be compared with its 8 neighbors as well as 9 pixels in next scale and 9 pixels in previous scales. If it is a local extremum, it is a potential key point. The features may be described using a descriptor vector for each feature point. A descriptor vector may include informational components of one or more features. The feature vector may represent a single pixel or a patch within an image. The descriptor vector may describe the intensity distribution of the pixels within the neighborhood of the feature. The descriptor vector may transform the image data into a reduced representation set of features.

[0082] Feature matching may include one or more matching operations. Feature matching may include establishing correspondences between two images of the same area being captured at different times, ancles, and / or weather conditions. The two images may have a different perspective towards the area. The features of the input image may be compared with the features of the reference image. To compare the feature of the input image with the features of the reference image, the descriptor vectors of the images may be compared. At least one feature point of the image data may be selected. At least a most similar feature point of the reference image may be selected. The most similar feature points may be selected as a matching pair. Feature points between two images may be matched by identifying their nearest neighbors. This step may be repeated. The matching may include using distance calculation (Brute- Force Matcher), nearest neighbor search (FLANN) or other matching algorithms. In another example Local Feature Transformers (LoFTR) may be used for local feature matching. A local feature Convolutional Neural Network (CNN) may extract coarse-level feature maps from the two images and fine-level feature maps from the two images. CNNs may possess the bias of translation equivariance and locality, which may be suited for local feature extraction. The down sampling introduced by the CNN may reduce the input length of the LoFTR module. The coarse feature maps may be flattened to 1-D vectors and positional encoding may be added. Positional encoding may give an element a unique position information in the sinusoidal format. By adding the position encoding to the feature maps, the transformed features may become position-dependent, which may be a basis for LoFTR to produce matches in indistinctive regions. The added features may be processed by a LoFTR module. The LoFTR module may comprise self-attention and / or cross attention layers. Next, a differentiable matching layer may be used to match the transformed features of the LoFTR module. The result of the matching may be a confidence matrix. The matches in the confidence matrix may be selected according to a confidence threshold and mutual-nearest-neighbor criteria, which may filter possible outlier coarse matches. The result may be the prediction of coarse-level matches. Next a coarse-to-fine module may be applied using a correlation-based approach. For selected coarse predictions, the position at fine-level feature maps may be located. A local window may be cropped from the fine-level feature map. Coarse matches may be refined within this local window to a sub-pixel level as the final match prediction.

[0083] The matching pairs e.g. the coordinate data of the matching pairs may be provided to a homography matrix generator. The perspective transformation may be conducted by constructing a homography matrix. Homography matrix may relate two cameras viewing the same planar surface. In other words, two 2D images viewing the same area from a different angle, may be related by a homography matrix.

[0084] The result of the transformation may be the perspective transformation of the input image in order to align the features such as tracks of the input image(s) and the reference image(s) and / or the static characteristic(s) included in the reference image(s). A simplified example for a result of the transformation is shown in Fig. 3c.

[0085] The transformed image may be the input image for the object detection model (Fig. 3a).

[0086] The material storage objects included in the input image(s) may be detected. Fig. 3d shows an example architecture of a model for an object detection. The transformed image including the object(s) may be the input for a deep learning model. The input may be a tensor. The deep learning model may be a convolutional neural network (CNN). The machine learning model may run an object detection algorithm. The CNN may identify the object 303 of interest such as the railcars. The CNN may include a convolution layer, which may include multiple alternating Convolution layers, rectified linear unit layers (ReLU) (not shown) and pooling layers, multiple fully connected layers (FC) and ReLU layers (not shown).

[0087] In the convolutional layer the convolution kernel may slide along the input matrix for the layer to extract a set of features. A feature map may be generated. The feature map may contribute to the input of the next layer.

[0088] The convolutional layer may include local and / or global pooling layers along with traditional convolutional layers. Pooling layers may reduce the dimensions of data by combining the outputs of neuron clusters at one layer into a single neuron in the next layer. Local pooling may combine small clusters. Global pooling may act on all the neurons of the feature map. There are two common types of pooling in popular use: max and average. Max pooling may use the maximum value of each local cluster of neurons in the feature map. Average pooling may use the average value. The FC layers may be configured to perform matrix multiplications. The FC layers, except for the last FC layer, may followed by a corresponding ReLU layer ( not shown ). The ReLu layers may be configured to provide nonlinear characteristics to the CNN system. The last FC layer may be a decision module (not shown) configured to make a prediction based on the output of the last FC layer. The detected material storage objects may be the output of the CNN module.

[0089] The material storage object(s) 303 may be indicated using a bounding box 301 as shown in Fig. 3e. The bounding box may be the smallest bounding box. The smallest bounding box may be a box with the smallest measurement such as an area which includes all points e.g. pixels of the object. The objects may be classified, e.g. classified as railcars. Input data for the machine learning model may be an image with one or more objects such as a photography. Output data of the machine learning model may be the detected object(s). The object may be marked by one or more bounding boxes (e.g. defined by a point, width, and height) and / or a class label for each bounding box. The machine learning model may be trained using input image(s) including material storage objects and / or reference image® of the respective area as input and output / or data. The deep learning model may be a Region-Based Convolutional Neural Network such as R-CNN, Fast R-CNN and Faster-RCNN.

[0090] In the next step, an object identifier 305 (as shown in Fig. 3e), which may be associated with the detected material storage objects, may be read using an optical character recognition model. The object identifier may be included in the input data such as the input images. The object identifier may be located on the detected material storage objects. The object identifier may be a number located on the detected material storage objects such as a number located at the top of a railcar. The object identifier of the detected material storage object associated with the geolocation of the one or more input images may be stored. The identifier of the detected object associated with the geolocation of the one or more input image(s) and / or the one or more detected objects may be the output of the algorithm.

[0091] In the next step, a center point 302 of the bounding boxes of the objects such as rail cars may be calculated. The center point of the object may be the midpoint of the material storage object. The midpoint of the object may be a point that may be equidistant to at least two points of the circumference or surface of the object. The circumference of the object may be defined by the bounding box.

[0092] The static characteristics 306 of the respective reference image and / or image set may be provided. The static characteristics may be associated with a static characteristics identifier. The static characteristics 306 associated with the static characteristic identifiers may be collected. The static characteristics 306 may be collected based on the reference data, the geographic metadata and / or the input data. The one or more material storage objects 303 and the static characteristics 306 may be matched. The object position of the one or more material storage objects 303 and the static characteristics 306 linked to the position may be matched. Matching may include determining the highest probability of the position of the at least one material storage object at the static characteristic. Highest probability may relate to a minimum distance of the center point(s) of the material storage object(s) to the static characteristic(s) . Highest probability of the position of the at least one material storage object at the static characteristic may relate to the material storage object having a larger distance of the center point to other static characteristic(s). Matching may include determining the closest static characteristic to which the object may be linked. Matching may include calculating a distance of the of the material storage object(s) in relation to the static characteristic(s). Matching may include calculating a rotation of the bounding box and matching this to a static characteristic(s) in order to determining the probability for linking of the material storage object to the respective static characteristic 307. The center point(s) of the material storage objects 302 and the respective static characteristics 306 may be matched. Matching may include calculating a distance of the of the center point(s) of the material storage object(s) in relation to the static characteristics). Matching may include superposing the center points of the material storage objects and the static characteristics. Matching may include calculating a distance of the at least one center point of the at least one material storage object in relation to the at least one static characteristic. Matching may include determining the shortest distance based on the calculation.

[0093] The result of the matching may be provided via a communication interface. The result of the matching may be displayed to the user. A data file may be generated based on the steps described in Fig. 2a-2c and / or 3a-b. The data file may include the object identifier, static characteristic identifier and the geographic metadata of the objects. The data may include the object identifier and the track(s). The data file may be a json file. The data file may be provided via a communication interface. The result of the matching may be displayed to the user. The result may be displayed within a table listing static characteristics and the linked material storage object and / or the one or more object identifiers associated with a material stored in the material storage object. The user interface may display a list of all object identifiers linked to the respective static characteristic identifier. The data file may support the monitoring of the storage area. For example, an operator may search for an object using the user interface to determine the exact location of the object.

[0094] Fig. 4 illustrates an example of a method for monitoring and / or controlling one or more material storage objects in a material storage area associated with a production network.

[0095] In block 401 image data of the material storage area including one or more material storage objects, wherein the image data is related to geographic metadata indicating a position of the area where the image data was captured, may be provided. Image data may relate to aerial image data of the area. Image data may relate to aerial image data of the area being captured by at least one dynamic system such as a drone. The image data may relate to geographic metadata indicating position, where the image was captured. Image data may relate to one or more input images. Image data may be generated by dynamic systems. Dynamic systems may relate to moving systems capturing the input image(s) such as dynamic image(s). The dynamic system may include at least a camera system being configured to acquire images. The images may be aerial images. The area may relate to a material storage area associated with the production network, e.g. a chemical production network. The material storage area may relate to an area of the production network. The material storage area may relate to an area, where the objects including mated al (s) are present until required, e.g. by a plant for producing a product or by a costumer. The one or more material storage objects in the area may relate to objects storing material(s) associated with the production network as described in the context of Fig. 1 . The input image(s) may be an image of the material storage area including one or more static characteristics associated with the infrastructure of the material storage area. The input image(s) may include at least one material storage object in the material storage area. The one or more static characteristic(s) may be indicative of a possible position of the one or more material storage object(s) according to the infrastructure.

[0096] In block 402 reference data of the material storage area may be provided. Reference data may relate to geographic metadata and include one or more static characteristics associated with the infrastructure of the material storage area, wherein the one or more static characteristics are indicative of a possible position of the one or more material storage object according to the infrastructure. Reference data of the area may be generated as described in the context of Fig. 2a-2c. The reference data may include one or more static characteristics associated with the infrastructure of the material storage area. A static characteristic may have a unique link to a static identifier indicating the position of the static characteristic and / or the object in combination with geographic metadata. The reference data may be provided in association with the metadata such as the geographic coordinates, static characteristic identifier, further IDs and / or a time stamp indicating the time when the image was captured. The indication of a possible position of the at least one object may be provided by the static characteristic as described in the context of Fig. 2a-c and Fig. 3a-e.

[0097] In block 403 an object position of the one or more material storage objects in the image as captured may be matched with the one or more static characteristics linked to the position. In other words, the one or more material storage objects in the image as captured may be matched with the one or more static characteristics in such a way that the one or more objects are linked to the one or more static characteristics. Matching may include allocating the one or more material storage objects to the one or more static characteristics as described in the context of Fig. 3a-e. The matching may allow to track the position with a high accuracy by linking the one or more objects to the respective one or more static characteristics. Therefore, the object(s) may be linked to the static characteristic with the highest probability for the position of the object(s). Matching may include calculating a distance of the at least one center point of the at least one material storage object in relation to the at least one static characteristic. The object(s) may be linked to the static characteristic having the shortest distance to the center point of bounding box of the one or more material storage object(s).

[0098] In block 404 one or more object identifiers associated with a material stored in the material storage object may be detected. Detection of the at least one object identifier may be performed as part of an object detection algorithm as described in the context of Fig. 3e. The object identifier may be read using optical character recognition. An object identifier may be uniquely linked to a material stored in the material storage object. An object identifier may be uniquely linked to a material stored in the material storage object and a material storage object. The object identifier may be located on the material storage object. The object identifier may be captured in the image data. In block 405 object identifier(s) linked to the static characteristic(s) for monitoring and / or controlling material storage object(s) may be provided. The object identifier(s) linked to the static characteristic(s) may be provided via a communication interface. The object identifier(s) linked to the static characteristic(s) may be displayed to the user. A user interface may display the one or more object identifiers linked to the one or more static characteristics. The object identifier(s) linked to the one or more static characteristic identifier(s) may be provided in order to monitor the position of the material(s) stored in the material storage object. The linking may be provided for monitoring and / or controlling the production of a product requiring material inputs stored in the material storage object(s) based on the position of the material(s) stored in the material storage object(s)as described in the context of Fig. 5. Based on the provided linking the controlling may include instructions for relocating one or more material storage objects. The relocation may influence the relocation of one or more additional material storage objects based on the linking the static characteristic. Based on the provided linking, the material(s) stored in the material storage object(s) may be provided in time to a production process requiring material input(s) as described in the context of Fig. 1.

[0099] Fig. 5 illustrates an example for a method for monitoring and / or controlling one or more material storage object(s) in a material storage area for use as one or more input material(s) for a production of one or more product(s).

[0100] In block 501 input material data associated with the production of a product requiring material inputs may be provided. Input material data may relate to an input material quantity, input material properties or the like, which may be required for the production process. The input material data associated with the production of a product requiring material inputs may relate to the production of the product requiring input material(s). The input material data associated with the production of a product requiring material inputs may include the production data, production site data such as the location and the like.

[0101] In block 502 one or more object identifiers associated with one or more materials stored in the material storage object may be provided. The object identifier may be uniquely linked to the material storage object and may be detected using an optical character recognition model as described in context of Fig. 3e.

[0102] In block 503 based on the one or more object identifiers associated with one or more materials stored in the material storage object, the material data associated with the material stored in the material storage object and the linked one or more static characteristics indicative of the position of the one or more material storage objects as generated according to the context of Fig. 2a-c, Fig. 3a-e and Fig.4 may be gathered. Through the linking, an exact position of the object on the static characteristic may be assigned and provided. The material data associated with the material stored in the material storage objects may relate to a material quantity, shelf life, material properties or the like, which may be required for a production process. The material data associated with the material stored in the material storage objects may relate to the geographic metadata of the material storage object. The material data associated with the material stored in the material storage objects may include the position of the material storage object.

[0103] In block 504 the input material data associated with the production of a product requiring material inputs may be matched with the material stored in the one or more material storage objects. Matching may include providing one or more materials stored in the material storage object being suitable as input material(s) for the production. Matching may include matching the input material type of the input material data associated with the production of a product requiring material inputs with the material type of the material data associated with the material stored in the one or more material storage objects. Matching may include matching the characteristics of the input material data associated with the production of a product requiring material inputs with the characteristics of the material data associated with the material stored in the one or mor material storage objects. Matching may include matching the input material quantity of the input material data associated with the production of a product requiring material inputs with the material quantity of the material data associated with the material stored in the one or mor material storage objects. Matching may include calculating and / or selecting a minimum deviation between the input material data associated with the production of a product requiring material inputs and the material data associated with the material stored in the material storage object. Matching may include selecting one or more materials stored in a material storage object being inside a predefined value range of the material data being suitable for the production.

[0104] For example, the input material data of acrylic acid for the production of poly acrylic acid requiring input material(s) may be matched to the material data associated with the acrylic acid stored in one or more material storage objects. Therefore, the viscosity of the acrylic acid stored in one or more material storage objects may have to be between 1 .3 - 1 .4 at 20 °C and the vapor pressure may have to be 3 mmHg. Five tons of acrylic acid may be required for the production.

[0105] In block 505 based on the matching, one or more materials stored in the material storage objects associated with one or more object identifiers as one or more input materials for the production of a product requiring material inputs may be determined. The material stored in the material storage object relating to a minimum number of additional material storage objects to be moved. For example, if in Fig. 1 material storage objects 102 and 106 may contain the identical material having the same quality, then material storage object 106 may be selected because no additional material storage object may have to be moved. The material stored in the material storage having a minimum distance to the production site may be determined. The minimum distance to the production site may be determined based on the infrastructure of the material storage area. The minimum distance to the production site may be determined based on the infrastructure from the material storage area to the production site. In the example of Fig. 1 material storage object 106 may be selected.

[0106] In block 506 one or more object identifiers associated with the material stored in the material storage object linked to the one or more static characteristics for monitoring and / or controlling the material stored in a material storage object may be provided. The object identifier(s) linked to the static characteristic(s) may be provided via a communication interface. The object identifier(s) linked to the static characteristic(s) may be displayed to the user. A user interface may display the one or more object identifiers linked to the one or more static characteristics. Monitoring and / or controlling data may be generated based on the one or more object identifiers associated with the material stored in the material storage object linked to the one or more static characteristics. Monitoring and / or controlling data may be generated for providing the material stored in the material storage objects to the production of a product using material inputs. Monitoring and / or controlling data may be generated based on the linking of the material storage objects to the one or more static characteristics. Monitoring and / or controlling data may be generated based on the infrastructure from the material storage area to the production site. Monitoring and / or control data may relate to the position of the material storage object storing the input material. The material storage object may be relocated based on the position. The relocation may influence further material storage objects linked to the same static characteristic or adjacent static characteristics(s). The monitoring and / or controlling data for monitoring and / or controlling the one or more material(s) stored in the material storage object(s) may be provided to the production system for producing the product. The monitoring and / or controlling data may be provided via a communication interface. The monitoring and / or controlling data may be displayed to the user. A user interface may display the monitoring and / or controlling data.

[0107] The present disclosure has been described in conjunction with preferred embodiments and examples as well. However, other variations can be understood and effected by those persons skilled in the art and practicing the claimed invention, from the studies of the drawings, this disclosure and the claims.

[0108] Any steps presented herein can be performed in any order. The methods disclosed herein are not limited to a specific order of these steps. It is also not required that the different steps are performed at a certain place or in a certain computing node of a distributed system, i.e. each of the steps may be performed at different computing nodes using different equipment / data processing.

[0109] As used herein ..determining" also includes ..initiating or causing to determine", "generating" also includes ..initiating and / or causing to generate" and "providing” also includes "initiating or causing to determine, generate, select, send and / or receive”. "Initiating or causing to perform an action” includes any processing signal that triggers a computing node or device to perform the respective action.

[0110] In the claims as well as in the description the word "comprising” does not exclude other elements or steps and the indefinite article "a” or "an” does not exclude a plurality. A single element or other unit may fulfill the functions of several entities or items recited in the claims. The mere fact that certain measures are recited in the mutual different dependent claims does not indicate that a combination of these measures cannot be used in an advantageous implementation.

[0111] Any disclosure and embodiments described herein relate to the methods, the apparatus, devices, the computer program element lined out above and vice versa. Advantageously, the benefits provided by any of the embodiments and examples equally apply to all other embodiments and examples and vice versa.

[0112] All terms and definitions used herein are understood broadly and have their general meaning.

Claims

CLAIMS1 . A method for monitoring and / or controlling one or more material storage object(s) in a material storage area associated with a production network, the method comprising: providing image data of the material storage area including one or more material storage object(s), wherein the image data is related to geographic metadata indicating a position of the area where the image data was captured; providing reference data of the material storage area, wherein the reference data is related to geographic metadata and including one or more static characteristic(s) associated with the infrastructure of the material storage area, wherein the one or more static characteristics are indicative of a possible position of the one or more material storage objects according to the infrastructure; matching an object position of the one or more material storage object(s) in the image as captured with the one or more static characteristic(s) linked to the position; detecting one or more object identifier(s) associated with a material stored in the material storage object; providing the one or more object identifier(s) linked to the one or more static characteristic(s) for monitoring and / or controlling one or more material storage object(s).

2. The method of any of the preceding claims, wherein the reference data of the material storage area is selected from a set of reference data of the material storage area by matching the related geographic metadata of the reference data with related geographic metadata of the of the one or more input image(s).

3. The method of any of the preceding claims, wherein the one or more static characteristic(s) associated with the infrastructure of the material storage area are derived from the reference data of the material storage area.

4. The method of any of the preceding claims, wherein a perspective transformation is applied to the one or more image data based on the reference data by selecting one or more feature point(s), extracting one or more feature descriptor(s) and matching the one or more feature(s).

5. The method of any of the preceding claims, wherein the one or more object identifier(s) associated with a material stored in the material storage object are detected employing an optical character recognition model.

6. The method of any of the preceding claims, wherein matching the object position of the one or more material storage object(s) in the image as captured with the one or more static characteristic(s) linked to the position includes determining a probability of the position of the one or more material storage object(s) in relation tothe one or more static characteristic(s) and / or assigning the one or more material storage object(s) with a highest probability to the static characteristic(s).

7. The method of any of the preceding claims, wherein matching the object position of the one or more material storage object(s) in the image as captured with the one or more static characteristic(s) linked to the position includes assigning the one or more material storage object(s) to the closest static characteristic.

8. The method of any of the preceding claims, wherein matching the object position of the one or more material storage object in the image as captured with the one or more static characteristic(s) linked to the position includes determining one or more center point(s) of the one or more material storage object(s) and calculating a distance of the one or more center point(s) of the one or more material storage object(s) in relation to the one or more static characteristic(s).

9. An apparatus for monitoring and / or controlling one or more material storage object(s) in a material storage area associated with a production network, the apparatus comprising: an image data providing interface configured to provide image data of the material storage area including one or more material storage object(s), wherein the image data is related to geographic metadata indicating a position of the area where the image data was captured; a reference data providing interface configured provide reference data of the material storage area, wherein the reference data is related to geographic metadata and including one or more static characteristics) associated with the infrastructure of the material storage area, wherein the one or more static characteristic(s) are indicative of a possible position of the one or more material storage object(s) according to the infrastructure; a matching unit configured to match an object position of the one or more material storage object(s) in the image as captured with the one or more static characteristic(s) linked to the position; a detector configured to detect one or more object identifier(s) associated with a material stored in the material storage object; an identifier providing interface configured provide the one or more object identifier(s) linked to the one or more static characteristic(s) for monitoring and / or controlling one or more material storage object(s).

10. A method for monitoring and / or controlling one or more material storage object(s) in a material storage area for use as one or more input material(s) for a production of one or more product(s), the method comprising the steps: providing input material data associated with the production of one or more product(s) using one or more material input(s); providing one or more object identifier(s) associated with one or more material(s) stored in the one or more material storage object(s);gathering, based on the one or more object identifier(s) associated with the one or more material(s) stored in the one or more material storage object(s), material data associated with the one or more materials) stored in the one or more material storage object(s) and the linked one or more static characteristics) indicative of the position of the one or more material storage object(s) as provided according to the method of claims 1 to 8 and the apparatus of claim 9; matching the input material data associated with the production of one or more product(s) using one or more input material(s) with the material data associated with the one or more material(s) stored in the one or more material storage object(s); determining, based on the matching, one or more material(s) stored in the one or more material storage object(s) associated with one or more object identifier(s) as one or more input material(s) for the production of the one or more product(s) using one or more input material(s); providing the one or more object identifiers associated with the one or more material(s) stored in the one or more material storage object(s) linked to the one or more static characteristic(s) for monitoring and / or controlling the one or more material(s) stored in the one or more material storage object(s) for use as one or more input material(s) for the production of the one or more product(s).

11. The method of claim 10, wherein determining, based on the matching, one or more material(s) stored in the one or more material storage object(s) associated with one or more object identifier(s) as one or more input material(s) for the production of the one or more product(s) using one or more input material(s) includes selecting the one or more material(s) stored in one or more material storage object(s) relating to a minimum number of additional one or more material storage object(s) to be moved.

12. The methods of claim 10 and 11, wherein determining, based on the matching, one or more material(s) stored in the one or more material storage object(s) associated with one or more object identifier(s) as one or more input material(s) for the production of the one or more product(s) using one or more input material(s) includes selecting the one or more material(s) stored in the one or more material storage object(s) having a minimum distance to the production site.

13. A method for producing one or more product(s) by using the one or more object identifier(s) as determined according to claim 10 for monitoring and / or controlling the one or more material(s) stored in the one or more material storage object(s) and providing the one or more material(s) stored to the production system for the production of the one or more product(s).

14. An apparatus for monitoring and / or controlling one or more material storage object(s) in a material storage area for use as one or more input material(s) for a production of one or more product(s), the apparatus comprising: an input material data providing interface configured to provide input material data associated with the production of one or more product(s) using one or more material input(s);an identifier providing interface configured to provide one or more object identifier(s) associated with one or more material(s) stored in the one or more material storage object(s); a gathering unit configured to gather, based on the one or more object identifier(s) associated with the one or more material(s) stored in the one or more material storage object(s), material data associated with the one or more material(s) stored in the one or more material storage object(s) and the linked one or more static characteristic(s) indicative of the position of the one or more material storage object as provided according to the method of claims 1 to 8 and the apparatus of claim 9; matching the input material data associated with the production of one or more product(s) using one or more input material(s) with the material data associated with the one or more material(s) stored in the one or more material storage object(s); a determining unit configured to determine, based on the matching, one or more material(s) stored in the one or more material storage object(s) associated with one or more object identifier(s) as one or more input material(s) for the production of the one or more product(s) using one or more input material (s); a data providing interface configured to provide the one or more object identifiers associated with the one or more material(s) stored in the one or more material storage object(s) linked to the one or more static characteristic® for monitoring and / or controlling the one or more material(s) stored in the one or more material storage object(s) for use as one or more input material(s) for the production of the one or more product(s).

15. Use of the one or more object identifier(s) associated with the respective one or more material(s) stored in the one or more material storage object(s) linked to the one or more static characteristic®, wherein the linking is provided according to the claims of 1 to 9, for generating monitoring and / or control data for the one or more material(s) stored in the one or more material storage object(s) based on the linking.

Images