Automated process monitoring

Abstract

Methods for automated process monitoring, wherein - image data (4) depicting a scene are generated by means of a camera system (2); - by means of a computing unit (3) based on the image data (4) it is determined that objects (6a, 6b, 7a, 7b, 7c, 8, 9, 10, 11, 14) are located within a specified area (B1, B2); - by means of the computing unit (3) for each of the objects (6a, 6b, 7a, 7b, 7c, 8, 9, 10, 11, 14) based on the image data (4) one of at least two predefined object classes is determined; - using the computing unit (3) depending on the specific object classes, it is checked whether a predefined rule assigned to the area (B1, B2) is fulfilled; - by means of the computing unit (3) an output signal is generated depending on a result of the check; - using the computing unit (3) based on the image data (4) for each of the objects (6a, 6b, 7a, 7b, 7c, 8, 9, 10, 11, 14) one of at least two predefined sub-areas (B1.1, B1.1.1, B1.2, B2.1, B2.1.1, B2.2, B2.2.1) of the area (B1, B2) is determined within which the respective object (6a, 6b, 7a, 7b, 7c, 8, 9, 10, 11, 14) is located; and - the verification of whether the rule is fulfilled is carried out depending on the specific sub-areas (B1.1, B1.1.1, B1.2, B2.1, B2.1.1, B2.2, B2.2.1).

Description

[0001] The present invention relates to a method and a device for automated process monitoring, a production method and a computer program product.

[0002] To ensure quality requirements in industrial or manufacturing processes, and to guarantee process stability, it is necessary to monitor the correct execution of these processes. This can be the case, for example, in order picking or sequencing processes, or in assembly or installation processes. Currently, this requires a significant amount of manual monitoring. Other monitoring or control procedures, such as those for theft prevention or monitoring access restrictions, also require a significant amount of manual monitoring.

[0003] Document US 2013 / 0182890A1 describes a device for detecting people on a conveyor belt. Camera image data is analyzed to identify one or more objects and then classify them. Actions are initiated if the object corresponds to a specified class of objects. In particular, the system can react if a person enters a specific area.

[0004] US patent 2015 / 0138332A1 discloses a lens tube consisting of an outer tube surrounding the lens holder frame. The lens holder frame holds the lens and is moved along the optical axis by a guide. An actuator performs this movement. Position is determined by a detector with a scale along the optical axis and an opposing sensor. The actuator is controlled based on the position data acquired by the detector. Either the lens holder frame holds the scale or the sensor, while the outer tube carries the other element. The element on the lens holder frame is positioned at a location corresponding to a vibration node of the lowest natural frequency to minimize vibrations.

[0005] US Patent 2007 / 0035622A1 describes a video surveillance method that detects and analyzes movement within a monitored area. Instead of using simple pixel changes or simple motion vectors, it employs a spatiotemporal signature model. This model utilizes multiple features (feature vectors) that describe both spatial and temporal properties of an object.

[0006] US patent 2013 / 0147790A1 describes a system that captures and processes stereo images to determine the position of an object (e.g., a user). This positional information is used to enable gesture control for computer applications.

[0007] US Patent 2012 / 0087539A1 discloses a method for detecting feature points (e.g., head, shoulders, limbs) of an object for 3D motion capture. The goal is to precisely determine the position and depth of these points in order to capture movements for applications such as motion capture or interactive games.

[0008] Against this background, it is an object of the present invention to provide an improved concept for automated process monitoring, which reduces manual control effort.

[0009] This problem is solved by a method according to claims 1, 8 and 10 or by the device according to claim 9.

[0010] The improved concept is based on the understanding that process monitoring often requires determining whether specific objects are located within certain predefined areas. Therefore, according to this improved concept, object classes are defined for objects within a predefined area based on image data from a camera system, and these object classes are then used to check whether a corresponding rule is fulfilled.

[0011] According to the improved concept, a method for automated process monitoring is described, in which image data depicting a scene is generated using a camera system. A processing unit determines, based on the image data, whether objects are located within a predefined area. For each object, the processing unit determines one of at least two predefined object classes based on the image data. Depending on the determined object class, the processing unit checks whether a predefined rule associated with the area is fulfilled. Based on the image data, the processing unit determines, for each object, one of at least two predefined sub-areas of the area within which the respective object is located. The rule fulfillment check is performed based on the determined sub-areas.Depending on the result of the check, the processing unit generates an output signal.

[0012] In particular, the processing unit can analyze the scene based on the image data and thus determine the presence of objects. The specified area is, in particular, a predefined two-dimensional region on an image corresponding to the image data. The camera system has, in particular, a predefined pose relative to the scene, such that the specified area corresponds to a corresponding real-world region within the scene. Specifically, the area can correspond to a sub-region of the image, i.e., not the entire image, where the sub-region is defined, for example, by its position and / or location within the image.

[0013] The rule assigned to the scope can be understood, for example, as a requirement that a certain number of objects of a specific object class must be located in the scope or in one or more sub-scopes of the scope. If this is the case, the rule is fulfilled; otherwise, it is not. The rule can also contain several such requirements, and the rule is fulfilled if any one of the requirements is met.

[0014] The output signal contains, in particular, information about whether the rule is fulfilled or not. Alternatively, the output signal can be generated only if the rule is fulfilled, or only if the rule is not fulfilled.

[0015] To determine the object classes for the objects, known image analysis algorithms or object detection algorithms can be used. For example, algorithms based on computer vision or machine-trained algorithms can be employed. For instance, well-known architectures of artificial neural networks, especially convolutional neural networks (CNNs), can be used.

[0016] The improved concept thus enables automated process monitoring, reducing the need for manual monitoring steps.

[0017] For example, depending on the output signal or based on a visual representation of the output signal, a user can manually release the scene, or the release can be automated based on the output signal.

[0018] The improved concept is advantageous because it avoids directly determining whether a rule is fulfilled based on the image data. Such an approach could be implemented, for example, using neural networks or other machine-trainable algorithms by providing large amounts of training data and annotating or labeling it according to the rule. However, in this case, a change to the rule would necessitate a complete retraining of the algorithm, which would be extremely resource- and time-consuming.

[0019] With the improved concept, it is only necessary to determine, for example using a machine vision algorithm or a machine-trained algorithm, which objects are located where in the scene. This information is then evaluated against the predefined rule, independent of the actual object recognition. If the rule is changed, only the last step, namely the comparison of the specific object classes with the predefined rule, needs to be adjusted. The algorithm that determines that the objects are in the area or that identifies the object classes does not need to be changed. In particular, the algorithm does not need to be retrained if it is a machine-trained algorithm.

[0020] According to at least one embodiment of the method, an object recognition algorithm is applied to the image data using the computing unit to determine that the at least one object is located within the specified area and to determine the object class for each of the objects. The object recognition algorithm can, for example, include an artificial neural network, in particular a CNN.

[0021] Since sophisticated algorithms for object recognition and object classification are known from the prior art, which are based on artificial neural networks, especially CNNs, the reliability of process monitoring can also be increased in such embodiments.

[0022] The sub-areas can themselves contain further sub-areas. These hierarchically lower-level sub-areas can also be referred to as sub-areas of the area.

[0023] For each of the at least two sub-areas, a position and / or location within the image or area is specified. Each of the at least two sub-areas is uniquely assigned an identifier. This identifier makes it possible to determine whether the respective sub-area is located within one or more other sub-areas and, if so, which ones. This assignment can, for example, be represented in a tree structure. Determining a sub-area can be understood as determining the identifier of the respective sub-area.

[0024] To verify whether the rule is fulfilled, the system specifically checks which objects are located in which sub-areas. This allows even more complex scenes to be monitored using this method, or more complex rules to be checked.

[0025] According to at least one embodiment, to verify whether the rule is fulfilled, it is checked whether a distribution of the specific object classes within the area corresponds to a predetermined target distribution.

[0026] In other words, the position of each object is determined, and the rule is used to check whether objects of certain object classes are located in certain parts of the area according to the target distribution.

[0027] According to at least one embodiment, to verify whether the rule is fulfilled, it is checked whether a distribution of the specific object classes on the at least two sub-areas corresponds to the specified target distribution.

[0028] In other words, the system checks whether a specific number of objects of a particular object class are located in a sub-area defined by a target distribution. The target distribution specifies, in particular, how many objects of a specific object class must be in each of the at least two sub-areas for the rule to be fulfilled. Multiple target distributions can be specified, and the rule is fulfilled, for example, if at least one of the target distributions is actually present.

[0029] According to at least one embodiment, in order to determine that the objects are within the area, an envelope is determined for each of the objects using the computing unit, and it is determined that the respective envelope is within the area.

[0030] In corresponding embodiments, in order to determine the sub-area for each of the objects, the computing unit can be used to determine the envelope shape for each of the objects and to ascertain that the respective envelope shape is located within the corresponding sub-area.

[0031] The bounding box, also known as the bounding figure, is a two-dimensional geometric shape that approximately represents the dimensions of the object in question or that encloses the object. For example, the bounding box can be defined by specifying its position and size. Its shape can also be determined or predetermined; for instance, it might be a rectangle.

[0032] In particular, the enveloping shape can be determined using the object recognition algorithm, for example, using a machine vision algorithm or a machine-trained algorithm. Determining enveloping shapes for objects within the framework of object recognition is an established approach that can be performed with high accuracy and reliability. This increases the reliability of process monitoring.

[0033] For example, the object recognition algorithm determines the object classes and the bounding figures or their positions. Whether the bounding figures are located within the area or the respective sub-areas is not necessarily determined by the object recognition algorithm.

[0034] According to at least one embodiment, in order to determine that the respective envelope figure is located within the area, a position of the envelope figure is compared with a predetermined binary image by means of the computing unit, wherein the binary image defines the area.

[0035] The binary image is, in particular, a two-dimensional arrangement of cells in a rectangular grid, where each cell of the rectangular grid corresponds to a pixel of the binary image. Each pixel can assume exactly one of two logical values, where, for example, the pixel assumes a first logical value if the corresponding pixel lies within the area, and a second logical value otherwise.

[0036] This makes it particularly easy to determine whether the outline, and therefore the object, is located within the defined area. This also reduces computational effort. In embodiments where sub-areas for the objects are defined, it can be determined, for example, that the respective outline is located within the corresponding sub-area by comparing the outline's position with the binary image.

[0037] For example, each pixel in the binary image can be assigned an identifier that specifies which subdomain the respective pixel belongs to. In the case of multi-level subdomain structures, i.e., if a subdomain itself contains another subdomain, each pixel might only be assigned the identifier of the lowest-level subdomain, since this allows the pixel's position to be reconstructed with respect to all higher-level subdomains, particularly using the identifiers.

[0038] According to at least one embodiment, the computing unit determines the position of a distinguished point of the envelope figure as the position of the envelope figure.

[0039] Depending on the shape of the bounding figure, the distinguished point can be, for example, its center point, centroid, or vertex. Preferably, the bounding figure is a rectangle, and the distinguished point is the center of the rectangle.

[0040] The designated point can, for example, be assigned exactly one pixel of the binary image. Based on the identifier of this pixel, it can then be determined that the shape or object is located within that area and, if applicable, within which sub-areas.

[0041] By simply determining the distinguished point of the enveloping figure and comparing its position with the binary image, it is possible to determine where the respective object is located in the scene with minimal computational and storage effort.

[0042] According to at least one embodiment, a visual representation of the area is displayed by means of an electronic display device, depending on the output signal.

[0043] For example, the formatting, such as color, of the visual representation can depend on the output signal. In particular, different formatting can be chosen depending on whether the output signal indicates that the rule is fulfilled or not. Alternatively or additionally, different symbols relating to the area can be displayed to indicate whether the rule is fulfilled. In such implementations, the result of process monitoring is presented in a way that is particularly quick and intuitive for humans to perceive.

[0044] In various embodiments, corresponding visual representations of the sub-areas can also be displayed by means of the display device depending on the output signal.

[0045] The improved concept also specifies a production method. This production method includes a process that is monitored using an automated process monitoring procedure according to the improved concept.

[0046] According to at least one embodiment of the production process, the process monitored using the improved process monitoring concept is a sequencing process, a picking process, or an assembly process.

[0047] The sequencing process can, for example, include packing a shopping cart, and the automated process monitoring procedure according to the improved concept checks the completeness of the shopping cart.

[0048] In the case of an assembly process, for example, it can be checked whether certain parts have been assembled in the correct place in a corresponding area.

[0049] However, the automated process monitoring method can also be used outside of production processes, for example to monitor access restrictions or to prevent theft.

[0050] According to the improved concept, a device for automated process monitoring is also specified. The device includes a processing unit and an interface configured to receive image data depicting a scene from a camera system or storage unit and transmit it to the processing unit. Based on the image data, the processing unit is configured to determine whether objects are located within a predefined area. For each object, the processing unit is configured to determine one of at least two predefined object classes based on the image data. Depending on the determined object class, the processing unit is configured to verify whether a predefined rule associated with the area is fulfilled and, depending on the result of the verification, to generate an output signal.

[0051] The interface can include one or more hardware components and / or one or more software components. For example, the interface can be part of the processing unit. The interface can be designed for wireless or wired data exchange between the processing unit and the camera system, or between the processing unit and the storage unit.

[0052] According to at least one embodiment of the device, the device contains the camera system or the storage unit and the camera system or the storage unit is coupled to the computing unit via the interface.

[0053] Further embodiments of the device according to the improved concept follow directly from the various configurations of the automated process monitoring method according to the improved concept and the production method according to the improved concept, and vice versa. In particular, a device according to the improved concept may be configured to perform an automated process monitoring method according to the improved concept, or the device may perform such a method.

[0054] According to the improved concept, a computer program with instructions is also specified. When the instructions or the computer program are executed by a processing unit, the instructions cause the processing unit to perform an automated process monitoring procedure, specifically according to the improved concept. This automated process monitoring procedure includes receiving image data depicting a scene and determining, based on the image data, that objects are located within a predefined area. The automated process monitoring procedure also includes determining, for each object based on the image data, one of at least two predefined object classes and, depending on the determined object class, checking whether a predefined rule associated with the area is fulfilled. Finally, an output signal is generated depending on the result of this check.

[0055] According to the improved concept, a computer-readable storage medium is also specified that stores a computer program according to the improved concept.

[0056] The computer program and the computer-readable storage medium can be referred to as the respective computer program products with the commands.

[0057] The invention also includes combinations of the features of the described embodiments.

[0058] The following describes exemplary embodiments of the invention. To this end, we will show: Fig. 1 a schematic representation of an exemplary embodiment of a device for automated process monitoring according to the improved concept; and Fig. 2A a schematic representation of a process step of an exemplary embodiment of a method for automated process monitoring according to the improved concept; Fig. 2B a schematic representation of a further procedural step of the procedure from Fig. 2A; Fig. 2C a schematic representation of a further process step of the process from Fig. 2A; Fig. 2D a schematic representation of a further process step of the process from Fig. 2A; and Fig. 2E a schematic representation of a further procedural step of the procedure from Fig. 2A.

[0059] The embodiments described below are preferred embodiments of the invention. In these embodiments, the described components each represent individual features of the invention that can be considered independently of one another. Each of these features further develops the invention independently and can therefore be considered part of the invention individually or in a combination other than that shown. Furthermore, the described embodiments can also be supplemented by other features of the invention already described.

[0060] In the figures, functionally identical elements are each provided with the same reference symbols.

[0061] In Fig. Figure 1 schematically illustrates an exemplary embodiment of a device 1 for automated process monitoring according to the improved concept. The device 1 has a computing unit 3 which includes an interface for connecting the computing unit 3 wirelessly or via a wired connection to a camera 2. The camera 2 can, for example, also be part of the device 1. The camera 2 can image a scene and, based on this, generate image data 4 and transmit it to the computing unit 3 via the interface. Optionally, the device 1 can have a display device 5, for example, a display or the like, which is coupled to the computing unit 3.

[0062] The functionality of device 1 is described below using exemplary embodiments of a method for automated process monitoring according to the improved concept with reference to the figures. Fig. 2A to 2E explained in more detail.

[0063] The Fig. Figures 2A to 2E schematically illustrate the steps of a picking process in which the automated process monitoring procedure is used to check the completeness of sequencing boxes. However, this should not be interpreted as a limitation of the improved concept's scope. In particular, the improved concept can be used to monitor any process that aims to ensure that specific objects are located in specific positions, provided these objects are accessible to camera surveillance.

[0064] Prior to implementing the automated process monitoring procedure according to the improved concept, a one-time process definition can be performed. For example, a still image from camera 2 can be saved, and a suitable algorithm, such as an artificial neural network or another object classification algorithm, can be selected to recognize objects expected in the process.

[0065] To define a suitable workspace, it's possible to specify the areas of the scene where objects are expected to be located. These areas can be drawn onto the still image using a grid. The granularity or resolution of the grid can be adjusted as needed. Sub-areas can also be defined within these areas using a tree structure. Areas on the same plane do not overlap. To represent the areas and sub-areas in the grid, a binary image and, optionally, a polygon can be stored in a database using processing unit 3. The binary image has, for example, the same size as the grid, i.e., a number of pixels corresponding to a number of cells in the grid used to define the workspace. For each pixel of the binary image, an identifier for the lowest sub-area is stored.The relationship to the higher-level areas is thus defined via the corresponding tree structure. The optional polygon representation can later be used to render a status monitor.

[0066] Furthermore, it can be defined which objects are expected for which area or sub-area. One or more objects can be specified for each area or sub-area. It is also possible to specify that no object is expected in a particular area or sub-area. Based on these definitions, a rule can be defined for each area and its sub-areas, if any, which determines whether the scene is correct or not. In particular, exactly one rule is defined for each area that is not a sub-area of ​​another area. The rule defines how many objects of which class are expected in the area and, if applicable, in which sub-areas of the area.

[0067] In an exemplary example, a seat belt is expected in a given area, and screws and / or nuts are expected in sub-areas of that area. The rule can then define, for example, that the respective area is complete and correct if the seat belt is present in the area, a certain number of screws are present in a first sub-area, and a certain number of nuts are present in a second sub-area.

[0068] Multiple rules or sub-rules can be defined for a single area. A rule is considered fulfilled when any one of its rules or sub-rules is met. For example, a global rule might stipulate that a scene is considered complete when the respective rule is fulfilled for all areas.

[0069] In this way, the process definition can be adapted very flexibly to the respective circumstances expected according to the process.

[0070] A status monitor can be displayed using the optional display device 5. The status monitor can be rendered from the stored polygons of the areas and sub-areas. It can, for example, schematically represent the areas and sub-areas and display them to a user. The status monitor can show, for instance, which areas are complete, i.e., for which areas the rule is fulfilled. If an error or deviation is detected, meaning the rule is not fulfilled, the user can take timely corrective action. In one exemplary embodiment, an area for which the rule is fulfilled is displayed in green; if the rule is not fulfilled, it is displayed in red.

[0071] After the process definition, a procedure for automated process monitoring can be carried out according to the improved concept. For this purpose, an image of the scene is taken using camera 2 and corresponding image data 4 is transmitted to the processing unit 3. In the example of the Fig. Figure 2A shows two large boxes 6a and 6b arranged side by side. The first box 6a contains a first small box 7a, located in the lower right corner of the large box 6a. The second large box 6b contains a second small box 7b, located, for example, in the upper part of the second large box 6b, and a third small box 7c, located, for example, in the lower part of the second large box 6b.

[0072] In the first small box 7a, in this example, there is a pointed chisel 11, and in the first large box 6a, outside the first small box 7a, there is a mandrel 10. In the second small box 7b, for example, there is a flat chisel 8, and in the third small box 7c, for example, there is a pin 9b. In the second large box 6b, outside the small boxes 7b and 7c, there is, for example, a toy car 14.

[0073] The objects described are of course purely exemplary and may differ depending on the design of the process to be monitored.

[0074] In the present example, areas and sub-areas can be defined according to the Fig. 2B must be defined. For example, the areas and sub-areas are each rectangular. However, this can vary depending on the use case. In the exemplary example of the Fig. In section 2B, two areas, B1 and B2, are provided, located adjacent to each other. The first area, B1, contains a rectangular sub-area of ​​the first level, B1.1, which in turn contains a rectangular sub-area of ​​the second level, B1.1.1. Another sub-area of ​​the first level, B1.2, of area B1 corresponds to area B1 without the sub-area B1.1.

[0075] The second area B2 contains two sub-areas of the first level B2.1 and B2.2. Sub-area B2.1 in turn contains a rectangular sub-area of ​​the second level B2.1.1, and sub-area B2.2 also contains a rectangular sub-area of ​​the second level B2.2.1. Another sub-area of ​​the first level B2.3 is given by area B2 without sub-areas B2.1 and B2.2.

[0076] For each of the areas B1 and B2, a specific rule is defined. This rule specifically concerns the expected distribution of objects of certain object classes across areas B1 and B2 and their respective sub-areas.

[0077] In the example described above, the rule for area B1 might specify, for instance, that a large box 6a must be located in area B1 and a small box 7a in sub-area B1.1. Sub-area B1.2 might contain a mandrel 10, and sub-area B1.1.1 a pointed chisel 11.

[0078] Accordingly, the rule for the second area B2 may stipulate that a large box 6b must be located in the second area B2, and a small box 7b, 7c must be located in sub-areas B2.1 and B2.2, respectively. For example, a flat chisel 8 must be located in sub-area B2.1.1, and a pen 9 in sub-area B2.2.1. For example, a toy car 14 must be located in sub-area B2.3.

[0079] Depending on the design of the process to be monitored, the rules may naturally vary. Furthermore, several rules may be specified for areas B1 and B2, which can be fulfilled alternatively.

[0080] The processing unit 3 then applies an object recognition or object classification algorithm, for example a CNN, to the image data 4 in order to determine a respective bounding box 12 and an associated object class for each of the objects, i.e., in particular for the boxes 6a, 6b, 7a, 7b, 7c as well as the further objects 8, 9, 10, 11, 14. Fig. Figure 2C shows an example of a bounding box 12 for the mandrel 10. For clarity, the other bounding boxes are not shown.

[0081] Using the processing unit 3, it is then determined, based on the bounding boxes 12, whether the detected objects are located within the areas or sub-areas specified by the rules. This is done as shown schematically in Fig. Represented in 2D, for example, the corresponding center point 13 is determined for each bounding box by identifying a corresponding cell of the grid or a corresponding pixel of the binary image. In the example of the Fig. For the sake of clarity, this is only shown in 2D for the Dorn 10.

[0082] Based on the defined object classes and the comparison of the center points 13 of the bounding boxes 12 with the binary image or the stored identifiers for the respective areas and sub-areas, the processing unit 3 can then check whether the respective rule for areas B1, B2 is fulfilled. Depending on this, the processing unit 3 can generate an output signal.

[0083] As in Fig.2E is schematically represented, and the display device 5 can be used to visually show for which of the areas B1, B2 and, optionally, for which of the sub-areas the regulations defined by the rules are met.

[0084] As described, the improved concept enables the automation of process monitoring for a wide variety of processes, thereby reducing manual control effort. The improved concept is universally applicable. Reference symbol list 1 Device 2 cameras 3 Computing Unit 4 Image data 5 Display unit 6a, 6b boxes Boxes 7a, 7b, 7c 8 flat chisels 9 pen 10 Dorn 11 pointed chisels 12 Bounding Box 13 Center 14 toy cars B1, B2 areas B1.1, B1.2, B2.1, B2.2, B2.3 subareas B1.1.1, B2.1.1, B2.2.1 Unterbereiche

Claims

Method for automated process monitoring, wherein: - image data (4) depicting a scene are generated by means of a camera system (2); - a computing unit (3) determines, based on the image data (4), that objects (6a, 6b, 7a, 7b, 7c, 8, 9, 10, 11, 14) are located within a predefined area (B1, B2); - for each of the objects (6a, 6b, 7a, 7b, 7c, 8, 9, 10, 11, 14), the computing unit (3) determines, based on the image data (4), one of at least two predefined object classes; - the computing unit (3) checks, depending on the determined object classes, whether a predefined rule assigned to the area (B1, B2) is fulfilled; - the computing unit (3) generates an output signal, depending on a result of the check; - the computing unit (3) determines, based on the image data (4) for each of the objects (6a, 6b, 7a, 7b, 7c, 8, 9, 10, 11, 14) one of at least two specified sub-areas (B1.1, B1.1.1, B1.2, B2.1, B2.1.1, B2.2, B2.2.1) of the area (B1, B2) within which the respective object (6a, 6b, 7a, 7b, 7c, 8, 9, 10, 11, 14) is located; and- the check to see if the rule is fulfilled is carried out depending on the specified sub-areas (B1.1, B1.1.1, B1.2, B2.1, B2.1.1, B2.2, B2.2.1). Method according to claim 1, wherein, to verify whether the rule is fulfilled, it is checked which of the objects (6a, 6b, 7a, 7b, 7c, 8, 9, 10, 11, 14) are located in which of the sub-areas (B1.1, B1.1.1, B1.2, B2.1, B2.1.1, B2.2, B2.2.1). Method according to one of the preceding claims, wherein, to verify whether the rule is fulfilled, it is checked whether a distribution of the specific object classes on the at least two sub-areas (B1.1, B1.1.1, B1.2, B2.1, B2.1.1, B2.2, B2.2.1) corresponds to a predetermined target distribution. Method according to one of the preceding claims, wherein, in order to determine that the objects (6a, 6b, 7a, 7b, 7c, 8, 9, 10, 11, 14) are located within the area (B1, B2), a circumscribing figure (12) is determined for each of the objects (6a, 6b, 7a, 7b, 7c, 8, 9, 10, 11, 14) by means of the computing unit (3); and it is determined that the respective circumscribing figure (12) is located within the area (B1, B2). Method according to claim 4, wherein, in order to determine that the respective envelope figure (12) is located within the area (B1, B2), a position (13) of the envelope figure (12) is compared with a predetermined binary image which defines the area (B1, B2) by means of the computing unit (3). Method according to claim 5, wherein a position of a distinguished point of the envelope figure is determined as position (13) of the envelope figure (12) by means of the computing unit (3). Method according to one of the preceding claims, wherein a visual representation of the area (B1, B2) is displayed by means of a display device (5) depending on the output signal. Production method, wherein the production method includes a process which is monitored by means of an automated process monitoring method according to one of the preceding claims. Device for automated process monitoring, the device (1) comprising a computing unit (3) and an interface configured to receive image data (4) depicting a scene from a camera system (2) or a storage unit, wherein the computing unit (3) is configured to: - determine, based on the image data (4), that objects (6a, 6b, 7a, 7b, 7c, 8, 9, 10, 11, 14) are located within a predefined area (B1, B2); - determine, for each of the objects (6a, 6b, 7a, 7b, 7c, 8, 9, 10, 11, 14) based on the image data (4), one of at least two predefined object classes; - determine, based on the image data (4), for each of the objects (6a, 6b, 7a, 7b, 7c, 8, 9, 10, 11, 14) one of at least two predefined sub-areas (B1.1, B1.1.1, B1.2, B2.1, B2.1.1, B2.2, B2.2.1) to determine the area (B1, B2) within which the respective object (6a, 6b, 7a, 7b, 7c, 8, 9, 10, 11, 14) is located; - depending on the specified object classes and depending on the specified sub-areas (B1.1, B1.1.1, B1.2, B2.1, B2.1.1, B2.2, B2.2.1) to check whether a predefined rule assigned to the area (B1, B2) is fulfilled; and - depending on a result of the check, to generate an output signal. A computer program product with instructions which, when executed by a computing unit (3), cause the computing unit (3) to perform a procedure for automated process monitoring comprising the following steps: - Receiving image data (4) depicting a scene; - Determining, based on the image data (4), that objects (6a, 6b, 7a, 7b, 7c, 8, 9, 10, 11, 14) are located within a predefined area (B1, B2); - Determining, for each of the objects (6a, 6b, 7a, 7b, 7c, 8, 9, 10, 11, 14) based on the image data (4), one of at least two predefined object classes; - Determining, based on the image data (4), for each of the objects (6a, 6b, 7a, 7b, 7c, 8, 9, 10, 11, 14) one of at least two predefined sub-areas (B1.1, B1.1.1, B1.2, B2.1, B2.1.1, B2.2, B2.2.1) of the area (B1, B2) within which the respective object (6a, 6b, 7a, 7b, 7c, 8, 9, 10, 11, 14) is located; - Check, depending on the specific object classes and depending on the specific sub-areas (B1.1, B1.1.1, B1.2, B2.1, B2.1.1, B2.2, B2.2.1), whether a predefined rule assigned to the area (B1, B2) is fulfilled; and - Generate an output signal depending on a result of the check.

Images