Method of at least partially automatically controlling a container handling apparatus for handling containers and container handling apparatus

Abstract

A method for at least partially automatic, preferably fully automatic, control of a container handling device (1) for handling containers (10), the container handling device having a transport device (2) for guiding the containers (10) along a predetermined transport path according to the occupancy rate of at least one transport area (20) along the transport path, wherein image data (E1, E2) is generated by means of at least one image acquisition device (4) that images the containers (10) located in the transport area (20), and an image evaluation device (6) performs an evaluation, particularly a real-time evaluation, based on the image data to determine an occupancy rate variable characterizing the occupancy rate of the transport area (20), according to the invention, the image evaluation device (6) performs segmentation of the image data (E1, E2) at least in segments to determine the occupancy rate variable.

Description

[0001] manual

[0002] The present invention relates to a method for at least partially automating a container handling apparatus for handling containers, and a container handling apparatus.

[0003] The present invention also relates to a method and a training data generation apparatus for automatically generating a training dataset for training, particularly for training, a container recognition model for machine learning of a post-trained container processing device, wherein containers can be guided or transported along a predetermined transport path.

[0004] Here, image data, which are features of the transport area along the transport path, can be generated or produced using an image acquisition device. Preferably, the transport area is the region in which containers are to be identified. The image data can be provided as an input variable to the container identification model, and specifically to the container identification model, for performing an evaluation of containers located in the transport area, particularly a real-time evaluation.

[0005] Preferably, the container is a plastic container (especially a PET container), a container whose main component is pulp, and / or a glass container and / or a metal can. The container can be used in the beverage and / or food and / or cosmetic industries. For example, it can be a metal can or a bottle, such as a glass bottle, a pulp bottle, or a plastic bottle.

[0006] The container to be identified can be a fully formed container and / or a partially formed container, such as a preform. It can also be empty or filled with product. Furthermore, the container to be identified may or may not have a closure.

[0007] Various objects can be identified using neural networks. This invention particularly relates to the identification of containers or container parts (lids, bottle necks, etc.).

[0008] In order for a neural network to recognize these containers, it must be trained. This requires training data, which specifically includes labeled images.

[0009] Here, the objects to be detected can be labeled using so-called "regions of interest" (ROIs). These can preferably be rectangles, circles, but can also be polygons. If multiple related objects exist in the image, multiple ROIs need to be created. This process is called labeling. In subsequent training, the algorithm learns to recognize these ROIs in other situations. In existing technologies, the labeling process is performed entirely manually or partially automatically.

[0010] In manual labeling, users manually assign annotation data to images. Semi-automatic labeling employs various methods. Most of these use a continuous sequence of images (video). In this case, the user manually labels a portion of the images, and with the help of an algorithm, this annotation data is extrapolated or interpolated to subsequent images. Labeling can also utilize existing neural networks. In this variant, the user must determine whether an object has been correctly labeled. Regardless of the variant, manual user intervention is always necessary.

[0011] Experience with known methods in the prior art shows that trained neural networks cannot recognize every container. Post-training for specific situations is always necessary. Therefore, training requires labeled data. Manual labeling is time-consuming and must be done manually.

[0012] In the field of filling systems, traditional occupancy assessment methods primarily rely on the use of sensors. These sensors are strategically positioned along transport and buffer sections and act as observers, responsible for monitoring and regulating the operation of each machine. They measure and collect crucial occupancy data, and these measurements allow for precise control of machine power, taking into account the speed of the conveyor. Therefore: the more accurate the measurement, the more precise the control over machine power and conveyor speed. Precisely measured occupancy can indicate whether a machine should reduce power or whether a conveyor should increase speed to prevent congestion.

[0013] Currently, the sensors used to assess occupancy rates in filling systems are congestion switches. However, these congestion switches only identify whether a section is fully occupied. Therefore, the occupancy rate starts at 100% from the point of the congestion switch. When the congestion switch is not in use, the occupancy rate is between 0 and <100%. However, using this type of congestion switch cannot determine a more accurate occupancy rate.

[0014] However, a significant drawback of this traditional approach is the inaccuracy of these sensors. This inaccuracy can be particularly problematic when it comes to transport or buffer systems consisting of multiple tracks or with complex geometries. Consequently, the data provided by these sensors often fails to offer an accurate representation of actual occupancy, posing a challenge to maintaining precise control over transport speed / machine power within the system.

[0015] The impact of inaccurate occupancy data goes beyond simple data inconsistency. It directly affects the power output of machines and production lines in the filling system. Machines adjusted based on erroneous occupancy information may become inefficient, potentially leading to production bottlenecks and reduced throughput. Furthermore, congestion and disruptions in transportation may occur more frequently due to this inaccurate data, as transport speed / machine power control is based on this inaccurate information. Transportation congestion not only affects production but also poses a risk of damaging transported products.

[0016] A method for monitoring, controlling and optimizing filling equipment is known from EP 2 132 129 B1, wherein a fine-grading occupancy rate is determined for each individual buffer path, wherein, in order to determine these fine-grading occupancy rates, recorded images are analyzed to determine how many bottles or cans are on the buffer path at the recording time, wherein each individual bottle or can is identified as an object of that type, i.e., identified as a bottle or a can.

[0017] A method for counting objects on a conveyor belt is known from EP 300 523 1 B1. Here, it is proposed to track the location of extracted features for each object in the image data of a frame by extracting features from at least one additional subsequent frame recorded by the same camera.

[0018] The purpose of this invention is to overcome the known shortcomings of the prior art and to provide a precise, fast, and robust method for controlling container handling equipment based on the occupancy rate of a transport area, as well as a corresponding container handling equipment. Another objective is to provide a method and a training data generation device for generating training data for training, particularly for post-training, machine learning-based container recognition models for container handling equipment, which can label or tag training images as little as possible without user intervention. Yet another objective is to provide a precise, fast, and robust method for identifying containers located in the transport area of ​​a container handling equipment, as well as a corresponding container handling equipment.

[0019] According to the invention, this objective is achieved by the subject matter of the independent claims. Advantageous embodiments and further improvements of the invention are the subject matter of the dependent claims.

[0020] In the method according to the invention for at least partially automatic, preferably fully automatic, control of a container handling device for handling containers, the container handling device has a transport device for guiding containers along a predetermined transport path. Here, the container handling device is controlled based on the occupancy rate of at least one (predetermined and / or predeterminable) transport area (of the transport device) along the transport path.

[0021] Here (especially under the operation of the container handling equipment), image data is generated by means of at least one image acquisition device (preferably having multiple image acquisition devices) (preferably at a common recording time), which images the containers located in the transport area, and in particular, a processor-based image evaluation device performs an evaluation based on the image data, especially a real-time evaluation, to determine an occupancy variable characterizing the occupancy rate of the transport area.

[0022] Preferably, the image data is generated during the continuous transport operation of the transport device, meaning that the image acquisition device does not cause or affect the transport speed of the container and / or is related to such an effect when collecting image data.

[0023] Specifically, the occupancy rate variable applies to containers located in the transport area, preferably to the number of containers located in the transport area and / or to the proportion of the transport area occupied by containers located in the transport area.

[0024] Preferably, the transport area is arranged in a daytime manner within the acquisition area of ​​the at least one image acquisition device, meaning that the entire (predetermined and / or predeterminable) transport area and / or containers located within the entire (predetermined and / or predeterminable) transport area can be or are acquired by the at least one image acquisition device. Preferably, the image data generated (using the at least one image acquisition device) is a feature of the containers located within the transport area (at the recording time).

[0025] The image acquisition device is preferably an image acquisition device of a container processing device. The image evaluation device is preferably an image evaluation device of a container processing device. Preferably, the image evaluation device evaluates the image data to be evaluated by means of a container evaluation model (especially a computer-implemented one).

[0026] Image acquisition devices (or multiple image acquisition devices) can be image recording devices, such as cameras (preferably black-and-white and / or color cameras); CMOS sensors (CMOS is an abbreviation for Complementary metal-oxide-semiconductor); CCD sensors; 3D sensors; X-ray-based image recording devices; optical elements; thermal imaging cameras; stereo cameras; LiDAR cameras, and combinations thereof.

[0027] The image data (generated and / or transmitted and / or provided to the image evaluation device by the image acquisition device) is preferably two-dimensional (spatial resolution) image data. Preferably, the image data that can be transmitted (or transmitted) and / or provided to the image evaluation device for evaluation does not include depth information (directly or immediately measured by the image acquisition device), i.e., particularly not including measurements of depth and / or distance measured in the recording direction of the image acquisition device, characterized by the distance from the image acquisition device to the object imaged in the image data. In other words, the image acquisition device preferably does not generate (and / or acquire) measurements that only characterize the distance and / or (relative) position of the image acquisition device relative to the imaged object. This distance and / or position of the image acquisition device relative to the object it images can be determined from image data generated by the image acquisition device at different locations from each other (e.g., via stereo recording and / or a LiDAR camera). Preferably, as described in more detail in the context of a method for automatically generating a container recognition model for training machine learning for container processing devices (with particular reference here), spatial coordinates can be calculated using 2D images (recorded by an image acquisition device) (e.g., if the diameter of the identified, transported, and / or processed container is known, or alternatively, a reference surface or reference line with a known geometric extent acquired by the image acquisition device).

[0028] Preferably, color images or color video sequences or color image sequences (for determining and / or generating image data) are recorded by an image acquisition device. However, it is also conceivable that the recorded and / or generated and / or determined image data are grayscale images or grayscale value images. In other words, it is conceivable that colorless image data is sent (or provided as an input variable) to an image evaluation device or image evaluation model. Thus, it is advantageous to acquire or image and evaluate a larger transport area of ​​the container processing equipment in the image data.

[0029] Image acquisition devices (or multiple image acquisition devices) are preferably suited for recording (static) (single) images and / or moving images or image sequences (or video sequences) and are intended for this purpose.

[0030] The image data (generated by the image acquisition device) to be provided to the image evaluation device (or image evaluation model) can be an image sequence and / or a single image and / or image data recorded or acquired (essentially) at a single recording moment or (essentially) simultaneously.

[0031] It is conceivable that the image data (to be provided and / or transmitted to the image evaluation device and / or image evaluation model, on which the image evaluation device performs evaluation) is image data determined by more than one single image. For example, two (different from each other) image acquisition devices (preferably substantially simultaneously) can record images separately. Preferably, images composed of multiple images and / or image data generated from multiple images are provided to the image evaluation device and / or image evaluation model.

[0032] Additionally or alternatively, it is conceivable to generate image data (on which an image evaluation device performs evaluation) based on multiple individual images (recorded and / or acquired by at least one image acquisition device, particularly recorded and / or acquired at different recording times). For example, a single frame of a video sequence (recorded by at least one image acquisition device) can also be used as a single image. For example, several frames (recorded sequentially in time) can be averaged and / or summed to generate image data (on which an image evaluation device performs evaluation).

[0033] Preferably, multiple image acquisition devices are synchronized and / or can be synchronized with each other to record or generate image data. Here, the multiple image acquisition devices preferably cover the entire transport area with their acquisition areas. However, it is also conceivable that the multiple image acquisition devices record image data at different recording times and process the image data in such a way (e.g., according to the transport speed of the transport device and / or the relative arrangement of the image acquisition devices) to obtain image data characterizing the occupancy rate of the transport area and / or the position and / or orientation of containers located within the transport area at the same time.

[0034] Here, at least one image acquisition device can preferably be arranged above the transport area. It is conceivable that the image acquisition device is arranged obliquely above the transport area, thereby acquiring image data specifically from an acquisition direction that surrounds the transport plane at an angle other than 90°. The transport plane can be, for example, the contact surface with the container provided by the transport device (e.g., a transport channel) during container transport. For example, if the container is transported upright, the transport plane can be the surface of the transport device that can contact the bottom of the container during transport.

[0035] The image acquisition device can be arranged symmetrically and / or centrally relative to the transport area in the width direction, particularly perpendicular to the transport direction or transport path. However, a lateral arrangement of the image acquisition device relative to the (central) transport path and / or transport area is also conceivable.

[0036] Preferably, the image acquisition device (at least partially, preferably completely) is arranged vertically above the transport area. In particular, the acquisition direction of the image acquisition device forms a 90° angle with at least a portion of the transport area and preferably with the entire transport area.

[0037] Preferably, the container handling equipment (in particular each individual transport unit) has exactly one image acquisition device for acquiring image data related to the containers located in the transport segment.

[0038] The preprocessed image data is preferably provided to the image evaluation device and / or the image evaluation model. The preprocessing steps for the image data preferably include cropping (e.g., cropping to a transport area or mapping image data points to a transport area), sharpening, and / or changing the brightness. Preferably, the color gamut of the image data is not changed, and in particular, the color image is not converted to a grayscale image. However, it is also conceivable to change the color values ​​or the color gamut of the image data.

[0039] However, it is also conceivable that the image evaluation device (alone or in combination) performs preprocessing of the image data and / or (e.g., the above-mentioned) preprocessing steps.

[0040] According to the present invention, the image evaluation apparatus performs segmentation of these image data at least in segments to determine occupancy variables. The image data is specifically the image data to be evaluated by the image evaluation apparatus (and / or specifically provided as input variables to the image evaluation model). However, it is also conceivable that the image evaluation apparatus performs preprocessing (e.g., as described above) of the image data to be evaluated (by which it is to be evaluated) before performing segmentation of the image data obtained through preprocessing.

[0041] Image evaluation models, particularly (preferred semantic) segmentation models, can be used to perform segmentation of the image data provided as input variables.

[0042] The determined occupancy rate variable is preferably used as a control variable for the container handling equipment, for example, for (automatically) performing processing functions on at least one and / or multiple containers. It is also conceivable that the at least one occupancy rate variable is used as a control variable to control and / or regulate container flow and / or container throughput and / or transport speed and / or container supply and / or container discharge through the container handling equipment.

[0043] Specifically, segmentation is understood as a (computer-implemented) process of image evaluation and / or image processing, particularly performed by an image evaluation device, wherein the image data (to be segmented) (evaluated by the image evaluation device and / or provided as input variables to an image evaluation model, particularly a segmentation model) is divided into segments (divided into useful image parts). Here, a segment is preferably a set (particularly preferably related) of data points that are distinguishable by specific or predetermined and / or predeterminable features or attributes, or that satisfy predetermined and / or predeterminable specific relationships (e.g., grayscale values ​​and / or color values ​​are within a predetermined color range).

[0044] Preferably, the image data is segmented or divided into segments relating to the (to be processed) container. Preferably, the segmentation or partitioning of the image data depends on an assessment of whether they (at least partially) represent the container (or whether they (at least partially) do not represent the container).

[0045] Preferably, the segmentation and / or partitioning of image data is performed relative to a predetermined and / or predefined identifiable region on the (to be processed) container.

[0046] Here, the identification area on the (to be processed) container is specifically understood to mean that the identification area can be a sub-region of the container (e.g., the upper side and / or the lower side of the container) or an area of ​​an element arranged on the container (preferably non-rotatable and / or non-translatable) (e.g., the container's closure and / or fittings, such as labels). This provides the advantage that areas that are particularly easy to identify by optical image evaluation and / or by image processing can be specified or selected as identification areas (e.g., by means of objects (e.g., (reflective) railings) relative to the rest of the container and / or relative to objects surrounding the container and / or transport area).

[0047] In a preferred method, the identification area (of the container to be processed) is selected from a range including lids, closures, top and bottom sides of cans, top and bottom sides of containers, sidewall areas of containers, mouth areas of the container to be identified, container fittings (e.g., labels applied to the container), markings arranged on the container, closures, container groups (the entire group), and combinations and sub-ranges thereof. Preferably, the identification area is a portion of the container's surface imaged (in a two-dimensional image). The identification area can be an element (e.g., a label) arranged on the container and / or a feature of a (partial) area of ​​the container.

[0048] For example, in a preferred method, the identification area can be a container cap, such as a bottle cap.

[0049] Preferably, the identification of individual containers is not performed during segmentation. In particular, for example, the object boundaries between adjacent containers are not determined during segmentation. In other words, the segments obtained by segmentation (e.g., as output variables in image evaluation models, particularly in segmentation models) do not specifically indicate the object boundaries and / or location and / or orientation of a single container among multiple adjacent containers. In other words, the segments whose image data points obtained by segmentation represent multiple (at least pairs) adjacent containers are not features of one or more object boundaries and / or container orientation and / or orientation extending between adjacent containers.

[0050] In particular, for example, based solely on segments obtained through segmentation, which combine image data points representing at least one container (located on the transport area and acquired by the generation of image data), it is impossible to distinguish whether there is approximately exactly one container on the transport area in an orientation that is located on the transport area, or whether two or more containers are located in their positions in an upright orientation (and collectively occupy approximately equal placement areas of the transport area).

[0051] The occupancy rate variable is determined based on at least one segment obtained through segmentation, and preferably based on at least multiple segments obtained through segmentation. This provides the advantage that the occupancy rate variable can be determined very quickly and robustly in this manner.

[0052] Preferably, the occupancy variable is determined based on the (determined) number of pixels or image data points of at least one segment, and more preferably based on the number of pixels or image data points of those segments associated with the container imaging image data points. For example, this number, or the variable derived therefrom, can be compared with (at least) a comparison variable. The comparison can be performed, for example, by forming ratios and / or differences. Furthermore, the comparison results obtained from the comparison can be compared with (predetermined and / or predeterminable) thresholds.

[0053] The (at least one) comparison variable may be predefined and / or may be predefined (e.g. by the operator).

[0054] The (at least one) comparison variable can be determined at least partially automatically and preferably fully automatically (by the container handling equipment). It is conceivable that a setup operation (particularly described in more detail below) for this purpose is provided for the container handling equipment, which is preferably different from the (intended) working operation of the container handling equipment (with the highest possible production volume or transport speed), in which containers are handled.

[0055] The comparison variable (at least one) is preferably stored in the image evaluation device. The stored comparison variable is preferably variable, and particularly preferably, the change can be determined by the operator and / or automatically (by the container processing device).

[0056] It is conceivable that the number of image data points and / or a measure of the area mapped by the image data (e.g., a plane relative to the transport area extending within it) are used as comparison variables, on which segmentation is performed. This comparison variable can serve as a reference variable for the number of image data points (or a measure of the area mapped by the image data points) of each segment, making objective evaluation possible.

[0057] In a further preferred method, the occupancy rate variable is determined based on the size of the transport area, which is a characteristic of the occupied area and / or maximum number of occupants of the transport area. This preferably allows for a more accurate evaluation of the image data because, when evaluating segments obtained through segmentation (or their segment sizes), no background image data points are considered—that is, image data points that represent the environmental area surrounding the transport area but not the transport area (or the containers on it).

[0058] (Therefore) it is preferable to select the size of the transport area as (at least one) comparison variable.

[0059] The maximum occupancy of a transport area should be specifically understood as the maximum number of containers that can be accommodated in the transport area.

[0060] The transport area size, which characterizes the maximum occupancy of the transport area, can also be understood as the area (part) occupied by the containers in the transport area under maximum loading conditions, i.e., when the containers are packed in the transport area with their maximum capacity in the most dense manner. Therefore, this transport area size can, for example, indicate or characterize multiple image data points / pixels, in each case, when the transport area is maximally occupied (when the containers are packed most densely), these image data points / pixels represent the container area of ​​one of the containers.

[0061] Coverage area is preferably understood as the area of ​​the transport region where the container can be contained and / or transported and / or guided, or its characteristic dimensions. Therefore, occupied area can represent a measure of the actual transport area area. However, it is also conceivable that occupied area represents a measure of the (total) area of ​​the imaged transport region (in image data).

[0062] The comparison variables and / or transport area dimensions can be independent of the type of container (to be processed). For example, if a conveyor belt is used as a transport device, the total surface area of ​​the conveyor belt located in the transport area can be chosen as the transport area size (or the corresponding size of a characteristic of that total surface area depicted in the image data). In particular, containers with circular cross-sections, such as metal cans or bottles known in the beverage industry, will not completely fill the transport area when packed in the most dense manner on a conveyor belt or in the transport area. However, the total surface area of ​​the conveyor belt can already provide a good approximation of the characteristic size of the maximum occupancy of the transport area (especially for containers with known (cross-sectional) geometries).

[0063] In this case, the aforementioned thresholds may depend on the cross-sectional area occupied by the container (to be processed). Therefore, several thresholds may be stored, for example, in the image evaluation device (e.g., in the form of a database), with each threshold assigned to a different container type.

[0064] In this context, the cross-sectional area and / or cross-sectional geometry specifically represent the corresponding cross-sectional dimensions relative to the cross-sectional plane perpendicular to the acquisition direction of (at least one) image acquisition device.

[0065] The comparison variables and / or transport area dimensions can depend on the type of container (specifically, cross-sectional area and / or diameter). For example, this might be the case if the transport area size is chosen as a characteristic of maximum occupancy, since denser or less dense packaging is possible depending on the cross-sectional geometry.

[0066] The container handling equipment preferably has at least one human-machine interface (HMI) and / or input device through which the transport area dimensions can be (manually) entered (particularly by the operator of the container handling equipment). It is also conceivable that the container handling equipment has a receiving device through which the transport area dimensions (e.g., from a cloud-based (back-end) server (e.g., the manufacturer of the container handling equipment)) can be sent to the container handling equipment (e.g., via a wireless communication connection).

[0067] It is also conceivable that the operator's terminal, particularly a mobile terminal, detects image data and / or location data that at least partially indicates the transport area and preferably indicates and / or characterizes the relative arrangement and / or orientation of the terminal with respect to the transport area. Particularly preferably, the transport area can be measured (geometrically) by the terminal, particularly the mobile terminal, and characteristic quantities of the geometric extent of the transport area are preferably transmitted to the receiving device of the container handling equipment.

[0068] In a further preferred method, the dimensions of the transport area are determined automatically (by the container handling equipment). This provides the advantage of very user-friendly commissioning, where the operator does not have to perform measurements and / or calculations, etc.

[0069] Preferably, the container handling equipment can be implemented in a setup operation, wherein the dimensions of the transport area are determined, particularly optically. In this case, the setup operation is preferably different from the (intended) working operation of the container handling equipment (with the highest possible production volume or transport speed), in which the container handling equipment is used for the (normal) production or handling of containers. For example, the transport speed of the transport device may vary at least in sections of the transport area.

[0070] In a further preferred method, in order to determine the size of the transport area, image data is automatically generated or acquired (and evaluated) by means of an image acquisition device while the transport device is in the set state, wherein the maximum number of containers that can be accommodated in the transport area is recorded.

[0071] The setting state specifically refers to the state of the transport area or transport device of the container handling equipment, in which the transport area is occupied to the maximum extent by the (to be processed) containers (i.e., in the state where the containers are most tightly packed in the transport area, where the gaps between the containers are particularly minimal).

[0072] Preferably, the size of the transport area is determined based on image data representing the setup status or maximum occupancy of the transport area, preferably the number and / or corresponding number of image data points (e.g., pixels) representing a portion of one of the containers in the transport area. Preferably, the size of the transport area is determined based on the image data acquired and / or generated in this manner by segmenting these image data. Preferably, the characteristic dimensions (particularly as the transport area size) of the segments, or segments representing all those segments respectively representing the container and / or the identification area of ​​the container, are determined (e.g., by determining the area and / or the number of data points).

[0073] Then, the transport area size can preferably be used as a comparison variable, which is compared with one or more segments that represent the container and / or the container identification area or the size derived therefrom, during the operation of the container handling equipment.

[0074] It is also conceivable that several different setup states (within the same container type) are possible. Therefore, slightly different arrangements of the containers are possible, such as offsets in the transport direction. Preferably, the image acquisition device acquires and / or generates image data at least once for each setup state. Thus, a corresponding transport area size can be determined for each setup state, and the transport area size used in the operational operation of the container handling equipment can be obtained by using these individual transport area sizes.

[0075] The use of an image acquisition device offers the advantage that it has the same configuration and / or arrangement as, or can be configured and / or arranged in the same way as, the container handling equipment, wherein the image acquisition device generates and / or acquires image data for determining occupancy variables. Therefore, these two records or image data can be directly compared with each other.

[0076] Preferably, the setup operation of the transport device (or its embodiments) can be triggered by an operator. Preferably, the setup state can be achieved through the setup operation (and thus indirectly through triggering the setup operation). In other words, the setup state can be generated by performing the setup operation. Thus, for example, by performing the setup operation, the container can be transported to the transport area (via the transport device) until maximum occupancy is reached. The approach to and / or reaching maximum occupancy of the transport area can be monitored by means of an image acquisition device and image data thus acquired from the transport area.

[0077] Therefore, image data can be acquired at time intervals (at different acquisition moments) using an image acquisition device, and the size of the transport area can be determined based on this data in each case. If this increases (continuously) compared to previous image data at a previous acquisition moment, it can be assumed that maximum occupancy has not yet been reached. On the other hand, if the transport area size remains unchanged, it can be assumed that maximum occupancy has been reached.

[0078] It is also conceivable that the setup operation is performed several times (and the transport area is simultaneously emptied and / or the containers located in the transport area are further transported), thus generating multiple setup states. It is conceivable that the transport area size is averaged and / or determined based on several setup states.

[0079] It is also conceivable, for example, that the transport area is empty and / or independent of its occupancy, and that the size of the transport area is determined based on image data depicting the transport area acquired and / or generated here (by means of an image acquisition device). Therefore, for example, the boundaries of the transport area can be (at least partially) identified by object recognition (e.g., performed by an image recognition device outside the image evaluation device or container handling equipment). Preferably, the area within the boundary and / or the number of image data points (e.g., pixels) is determined based on the identified boundaries of the transport area. These variables (or variables derived therefrom and / or their characteristic variables) can, for example, be used as the size of the transport area.

[0080] Object recognition can, for example, identify and / or identify the side walls and / or railings and / or side covers of a transport area and / or the edges or transitions of the transport area to its surrounding areas (e.g., corridor floors or railings). Preferably, this can derive (e.g., lateral) boundaries.

[0081] Additionally or alternatively, users may identify and / or indicate at least one boundary (or the entire boundary or edge) of the transport area depicted in the image data, for example, through the human-machine interface and / or receiving device of the container handling equipment.

[0082] Here, based on the boundary, the feature (surface) size of the transport area and / or the number of image data points corresponding to the surface of the transport area (which map to the transport area) can be used as the transport area size.

[0083] The (manual and / or automatic) geometric marking or identification of the layout of the transportation area in the image data provides the advantage that only image data points of the area mapped to the transportation area are used when determining occupancy variables and / or transportation area dimensions. In this way, it is possible to determine the occupancy variables by image areas that are mistakenly identified as containers but are located outside the transportation area (e.g., through reflections or illumination on metal bodies such as railings).

[0084] It is also conceivable that the size of the transport area could be determined by an image evaluation model using machine learning, which has been trained on multiple image data and associated transport area sizes.

[0085] In a further preferred method, to determine the occupancy variable, the image data is evaluated such that multiple adjacent containers located in a transport area are identified as a uniform container cluster, which does not indicate differences between individual containers. This provides the advantage of faster image data evaluation because it does not require distinguishing individual containers or their orientation. Specifically, segments obtained through partitioning are assigned to each container cluster in the transport area.

[0086] In contrast to container identification, where each container is identified and / or distinguished (and for example labeled and / or represented as a container), individual containers in a unified container cluster are no longer distinguishable after being split.

[0087] Specifically, image recognition or image data is not used to determine occupancy variables, but rather to identify all containers in the transport area in a form that is distinguishable from one another or individually identifiable.

[0088] In a further preferred method, the occupancy rate variable is determined independently of the container's location and / or orientation within the transport area. This also offers the advantage of enabling the rapid and robust determination of assessment results meaningful for the occupancy rate.

[0089] In particular, the segments obtained through segmentation no longer contain information about the number and / or orientation of the containers corresponding to that segment located in the transportation area (mapped from the image data points of that segment).

[0090] In a further optimized method, the segmentation is semantic segmentation. Specifically, the segmentation is not instance segmentation. In this way, the occupancy variable is also determined quickly and robustly, which is crucial for the sometimes extremely high transport speeds of containers commonly found in the beverage industry.

[0091] In a further preferred method, by segmentation, if image data points are assigned to containers and / or container clusters, the category “container” selected from multiple, preferably two categories, is assigned to the image data points, and / or if image data points are not assigned to containers and / or container clusters, they are assigned to the category “background”.

[0092] It is conceivable that this allocation and / or assignment would be performed based on the identification area of ​​the container (e.g., bottle cap). Therefore, containers located in the shipping area could also be identified on their bottle caps.

[0093] In a further preferred method, the occupancy variable is determined based on those image data points assigned to the category "container" during the segmentation.

[0094] In a further preferred method, segmentation is performed based on a predetermined and / or predeterminable color range. Specifically, segmentation can be performed based on a predetermined color filter. In this case, the color range may (at least partially and preferably completely) include and / or consist of the identification range of the container and / or those color values ​​possessed by the container. Preferably, the color filter is selected such that it identifies (substantially) those image data points that have the color values ​​of the identification region and / or the container. In this way, segmentation that is both fast and efficient can be advantageously achieved.

[0095] In a further optimized method, segmentation is performed using a machine learning-based segmentation model.

[0096] The segmentation model for machine learning is preferably based on (artificial) neural networks. Preferably, the neural network is formed as a deep neural network (DNN), wherein the parameterizable processing chain has multiple processing layers, and / or so-called convolutional neural networks (CNNs) and / or recurrent neural networks (RNNs) and / or other DNN layer classes.

[0097] The machine learning segmentation model is preferably a segmentation model that has been (completed) learned or trained.

[0098] Preferably, the machine learning segmentation model has been / is being trained with a training dataset that includes multiple image data (provided as input variables to the segmentation model) depicting a transportation area containing containers, and segmentation data obtained by assigning these image data separately after segmentation. Segmentation can be performed as described above.

[0099] In this scenario, the training dataset could consist only of data relevant to exactly one container type and / or a predetermined container processing device. However, it is also conceivable that the training dataset includes image data (and their respective segmentation data) acquired and / or generated in different container processing devices (of the same and / or similar design). Using image data across container processing devices offers the advantage that the segmentation model trained with it is more robust to structural variations (e.g., the contours of transport areas) and / or brightness variations and / or environmental variations.

[0100] The training dataset to be generated is preferably used for (further) fine-tuning of the container recognition model or for this purpose.

[0101] It is conceivable that the container recognition model has been trained using a general, preferably independent, training dataset that is independent of specific or particular container processing devices / the container recognition model is in a state of having been trained using the aforementioned model.

[0102] In a further preferred method, at least one additional transport area is predetermined, preferably multiple transport areas. Preferably, the (continuous) transport sections of the transport device are divided into multiple transport areas, and particularly preferably (especially without overlap) consist of these transport areas. The transport areas are arranged specifically along the transport path.

[0103] Preferably, in each case, image data is generated for at least one additional transport area or a corresponding transport area among multiple transport areas by at least one image acquisition device. In this case, a single image acquisition device can acquire image data. However, it is also conceivable that multiple image acquisition devices collect image data. It is also conceivable that in each case, one image acquisition device is precisely (uniquely) assigned to one transport area.

[0104] In each case, the generated image data preferably depicts the container located in the corresponding (at least one additional) transport area.

[0105] Preferably, based on the corresponding image data, the image evaluation device performs an evaluation in each case, particularly a real-time evaluation, to determine the occupancy variable associated with the corresponding transport area, which is a characteristic of the container located in the corresponding (or at least one other) transport area.

[0106] Here, the image evaluation device performs segmentation of these image data at least in segments to determine the occupancy variable.

[0107] Preferably, in each case, the corresponding occupancy rate variable is determined in one of the above (preferred) methods.

[0108] It can be imagined that each occupancy rate variable serves as a control variable for controlling the drive unit (of the transport device itself). In other words, each drive unit of the transport device can be assigned a transport area, and the drive unit is controlled according to the (determined) occupancy rate variable of that transport area. However, it is also conceivable to control the drive units based on the (determined) occupancy rate variables of multiple transport areas.

[0109] Preferably, the transport section of the transport device is divided into multiple transport zones according to the location of the drive unit of the transport device and / or the location of the area of ​​the transport device controlled by different drive units.

[0110] Additionally or alternatively, dividing a (particularly continuous) transport segment into multiple transport zones (especially determining the occupancy rate variable in each case) can depend at least in part on at least one geometric parameter that is characteristic of the (at least part) geometric profile of the transport segment. The geometric parameter can be, for example, characteristic variables of curvature and / or width (segments of the transport zone) and / or branching and / or openings. Thus, in cases where a transport zone within the segment has strong curvature (small radius of curvature) and / or relatively large width (compared to other segments of the transport zone), a smaller transport segment subdivision can be selected (relative to the geometric extent along the main transport path). Width specifically refers to the geometric extent of the corresponding segment perpendicular to the main transport path.

[0111] In the proposed preferred method, the focus is particularly on segmentation, especially the segmentation of bottle caps in, for example, filling systems, particularly in the beverage industry. Preferred methods include using (particularly semantic) segmentation models, such as neural networks or color filters, and adjusting the power of individual machines in the container handling equipment and / or system based on the number of identified bottle cap pixels relative to the total number of pixels in a specific buffer or transport segment (transport area size).

[0112] This system uses image recording equipment and an image processing computer for data acquisition. Specifically, a semantic segmentation model, which can be a neural network or a color filter, is used to identify bottle caps. In the related data processing system, advanced image analysis and segmentation techniques are applied to identify and segment bottle caps based on pixel analysis.

[0113] The process preferably involves scanning the buffer and transport sections of the filling system using a camera. However, due to pixel limitations, the primary focus is on cap segmentation. This segmentation process is performed carefully regardless of the cap's position or orientation. The system sums the identified pixels corresponding to the caps and compares this number with the total number of pixels in the buffer or transport section. Based on this comparison, the system adjusts the throughput of each machine in the filling system.

[0114] Using advanced semantic segmentation models and pixel-based analysis, this preferred approach provides a more accurate representation of occupancy, thereby improving machine efficiency and reducing the likelihood of transport congestion. This not only enables smoother production but also contributes to a more reliable and efficient filling process in the beverage industry.

[0115] Preferably, the container handling equipment is selected from the group consisting of: transport equipment (e.g., conveyor belts) for transporting containers, buffering equipment for temporary buffer containers, pasteurization equipment (e.g., tunnel pasteurizers), heating equipment for heating preforms, molding equipment for molding plastic preforms into plastic bottles, sterilization equipment for sterilizing containers (especially plastic preforms), manufacturing equipment for manufacturing glass bottles, manufacturing equipment for filling containers with products, inspection equipment for inspecting plastic preforms or bottles, labeling equipment for labeling containers, sealing equipment for sealing containers (especially filled containers), control devices, packaging devices, direct printing equipment for printing containers, assembly equipment for assembling multiple containers into components or groups, etc.

[0116] Preferably, the container flow is a sequential or sequential (especially continuous) flow of containers (along the transport path). The container flow can be guided or transported (by transport means) in certain areas, preferably within the entire container handling equipment (as a mass flow), in a single-channel or multi-channel manner.

[0117] The transport equipment may also be a high-density transport vehicle, used to transport multiple containers preferably in a multi-channel and / or unordered manner. The transport device may also be a buffer zone, used to buffer multiple containers preferably in a multi-channel and / or unordered manner.

[0118] The container can preferably be transported or guided at least in sections and preferably upright or vertically along the entire transport area (by the transport device).

[0119] Preferably, the transport device is adapted and designed to guide or transport multiple containers in at least segmented manner, preferably along the entire transport area, where the containers are under back pressure.

[0120] Preferably, the transport device is adapted and designed to transport and / or guide (at least within the transport area) at least one container per hour, preferably at least 5,000 (especially to be identified) containers per hour, preferably at least 20,000, preferably at least 100,000 (especially to be identified) containers per hour, particularly preferably at least 140,000 (especially to be identified) containers per hour, and this is performed during the working operation of the container handling equipment. Preferably, the transport device is adapted and designed to transport and / or guide up to 150,000 (especially to be inspected) containers per hour (at least within the transport area), and this process is carried out during the working operation of the container handling equipment.

[0121] Preferably, the transport device in a single-lane transport area is suitable and intended (at least in the single-lane transport area) to transport and / or guide at least 100,000 containers per hour, and / or at most 150,000 containers per hour, and the process is carried out within the working operation of the container handling equipment.

[0122] The present invention also relates to a method for at least partially automatically determining the occupancy rate of at least one transport area in a container handling apparatus for handling containers relative to the containers located in the transport area.

[0123] In this case, the container handling equipment has a transport device, which allows the (to be processed) containers to be guided and / or transported along a predetermined transport path (particularly toward the transport area).

[0124] In this scenario, a transport area is arranged along a transport path, wherein image data is generated by means of at least one image acquisition device, which images the containers located in the transport area, and based on this image data, an image evaluation device (in particular a container processing device) performs an evaluation, particularly a real-time evaluation, to determine a variable characterizing the occupancy rate of the containers located in the transport area.

[0125] According to the present invention, the image evaluation device performs segmentation of these image data at least in segments to determine the occupancy variable.

[0126] In this context, the method may include, individually or in combination with each other, all the method steps described above in conjunction with the method for at least partially automated, preferably fully automated, control of container handling equipment, particularly for determining occupancy variables and / or for performing partitioning.

[0127] The present invention also relates to a container handling apparatus for handling containers having a transport device adapted and intended to transport and / or guide containers along a predetermined transport path.

[0128] The container handling equipment also includes at least one image acquisition device for generating image data of the container located in the transport area.

[0129] The container handling equipment also has an image evaluation device that is suitable for and designed to determine occupancy variables characterizing the occupancy rate of the transport area, and is designed to perform evaluations based on image data, particularly in real time.

[0130] According to the present invention, the image evaluation apparatus for determining the occupancy rate variable is adapted and designed to perform image data segmentation at least segment by segment. Furthermore, the container processing apparatus has a control device for controlling the container processing apparatus at least partially automatically, preferably fully automatically, based on the determined occupancy rate variable.

[0131] In this case, the container handling equipment may have, individually or in combination, all the features described above in conjunction with the methods for at least partially controlling the container handling equipment and / or for at least partially automatically determining the occupancy rate of the transport area in the container handling equipment (in particular the container handling equipment) and is suitable for and intended to perform all the method steps described above in conjunction with both methods (in particular also according to the preferred embodiment).

[0132] Conversely, the container processing devices mentioned in the context of both approaches may individually or in combination with each other possess the characteristics of the container processing devices described herein.

[0133] The present invention also relates to a method, particularly a computer-implemented method, for automatically generating a training dataset for training, particularly for post-training, a (trainable) machine learning container recognition model (particularly a (processor-based) real-time image evaluation device) of a container processing device, wherein (particularly during the operation of the container processing device) containers can be guided and / or transported (by means of a transport device), preferably in the form of a container flow.

[0134] The machine learning container recognition model can preferably be the image evaluation model and / or segmentation model described above.

[0135] It is also conceivable that, instead of generating a training dataset for training, especially for post-training the aforementioned container recognition model, the method according to the invention (only) targets the automatic labeling of image data (wherein, the training dataset is generated based on the labeled image data, preferably in a subsequent step independent of or different from the method, for example by associating the generated annotation data with the corresponding image data).

[0136] In this scenario, image data can be generated using at least one image acquisition device (and preferably multiple image acquisition devices). This image data represents features of a transport area (and containers located therein) along the transport path (predetermined and / or potentially predetermined and / or particularly potentially acquireable by at least one image acquisition device), and can be provided as input variables to a container identification model for performing an evaluation of containers located in the transport area, particularly a real-time evaluation (e.g., for determining the occupancy rate of the transport area). The image acquisition device is preferably an image acquisition device of a container processing device. Preferably, the transport area is the area in which containers are to be identified. In particular, the containers located in the transport area are imaged using image data generated by at least one image acquisition device (and preferably multiple image acquisition devices).

[0137] In other words, (during the operation of the container handling equipment), images are recorded using at least one image acquisition device, in which a transport area (and the containers located therein) are captured or imaged. These are preferably evaluated in real time by a real-time image evaluation device using a container recognition model. For example, using the container recognition model as an evaluation result or recognition result, the real-time image evaluation device can determine and / or provide evaluation variables for outputting and / or transmitting characterizing at least one of the following: the number of containers identified in the transport area (or each container identified in the transport area) and / or the location (or location) of the (identified) containers and / or the orientation of the containers (or each container identified in the transport area) and / or the type of the containers (or each container identified in the transport area) and / or the state of the containers (or each container identified in the transport area) and / or the speed of the containers (or each container identified in the transport area) and / or the occupancy rate of the transport area or segment along the transport path relative to the containers located therein and / or the distribution of containers in predetermined areas within the transport area.

[0138] The evaluation variables are preferably used as control variables for the container handling equipment, for example, for performing processing functions (automatically) on at least one and / or multiple containers. It is also conceivable that at least one evaluation variable is used to control and / or regulate container flow and / or container throughput and / or transport speed and / or container supply and / or container discharge through the container handling equipment.

[0139] Image acquisition devices (or multiple image acquisition devices) can be image recording devices, such as cameras (preferably black-and-white and / or color cameras); CMOS sensors (CMOS is an abbreviation for Complementary metal-oxide-semiconductor); CCD sensors; 3D sensors; X-ray-based image recording devices; optical elements; thermal imaging cameras; stereo cameras; LiDAR cameras, and combinations thereof.

[0140] The image data (generated by the image acquisition device and / or provided to the image evaluation device) is preferably two-dimensional (spatial resolution) image data. Preferably, the image data that can be provided to or supplied to the container recognition model to perform the evaluation does not include depth information (measured directly or immediately by the image acquisition device), i.e., particularly not measurements (depth and / or distance) measured in the recording direction of the image acquisition device, characterized by the distance from the image acquisition device to the object imaged in the image data. In other words, the image acquisition device preferably does not generate (and / or acquire) measurements that only characterize the distance and / or (relative) position of the image acquisition device relative to the imaged object. This distance and / or position of the image acquisition device relative to the object it images can be determined from image data generated by the image acquisition device at different locations (e.g., via stereo recording and / or a LiDAR camera). Therefore, if the diameter of the identified container or the container being transported and / or the container to be processed is known, the spatial coordinates can be calculated back using the 2D image (recorded by the image acquisition device). Alternatively, a chessboard with known side lengths can also be used as an inverse calculation method. Therefore, a predetermined (preferably entirely within a plane) reference line or reference surface with an approximately checkerboard configuration (preferably having at least two reference lines extending at least segmentally in different spatial directions and having predetermined and / or measured arc lengths) can be designated as a reference variable for determining the three-dimensional spatial coordinates relative to the desired 2D image data points. Based on the reference surface imaged in the 2D image data or the reference lines from which the image is imaged and their known or predetermined geometric dimensions, the distance to the 2D image data points in the 2D image, particularly the (3D) spatial coordinates, can be determined.

[0141] Preferably, color images or color video sequences or color image sequences (used to determine and / or generate image data) are recorded by an image acquisition device. However, it is also conceivable that the recorded and / or generated and / or determined image data are grayscale images or grayscale value images. In other words, it is conceivable that colorless image data can be provided as an input variable to the container recognition model. Thus, it is advantageous to acquire or image and evaluate a larger transport area of ​​the container processing equipment in the image data.

[0142] Image acquisition devices (or multiple image acquisition devices) are preferably suited for recording (static) (single) images and / or moving images or image sequences (or video sequences) and are intended for this purpose.

[0143] The image data (generated by the image acquisition device) to be provided to the container recognition model can be an image sequence and / or a single image and / or image data recorded or acquired (essentially) at a single recording moment or (essentially) simultaneously.

[0144] It is conceivable that the image data to be provided to the container recognition model is image data determined from more than one single image. For example, two (different from each other) image acquisition devices (preferably substantially simultaneously) can record images separately. Preferably, images composed of multiple images and / or image data generated from multiple images are provided to the container recognition model.

[0145] The preprocessed image data is preferably provided to the container recognition model. The preprocessing steps for the image data preferably include cropping, sharpening, and brightness adjustment. Preferably, the color gamut of the image data is not altered, and in particular, the color image is not converted to grayscale.

[0146] The machine learning container recognition model is preferably based on an (artificial) neural network. Preferably, the neural network is formed as a deep neural network (DNN), wherein the parameterizable processing chain has multiple processing layers, and / or so-called convolutional neural networks (CNNs) and / or recurrent neural networks (RNNs) and / or other DNN layer classes.

[0147] The machine learning container recognition model is preferably a container recognition model that has already been learned or trained. In other words, the container recognition model to be trained or further trained more accurately using the training dataset to be generated is in a state after the training process has been completed. The training dataset to be generated is preferably used for (further) fine-tuning of the container recognition model or for this purpose.

[0148] It is conceivable that the container recognition model has been trained using a general, preferably independent, training dataset that is independent of specific or particular container processing devices / the container recognition model is in a state of having been trained using the aforementioned model.

[0149] To generate a training dataset, the method according to the invention includes providing and / or evaluating (pre-defined and / or pre-defined) image data.

[0150] The (predetermined and / or predeterminable) image data provided and / or evaluated for generating the training dataset is preferably image data generated or determined by (at least) the image acquisition device of the container processing device. In this case, the container processing device is preferably the container processing device whose container recognition model will be trained or retrained with the training dataset to be generated. In this case, (at least) one image acquisition device is preferably the image acquisition device of the container processing device whose container recognition model will be trained or retrained with the training dataset to be generated. This provides the advantage that the (post)training process has been specifically coordinated for the (finely configured) container processing device, so that the specific conditions of that particular container processing device, such as the optical characteristics of the image acquisition device or specific lighting conditions or reflection conditions or set conditions (such as container guidance), can be taken into account, for example, directly in the container processing device.

[0151] However, it is also conceivable that (the pre-defined and / or pre-defined data used to generate the training dataset) has been or is being generated or transmitted by (at least partially and / or completely) container processing devices with the same (but different) structure.

[0152] Preferably, image data (provided and / or evaluated and / or predetermined and / or pre-determinable for generating the training dataset) is recorded and / or generated and / or determined at a location different from (relative to the location where the training dataset is generated), preferably directly near or at the container processing device, where the container recognition model of the container processing device will be post-trained and / or trained with the training data to be generated, preferably during the (current) operation of the container processing device. Thus, the different location is particularly located outside and preferably spaced from the building and / or operating area in which the container processing device is arranged. Preferably, the image data (pre-determined and / or pre-determinable for generating the training dataset) generated or determined at or within the container processing device is transferred to an external storage device (relative to the container processing device), which is particularly accessible via the Internet (and / or via public and / or private networks, especially at least partially wired and / or wireless networks).

[0153] Image data sequences can also be loaded or transferred to external storage devices (based on the cloud). Preferably, a training dataset can be generated there based on the image data sequences. Therefore, a container recognition model can be trained there.

[0154] Preferably, the method for generating the training dataset includes retrieving and / or receiving image data (pre-reserved and / or pre-preservable for generating the training dataset) from an external storage device.

[0155] However, it is also conceivable to receive image datasets as image data using user data obtained as part of user input.

[0156] The image data is preferably realistic image data. However, it is also conceivable that synthetic image data and / or (especially augmented images based on realistic image data) are intended as image data. This provides the advantage that, for example, situations or states that do not occur or only rarely occur in the normal operation of the container handling device (e.g., a hidden / fallen container) can be reset or simulated and used for training.

[0157] Preferably, image data (used to generate a training dataset) and / or to be evaluated (pre-defined and / or pre-defined) and / or acquired by an image acquisition device is used to image a transport area along the transport path of a container to be transported in a container processing device (from a transport device), wherein preferably, at least one container and preferably multiple containers are located in the transport area. These containers are preferably containers of the type for which evaluation is performed in a machine learning container recognition model to be trained and / or to be post-trained.

[0158] Preferably, at least a portion (particularly preferably all) of the image data images the same transport area (especially the same container handling equipment). Different image data (different image records) preferably have different numbers and / or distributions of containers within the transport area.

[0159] According to the present invention, a method is provided for subdividing a transport area into at least one transport segment, preferably at least two, preferably multiple transport segments, and preferably transport segments that do not overlap with each other. In this case, each transport segment forms a segment of the transport area connected along the transport path. For example, it is conceivable that the subdivision instruction divides the transport area into at least 5, preferably at least 10, preferably at least 13, and particularly preferably at least 20 different transport segments.

[0160] Preferably, the subdivision is provided such that the transport area can consist of multiple transport segments. Preferably, each area of ​​the transport area is contained within only one transport segment. Preferably, the multiple transport segments completely cover the transport area. It is also conceivable that the multiple transport segments only cover the transport area in sections, such as the central area of ​​the transport area.

[0161] According to the present invention, image segment data is determined and / or generated from image data based on the subdivision of the transport area. In this case (especially for evaluating image data to be evaluated to generate a training dataset), the image segment data is provided to the training data container recognition model as an input variable for evaluating the image segment data about containers located in the transport segment.

[0162] Image segment data can be determined and / or generated from image data by cropping (or cropping separately) and / or subdividing and / or segmenting and / or selecting image data corresponding to the subdivision of the transportation area.

[0163] Image segment data can be generated by processing image data before and / or after selection (cropping, trimming, subdivision, and / or segmentation) to determine and / or generate image segment data by sharpening and / or changing brightness.

[0164] Specifically, the training data container recognition model is not a machine learning container recognition model (of the container processing device) that is to be trained and / or retrained with the training dataset to be generated.

[0165] The training data container recognition model is preferably a (trained) machine learning model. Preferably, the training data container recognition model is adapted and designed to map image data (here, image segment data provided as input variables) to output variables, the output variables being features of containers in the image data or image segment data provided as input variables.

[0166] The training data container for machine learning to identify the model is preferably based on (artificial) neural networks. Preferably, the neural network is formed as a deep neural network (DNN), wherein the parameterizable processing chain has multiple processing layers, and / or so-called convolutional neural networks (CNNs) and / or recurrent neural networks (RNNs) and / or other DNN layer classes.

[0167] Preferably, the image segment data is evaluated using at least one (computer-implemented) computer vision method by applying a training data container recognition model to the image segment data provided by the seat input variable, in which a (computer-implemented) perception task and / or detection task is performed, such as a (computer-implemented) 2D object recognition method and / or a 3D object recognition method and / or a (computer-implemented) method for (preferably semantic) segmentation and / or a (computer-implemented) object classification (“image classification”) and / or a (computer-implemented) object localization and / or a (computer-implemented) edge recognition method.

[0168] Preferably, by using a training data container recognition model, the same type of computer vision method as using a container recognition model to evaluate the image data provided to it can be performed to evaluate the image data provided to it.

[0169] Preferably, (with the aid of a training data container recognition model) a recognition result (as an evaluation result) is determined, preferably in relation to the image segment data provided separately and / or the image data from which the image segment data is determined. The recognition result is preferably a recognition result or evaluation variable of the same type determined by the container recognition model.

[0170] Therefore, by training a data container recognition model or using the trained data container recognition model, evaluation variables representing at least one of the following can be determined and / or provided for output and / or transmission: the number of containers identified in the transport segment of the corresponding image segment data and / or the location (or location) of the identified containers (or each container identified in the transport segment) and / or the orientation of the containers (or each container identified in the transport segment) and / or the type of the containers (or each container identified in the transport segment) and / or the state of the containers (or each container identified in the transport segment) and / or the speed of the containers (or each container identified in the transport segment) and / or the occupancy rate of the transport segment or segment along the transport path relative to the containers located therein and / or the distribution of containers in predetermined areas within the transport segment.

[0171] In a preferred method, all image segment data associated with the subdivision of the transport area are provided as input variables to the training data container recognition model. Preferably, the evaluation of image data (based on which the image segment data is determined) about containers located in the transport area is performed from the evaluation of the individual image segment data (using the training data container recognition model).

[0172] In this way, the evaluation results of the image segment data generated from the subdivision of the image data are recombined, and a reference to the (original) image data is established. Therefore, the evaluation results can be associated with those regions of the image data corresponding to the image segment data. In this way, evaluation results associated with the image data (covering the transportation area) are obtained. Annotation data associated with the corresponding image data is advantageously determined from the evaluation results. In this way, a training dataset is advantageously generated, which includes the corresponding image data and annotation data (determined based on the image segment data determined according to the subdivision, through its evaluation results).

[0173] The provided segmentation can be applied to all image data. However, it is also conceivable to provide multiple segments that are applied to different image data (e.g., different transport areas or the same transport area can be mapped) to determine and / or generate image segment data.

[0174] In this context, evaluating image segment data rather than image data mapping transportation areas offers the advantage of using a smaller training container to identify the model. This advantageously allows for faster and less complex generation of the training dataset (e.g., in terms of providing or generating the training container recognition model and / or the required data processing time).

[0175] For example, a smaller training container recognition model can be understood as a training container recognition model with relatively few parameters to be trained and / or post-trained. For example, a (smaller) training container recognition model can be used that includes (only) a set of fewer than 10 million, for example, only 7 million, training and / or post-trained parameters, compared to a (larger) training container recognition model with a set of, for example, 100 million training and / or post-trained parameters.

[0176] With such a smaller training container recognition model, it is faster to obtain evaluation results for individual image segments compared to a larger training container recognition model. This offers the advantage that, on the one hand, the training process can be completed more quickly, and on the other hand, the provided image data can be evaluated even in real time (especially relative to the operation of the container processing device).

[0177] Preferably, the training container recognition model used is a model that only provides it with image data (or image segment data) as input variables or processes it, and its image size does not exceed 1000 × 1000 pixels, preferably 900 × 900 pixels, more preferably 750 × 750 pixels, and particularly preferably 640 × 640 pixels.

[0178] The segmentation is preferably carried out in such a way that the image segment data (provided as input variables to the training container recognition model) does not exceed 1000 × 1000 pixels, preferably 900 × 900 pixels, more preferably 750 × 750 pixels, and particularly preferably 640 × 640 pixels.

[0179] In a preferred method, the subdivision of the transport area is predetermined by and / or can be predetermined by the operator. Preferably, the method may include receiving and / or retrieving characteristic data of at least one subdivision. The data characterizing at least one subdivision can preferably be input by the operator via a human-machine interface and / or stored and retrieved from an (external) storage device after being received via operator input.

[0180] The subdivision can preferably be changed by the operator.

[0181] In a further optimized method, the transportation area is subdivided such that the resulting transportation segments have a capacity of up to 300 containers, preferably up to 250 containers, and preferably up to 200 containers in each case. The containers are those in the image data to be evaluated. This also advantageously allows the training data container recognition model to be used to generate a training dataset whose high recognition accuracy is limited to input image data with approximately 300, 250, or 200 containers, and whose recognition accuracy decreases when the number of containers to be recognized is very high. This achieves high-quality training data to be generated.

[0182] In a further optimized method, the transportation area is subdivided such that the maximum capacity of containers in each transportation segment is between 10 and 300, preferably between 20 and 200. This advantageously achieves both rapid evaluation of the provided image data and maximum possible recognition accuracy.

[0183] The provided subdivision (at least one) can be a conventional subdivision. In this case, the rules are specifically understood as rules for subdivision relative to the main transport path rules. In particular, the extent seen along the main transport path or the geometric extent of the transport segments resulting from the subdivision is substantially the same in each case. The main transport path can be a central and / or average transport path. It can also be imagined that, geometrically, the main transport path is a central transport path (where the geometric center specifically refers to the width of the transport area, which is specifically perpendicular to the corresponding transport direction).

[0184] However, it is also conceivable that the subdivision is irregular. Irregular subdivision can offer the advantage that, for example, the local geometric conditions of the transport area can be taken into account, which can make the identification of containers located in the corresponding transport sections relatively easy and / or particularly difficult. Thus, for example, in the case of narrow, winding sections of the transport area and / or transport sections where containers to be transported often accumulate and / or even wedge / fall out, there may be situations where identification is difficult. Here, a relatively finer subdivision is preferred.

[0185] In the preferred method, the subdivision of at least one segment depends on at least one geometric parameter, which is a characteristic of the geometric contour of the transport area (at least the segment). The geometric parameter can be, for example, characteristic variables such as curvature and / or width (of the segment of the transport area) and / or branches and / or openings. Therefore, in cases where the transport area in a segment has strong curvature (small radius of curvature) and / or a relatively large width (compared to other segments of the transport area), a smaller transport segment subdivision (relative to the geometric extent along the main transport path) can be selected. Width specifically refers to the geometric extent of the corresponding segment perpendicular to the main transport path.

[0186] The determination of image segment data is preferably carried out in such a manner that the image data is subdivided into transport segments not only relative to their arrangement and / or along the extension of the main transport path, but also cut to a certain size in a direction perpendicular to the main transport path and / or the transport path, in order to determine the image segment data. This provides the advantage that image data points that do not map the transport area can be removed in this way, as these image data points are not necessary for evaluating image data about containers located on the transport area. In this way, faster and more accurate identification of containers can be advantageously achieved.

[0187] The proportion of image segment data points in the mapped transportation segment in the total image segment data points (provided to the training data recognition model) is preferably greater than 30%, preferably greater than 40%, preferably greater than 50%, and particularly preferably greater than 60%.

[0188] In a further preferred method, the transportation area is subdivided automatically. For example, it is conceivable that the transportation area depicted in the image data can be automatically identified (through image evaluation and / or recognition methods), and the subdivision of the transportation area into transportation segments can be determined (automatically) as a function of the (geometric) contour of the transportation area.

[0189] It is also conceivable that the subdivision of the transportation area is determined iteratively. For example, it is conceivable that for a predetermined subdivision, a recognition accuracy is determined, and as a function of the recognition accuracy, it is determined whether finer subdivision can achieve an increase in recognition accuracy. To this end, it is preferable to compare the determined recognition accuracy with a predetermined and / or predeterminable threshold.

[0190] Additionally or alternatively, it is also conceivable to analyze the (determined) (geometric) profile of the main transport routes in the transport area and / or parts of the transport area with regard to at least one geometric parameter (e.g., curvature behavior), and to automatically determine the subdivisions based thereon.

[0191] In a preferred method, the image data (provided and / or predetermined and / or recorded by at least one image acquisition device for generating a training dataset) exists as color image data, a color video sequence, or a color image sequence. Preferably, the color image data is provided to the container recognition model and / or the color image segment data is provided to the training data recognition model.

[0192] The image segment data provided to the training data recognition model preferably has the same color spectrum as the corresponding image data.

[0193] Specifically, when determining image segment data based on image data, color filters are not used. The advantage of this is that higher recognition accuracy can be achieved. Conversely, in this case, the use of color filters can be omitted because, in terms of the computational workload, the separate processing of the image segment data generated from the segmentation is sufficiently efficient to utilize the training data recognition model.

[0194] In a further preferred method, the color values ​​of the image segment data provided to the training container recognition model and / or the image data provided to the container recognition model are substantially extended over the same color range as the corresponding or corresponding original image data acquired by the image acquisition device.

[0195] In a further preferred method, the container recognition model and / or test data container recognition model are used for preferred semantic segmentation of image data or image segment data (to be evaluated in each case).

[0196] In the case of semantic segmentation, in particular image data or image segment data or data derived therefrom, each pixel is assigned a category (for the classification of objects), in this case, such as "container" or "non-container" (category annotation).

[0197] These categories can be, for example, categories used to classify container types (e.g., metal cans, bottles, glass bottles, PET bottles, etc.) and / or categories used to classify corresponding container dimensions (e.g., fill volume, container height, diameter) and / or categories used to classify container equipment (e.g., body label, neck label, seal type, seal color, etc.).

[0198] In a further advantageous approach, the occupancy rate of transport segments and / or transport areas is determined based on preferred semantic segmentation.

[0199] In the preferred method, generating annotation data and preferred training data (as a function of the recognition result and / or as a function of the performed, preferred semantic segmentation) includes assigning the generated annotation data to the corresponding image data.

[0200] Preferably, preferred semantic segmentation of the image data (at least the transportation area) is performed based on preferred semantic segmentation of the image segment data performed by the training data recognition model and / or using the training data recognition model (generated according to the subdivision of the transportation area). In this case, all image data points that do not correspond to image segment data points can be assigned the category "non-container".

[0201] Preferably, annotation data is generated for the image data based on preferred semantic segmentation of multiple image segment data and / or image data (at least the transport area of ​​the image data, based on which the image segment data is determined) (characterized by preferred semantic segmentation of the image data).

[0202] Preferably, based on the performed preferred semantic segmentation, at least one occupancy variable is determined, which characterizes the occupancy rate of the transport area and / or transport segment of the image data. It is also conceivable to determine an occupancy variable relative to the transport area based on the corresponding occupancy variable for the transport segment.

[0203] Pixel-based or data-point-based occupancy determination offers the advantage of very accurate occupancy determination.

[0204] The occupancy rate variable can be determined based on the container type.

[0205] Preferably, the position (and / or orientation) of the identified container is determined in the image data and / or the coordinates of the identified container (preferably each) are determined (and assigned to the corresponding image data) in the (predetermined) coordinate system (world coordinate system) of the (corresponding) container processing device.

[0206] In a further preferred method, annotation data is generated as a function of the recognition result, and preferably training data including the association between the generated annotation data and the image data is generated. The annotation data may include information about (predefined) categories, labels, recognition and / or labeling of specific objects, localization and / or segmentation, and / or video annotations (feature points, polygons, bounding boxes used to label objects in different frames). For example, it is conceivable that the annotation data includes the coordinates or position information of containers detected in the image data. It is also conceivable that the annotation data includes or indicates the labels of the identified containers, such as by means of rectangles and / or boundary lines and / or borders.

[0207] It is conceivable that for each image data to be labeled, a (text) file with annotation data associated with the image data is generated. Preferably, the associated training dataset includes the image data and its associated annotation data.

[0208] Preferably, more than 100, preferably more than 1000, different images or image data are labeled in the manner described above, or annotation data is created for this purpose (and assigned to the corresponding image data). Preferably, a training dataset is generated from this.

[0209] Preferably, the container handling equipment is selected from the group consisting of: transport equipment (e.g., conveyor belts) for transporting containers, buffering equipment for temporary buffer containers, pasteurization equipment (e.g., tunnel pasteurizers), heating equipment for heating preforms, molding equipment for molding plastic preforms into plastic bottles, sterilization equipment for sterilizing containers (especially plastic preforms), manufacturing equipment for manufacturing glass bottles, manufacturing equipment for filling containers with products, inspection equipment for inspecting plastic preforms or bottles, labeling equipment for labeling containers, sealing equipment for sealing containers (especially filled containers), control devices, packaging devices, direct printing equipment for printing containers, assembly equipment for assembling multiple containers into components or groups, etc.

[0210] Preferably, the container flow is a sequential or sequential (especially continuous) flow of containers (along the transport path). The container flow can be guided or transported (by transport means) in certain areas, preferably within the entire container handling equipment (as a mass flow), in a single-channel or multi-channel manner.

[0211] The transport equipment may also be a high-density transport vehicle, used to transport multiple containers preferably in a multi-channel and / or unordered manner. The transport device may also be a buffer zone, used to buffer multiple containers preferably in a multi-channel and / or unordered manner.

[0212] The container can preferably be transported or guided at least in sections and preferably upright or vertically along the entire transport area (by the transport device).

[0213] Preferably, the transport device is adapted and designed to guide or transport multiple containers in at least segmented manner, preferably along the entire transport area, where the containers are under back pressure.

[0214] Preferably, the transport device is adapted and designed to transport and / or guide (at least within the transport area) at least one container per hour, preferably at least 5,000 (especially to be identified) containers per hour, preferably at least 20,000, preferably at least 100,000 (especially to be identified) containers per hour, particularly preferably at least 140,000 (especially to be identified) containers per hour, and this is performed during the working operation of the container handling equipment. Preferably, the transport device is adapted and designed to transport and / or guide up to 150,000 (especially to be inspected) containers per hour (at least within the transport area), and this process is carried out during the working operation of the container handling equipment.

[0215] Preferably, the transport device in a single-lane transport area is suitable and intended (at least in the single-lane transport area) to transport and / or guide at least 100,000 containers per hour, and / or at most 150,000 containers per hour, and the process is carried out within the working operation of the container handling equipment.

[0216] The present invention also relates to a method for training, particularly for post-training machine learning, a container recognition model, preferably a real-time image evaluation device, and a container processing device, wherein (in the operation of the container processing device) containers can be guided along a predetermined transport path, particularly in the form of a container flow.

[0217] In this context, image data can be generated (during operation) using an image acquisition device. This image data represents the features of the transport area (imaged) along the transport path (preferably relative to the container of interest to be identified). The image data can be provided as input variables to a container identification model to perform an evaluation of the containers located within the transport area, particularly in real-time.

[0218] According to the present invention, training data generated according to one of the above methods (especially according to the above embodiments) is used to train and / or post-train a machine learning container recognition model.

[0219] Therefore, in the context of the method according to the invention, it is also suggested that labeled or annotated image data (as a training dataset) be used for training a container recognition model, preferably for post-training, wherein the underlying image data is subdivided into smaller image segments to be evaluated by determining image segment data according to the provided subdivision. These smaller image segments (image segment data) are preferably evaluated (regarding containers located in a transport area or transport segment) by a machine learning-trained container recognition model. Based on the evaluation of individual smaller image segments, annotation data of the image data (regarding containers shown in the image data, e.g., occupancy of a transport area) is preferably generated.

[0220] The container identification model and the container processing device can be configured individually or in combination with each other with all the features described above in conjunction with the above methods (and vice versa).

[0221] Preferably, the container processing device records image data for generating a training dataset using one or more image acquisition devices and transmits it to an external storage device. Preferably, the image data (in each case) images the transport area of ​​the container processing device. Preferably, at different recording times and with different distributions of containers in the transport area, image data (generated and determined by the image acquisition device of the container processing device) is sent to the external storage device.

[0222] Image data stored on an external storage device is preferably retrieved (particularly by or triggered by a training data generation device). Preferably, a training dataset is generated based on the retrieved image data (by means of the method described above according to a preferred embodiment). In this case, the generation of the training dataset is preferably performed remotely relative to the container processing device.

[0223] Preferably (preferably by a container processing device), the training dataset generated (by a training data generation device) is retrieved from an external storage device.

[0224] Preferably, the data to be processed, particularly image data recorded by means of at least one image acquisition device, is provided as input variables to the container recognition model or (artificial) neural network. Preferably, the container recognition model or artificial neural network maps the input variables to output variables as a function of a parameterizable processing chain, wherein the output variables are preferably the state of the identified containers, the position of each identified container, the speed of each identified container, and / or the distribution and / or number of containers in the transportation area.

[0225] It is also conceivable that the container identification model determines and provides (for transmission and / or output) as output variables, which are the characteristics of the (current) location / position of (each) identified container, and / or the characteristics of the identified container type and / or the occupancy rate of the transport area and / or transport segment.

[0226] Preferably, the container handling equipment and / or real-time evaluation equipment, as a function of the output variables of the container identification model, determine characteristic variables as the speed of the identified containers (in the transportation area) and / or the state of the identified containers (in the transportation area) and / or the distribution and / or quantity of the identified containers in the transportation area.

[0227] Preferably, the generated training data is used to (post)train a machine learning container recognition model or an artificial neural network, wherein the parameterizable processing chain is parameterized through training. Preferably, an iterative training process is selected and performed until a predetermined recognition accuracy is achieved.

[0228] Preferably, the external storage device is a non-volatile storage device, particularly a cloud-based storage device, and / or an external server (containing storage devices), wherein the storage device is accessed, in particular, via the Internet (and / or via public and / or private networks, particularly at least partially wired and / or wirelessly connected). The external server should be specifically understood as an external server associated with container processing devices and / or real-time evaluation devices, particularly a backend server.

[0229] For example, an external server serves as the backend, particularly for container processing equipment manufacturers or service providers. This external server is adapted for managing image data (specifically, multiple image acquisition devices and / or multiple container processing devices) and / or regulating the container processing devices. The functions of the backend or external server can be executed on an (external) server farm. The (external) server can be a distributed system.

[0230] The present invention also relates to a method for identifying containers located in a transport area (of a container handling facility) and / or for determining the occupancy rate of the transport area of ​​a container handling facility.

[0231] According to the present invention, a machine learning container recognition model trained according to the above-described method for training, particularly for post-training, machine learning is used to identify and / or track characteristic variables of containers located in a transport area and / or to determine the occupancy rate of the transport area and / or characteristic (occupancy) variables of at least one transport segment of the transport area.

[0232] It is conceivable that a segmentation of the transport area is provided as described above. Preferably, the transport area is subdivided into transport segments according to the segmentation, and at least one occupancy variable characterizing the occupancy rate in the respective transport segment is determined.

[0233] The container recognition model is preferably suited to and designed to perform (as described above) preferred semantic segmentation (of the image data). Preferably, the corresponding occupancy variable (see, as described above, categories "container" and "non-container") is determined using the per-data-point or per-pixel class assignment obtained through preferred semantic segmentation. For example, to determine the corresponding occupancy rate of a transport area or transport segment, a relationship can be formed between the proportion of data points or pixels with the category assignment "container" and the total number of data points or pixels representing the corresponding transport area or transport segment.

[0234] Preferably, the container processing equipment has a real-time image evaluation device, particularly a processor-based real-time image evaluation device, which is suitable and intended to perform an evaluation, particularly a real-time evaluation, of containers located in the transport area by means of a machine learning container recognition model, for which image data can be provided as input variables to the container recognition model (and the real-time image evaluation device).

[0235] Therefore, in the context of the method according to the invention, it is also suggested to use a highly accurate machine learning container recognition model obtained by post-training as described above to identify containers (and derive the recognition results) and / or determine the number and / or speed and / or distribution and / or occupancy of containers located in the transportation area.

[0236] Preferably, the container identification model and the container processing device may be equipped with all of the above features individually or in combination with each other (or vice versa).

[0237] It is also conceivable that the container identification model determines and provides (for transmission and / or output) as output variables, which are characteristic location parameters of the (current) location of (each) identified container and / or characteristic parameters of the identified container type.

[0238] Preferably, the container handling equipment and / or real-time evaluation equipment, as a function of the output variables of the container identification model, determine characteristic variables, which are the preferred speed of each identified container (in the transport area) and / or the preferred state of each identified container (in the transport area) and / or the location and / or distribution and / or quantity and / or container type and / or occupancy rate of the identified containers in the transport area.

[0239] Preferably, at least one detected container is tracked based on the detection results and / or the determined quantity and / or distribution and / or location and / or container type. For this purpose, image data recorded at different times is used, and evaluation is performed in each case (particularly in real-time). Preferably, the location of the container identified in each case is determined based on the image data recorded at different times. Furthermore, preferably, one or more velocities of the container are determined based on the determined location and recording time and / or based on the transport speed achieved by the transport device. Preferably, based on the determined location and / or speed of the container, the expected dwell area of ​​the container at at least one future time point is determined by applying a Kalman filter.

[0240] Preferably, this tracking is applied to multiple, and more preferably all, containers located in the transport area.

[0241] It is also conceivable to provide a virtual congestion switch based on characteristic (occupancy) variables determined for the occupancy rate of a transport area and / or for identified containers, their location and / or speed and / or their determined future expected dwell areas, wherein container congestion resulting in a predetermined probability (preferably occurring in the future) is determined or predicted. Preferably, in the event of determined or predicted container congestion, a warning message is provided for transmission and / or output to the user.

[0242] It is also conceivable that different warning levels could be determined as a function of occupancy or container distribution in the transportation area, and corresponding messages could be provided to transmit and / or output to the user.

[0243] The present invention also relates to a preferably processor-based training data generation device for automatically generating training datasets for training, particularly for training, a machine learning container recognition model for a post-training container processing device, wherein containers can be guided along a predetermined transport path, wherein image data can be generated by means of an image acquisition device, the image data being features of the transport area along the transport path, and can be provided as input variables to the container recognition model for performing an evaluation, particularly a real-time evaluation, of containers located in the transport area.

[0244] In this context, the training data generation device (used to generate training datasets) is suitable for and designed to evaluate image data (especially for determining recognition results).

[0245] According to the present invention, the training data generation device is adapted and intended to determine image segment data from image data based on the provided division of a transport area into at least one transport segment, preferably at least two, preferably multiple transport segments, preferably non-overlapping transport segments, and to provide the image segment data as input variables to a training data container recognition model for evaluating the image segment data associated with containers located in the transport segments and determining the recognition result.

[0246] The training data generation device can be identified and / or configured to perform one or more of the aforementioned method steps for training, particularly for post-training machine learning container recognition models (container processing devices).

[0247] The present invention also relates to a container processing apparatus for processing containers, the container processing apparatus having a transport device adapted and intended to transport containers along a predetermined transport path, the transport device having an image acquisition device by means of which image data can be generated, the image data being features of the transport area along the transport path.

[0248] The container handling equipment has a real-time image evaluation device, particularly a processor-based real-time image evaluation device, which is suitable for and designed to perform evaluation, particularly real-time evaluation, of containers located in the transport area by means of a machine learning container recognition model, for which image data can be provided as input variables to the container recognition model (and the real-time image evaluation device).

[0249] The machine learning container recognition model is preferably a container recognition model that has been trained and / or post-trained using a training dataset generated according to one of the methods described above.

[0250] Preferably, the container is processed based on the recognition result of the container recognition model or (at least) the output variable of the container recognition model or based on the container recognized by the container recognition model.

[0251] In this case, the container handling device may have, individually or in combination, all the features described above in conjunction with the container handling device (and vice versa), and is adapted to perform all the method steps described above in conjunction with the method for identifying containers located in the transport area of ​​the container handling device.

[0252] Container recognition models may possess one or more of the features mentioned above, either individually or in combination.

[0253] The present invention also relates to a computer method or computer method product, comprising method apparatus, particularly method code, which represents or encodes at least some and preferably all of the method steps of the method (described above) according to the present invention (for generating a training dataset and / or for training a container recognition model for a container processing device), and preferably represents or encodes an embodiment of the preferred embodiments described above, and is designed for execution by a processor device.

[0254] The present invention also relates to a data storage device that stores at least one embodiment or a preferred embodiment of a computer program according to the present invention.

[0255] The present invention has already described containers to be identified. In this context, the invention can also be transferred to objects to be identified in machine image processing (machine learning-based image evaluation models) commonly used for image recognition in the beverage and / or pharmaceutical industries, particularly in container handling equipment in the beverage and / or pharmaceutical industries. The applicant also reserves rights to claims on related subject matter, particularly with respect to methods for generating training datasets and training data generation equipment.

[0256] Further advantages and implementation methods are evident from the accompanying drawings:

[0257] in:

[0258] Figure 1 A schematic diagram illustrating image recording by a container processing device and an illustration of a preferred method for generating a training dataset are shown.

[0259] Figure 2 , Figure 4 Different input images are shown;

[0260] Figure 3 , Figure 5 Showing the evaluation from Figure 2 , Figure 4 The corresponding evaluation image is obtained from the input image; and

[0261] Figure 6 A rough schematic structure of a container processing apparatus according to a preferred embodiment of the present invention is shown.

[0262] Figure 1 A schematic diagram of image recording BD1 of container handling equipment 1 is shown. Container handling equipment 1 has a transport device 2 (here, a bulk conveyor, in which containers 10 (here, metal cans) are transported in a multi-row and disordered manner). Reference numerals 16 and 18 respectively denote the lateral boundaries or guide rails of the transport device 2. They may, for example, define the area of ​​the transport device (e.g., a conveyor belt).

[0263] As can be seen, the transport area of ​​the transport device 2 shown is divided into different transport sections, and these transport sections are... Figure 1 The figures are indicated by the reference numerals B1, B2, B3, B4, ..., B9, B10.

[0264] The evaluation of image records using a machine learning-based container recognition model can generate the corresponding number of containers within the respective transport segments B1, B2, ..., B10. Optionally, or based on the number of containers, the corresponding occupancy rate of the respective transport segment can be indicated and / or determined (this occupancy rate corresponds to the ratio between the number of containers currently transported in the transport segment and the maximum occupancy number).

[0265] In transport segment B1, 54 containers are transported in image data BD1. For the maximum occupancy of 119 containers (assuming this), this corresponds to an occupancy rate of 45.38%.

[0266] In transport sections B2-B5, at the time image data BD1 was recorded, there was no transport container 10. Therefore, the occupancy rate of these sections was 0%.

[0267] Transport sections B6 and B7 are only partially, not completely, occupied by container 10.

[0268] In transport segments B9 and B10, as shown in image data BD1, the maximum number of containers that can be occupied within the respective transport segment is considered. Here, the (instantaneous) occupancy rate of transport segments B9 and B10 is 100%.

[0269] Figure 1 A diagram of a preferred method for generating the training dataset is also shown. Here, the transport region imaged in the recorded image data BD1 is decomposed into individual transport segments B1, ..., B10, and the image data (corresponding to the entire image BD1) segmented into individual transport segments is provided to an image evaluation model (e.g., a segmentation model) and / or a container recognition model as input image data for evaluation relative to the container 10 located in the transport segment.

[0270] The applicant has found in a complex series of experiments that when the image evaluation model or recognition model is applied to the full image BD1, the recognition accuracy is low; after segmentation (shown here by, for example, sub-region B3), all objects (here, containers or metal cans) are identified.

[0271] Therefore, this means that image evaluation models or container recognition models with preprocessing (e.g., cropping, sharpening, and / or brightness) are preferred as automatic taggers.

[0272] Figure 2 and Figure 4 Different input image data (input images) E1 and E2 are shown respectively, which in turn depict the container handling device 1 with the transport device 2 and the container 10 (in this case, a metal can) transported therefrom, and the input image data are evaluated by a processor-based image evaluation device (preferably using an image evaluation model or a recognition model based on machine learning).

[0273] Reference numerals 16 and 18 again indicate the corresponding lateral boundaries of the receiving area 14 of the transport device 2, in which the container 10 can be received for transport.

[0274] Here, the image evaluation device (e.g., a machine learning recognition model) preferably performs (semantic) segmentation. In (semantic) segmentation, a class is assigned to each pixel of the (input) image or each data point of the input image data. Here, "container" (or "metal can") is used as the class. All image data or pixels belonging to the (identified) container (or metal can) are assigned to the class "container". Figure 3 and Figure 5 In the evaluation images A1 and A2, these image data points or pixels are represented in white and indicated by reference numeral P10. The category "container" (or corresponding to the category "metal can") should be specifically understood here as "image points belonging to (any) container (of the container mass flow)," because a single container or a single metal can cannot be assigned to a point for classification. Even if multiple "container pixels," i.e., data points of the category "container," are identified, the evaluation results do not specifically include information about whether they are one or multiple containers (or one or multiple metal cans), or, for example, whether two adjacent data points of the category "container" belong to the same container or map to (adjacent) regions of the same container.

[0275] All other image data points that do not belong to the object identified as a container (or metal can) are assigned pixel values ​​different from the category “container” (or “metal can”), such as black pixel data points, “background pixel value” PH.

[0276] exist Figure 3 and Figure 5In the image, all image data points that are identified by the machine learning recognition model as not belonging to the container (or metal can) are mapped to black pixels.

[0277] These diagrams produce binary representations of white pixels (containers) and black pixels (non-containers).

[0278] Preferably, the occupancy rate of the transport device (the predetermined transport area) can be quickly determined through such evaluation, for example by determining the white image portion P10 of the corresponding evaluation images A1 and A2.

[0279] Figure 6 A rough schematic structure of a container processing apparatus 1 for processing a container 10 according to a preferred embodiment of the present invention is shown. Reference numeral 2 indicates a transport device for transporting the container 10 to a processing apparatus 12 for processing the container 10.

[0280] Reference numeral 4 indicates an image acquisition device (here, a camera) that captures images from above of the transport area indicated by reference numeral 20. Specifically, image data depicting the container 10 located in the transport area 20 is generated.

[0281] The acquired image data (optionally after preprocessing) is transmitted to the image evaluation device 6. The image evaluation device 6 performs an evaluation based on the image data to determine occupancy variables that characterize the occupancy rate of the transport area 20.

[0282] The image evaluation device 6 determines the occupancy rate variable based on at least segmented image data. Preferably, the occupancy rate variable is used to determine control variables for controlling the container processing device 1.

[0283] Reference numeral 8 indicates a control device, which preferably includes a container handling device 1, and is used to control the container handling device according to control variables, for example, to (automatically) perform processing functions on at least one and / or multiple containers. It is also conceivable that at least one control variable is used to control and / or regulate container flow and / or container throughput through the container handling device and / or transport speed and / or container supply and / or container discharge.

[0284] The applicant reserves the right to claim rights to all features disclosed in the application documents that are essential to this application and are novel compared to the prior art, whether individually or in combination. Furthermore, it should be noted that features that may be advantageous in themselves are also described in individual figures. Those skilled in the art will readily recognize that a particular feature described in a figure may be advantageous even without employing other features in that figure. Additionally, those skilled in the art will recognize that advantages can also be derived by combining several features shown in a single figure or different figures.

[0285] List of reference numerals

[0286]

Claims

1. A method for at least partially automatic, preferably fully automatic, control of a container handling apparatus (1) for handling containers (10), the container handling apparatus having a transport device (2) for guiding the containers (10) along a predetermined transport path according to the occupancy rate of at least one transport area (20) along the transport path, wherein image data (E1, E2) is generated by means of at least one image acquisition device (4) to image the containers (10) located in the transport area (20), and an image evaluation device (6) performs an evaluation based on the image data, particularly a real-time evaluation, to determine an occupancy rate variable characterizing the occupancy rate of the transport area (20), Its features are, The image evaluation device (6) performs segmentation of these image data (E1, E2) at least in segments to determine the occupancy variable.

2. The method according to claim 1, Its features are, The occupancy rate variable is determined based on at least one segment obtained through segmentation, and preferably based on at least multiple segments obtained through segmentation.

3. The method according to any one of the preceding claims, Its features are, The occupancy rate variable is determined based on the size of the transport area, which is a characteristic of the occupied area and / or maximum occupancy of the transport area.

4. The method according to the preceding claims, Its features are, Automatically determine the size of the transportation area.

5. The method according to any one of the preceding two claims, Its features are, In order to determine the size of the transport area, the image acquisition device (4) automatically generates (and evaluates) image data in a state where the transport device (2) is preferably set up by the operator by triggering the setting operation of the transport device (2), wherein the maximum number of containers (10) that can be accommodated in the transport area (20) is recorded.

6. The method according to any one of the preceding claims, Its features are, To determine the occupancy variable, the image data were evaluated in such a way that multiple adjacent containers (10) located in the transport area (20) were identified as a uniform container cluster that did not indicate the differences between individual containers (10).

7. The method according to any one of the preceding claims, Its features are, The occupancy rate variable is determined independently of the location and / or orientation of the container (10) located in the transport area (20).

8. The method according to any one of the preceding claims, Its features are, Segmentation is semantic segmentation.

9. The method according to any one of the preceding claims, Its features are, By segmenting, if image data points are assigned to containers and / or container clusters, the category "Container" (P10), selected from multiple, preferably two categories, is assigned to the image data points, and if image data points are not assigned to containers and / or container clusters, they are assigned to the category "Background" (PH).

10. The method according to the preceding claims, Its features are, The occupancy variable is determined based on those image data points (P10) assigned to the category "container" (P10) during the segmentation.

11. The method according to any one of the preceding claims, Its features are, This segmentation is based on a predetermined and / or predefined color range.

12. The method according to any one of the preceding claims, Its features are, The segmentation is performed using a machine learning-based segmentation model.

13. The method according to any one of the preceding claims, Its features are, At least one additional transport area (B7) is predetermined, and image data is generated by means of at least one image acquisition device (4) that images the container (10) located in the at least one additional transport area (20), and an image evaluation device (6) performs an evaluation, particularly a real-time evaluation, based on the image data to determine an occupancy rate variable that is a characteristic of the container (10) located in the at least one additional transport area (20), wherein the image evaluation device (6) performs at least segmentation of the image data to determine the occupancy rate variable.

14. A method for at least partially automatically determining the occupancy rate of a transport area (10) in a container processing device (1) for processing containers (10) located in the transport area (20), wherein the container processing device (1) has a transport device (2) by means of which the containers (10) can be guided along a predetermined transport path, wherein the transport area (20) is arranged along the transport path, wherein image data (E1, E2) is generated by means of at least one image acquisition device (4) that images the containers (10) located in the transport area (20), and the image evaluation device (6) performs an evaluation, particularly a real-time evaluation, based on the image data for determining a variable characterizing the occupancy rate of the containers (10) located in the transport area (20). Its features are, The image evaluation device (6) performs segmentation of these image data (E1, E2) at least in segments to determine the occupancy variable.

15. A container processing apparatus (1) for processing containers (10), the container processing apparatus (1) comprising: a transport device (2) suitable for and intended to transport the containers (10) along a predetermined transport path; at least one image acquisition device (4) for generating image data (E1, E2) depicting the containers (10) located in the transport area (20); and an image evaluation device (6) adapted and designed for performing evaluation based on the image data (E1, E2), particularly real-time evaluation. Its features are, The image evaluation device (6) is adapted and intended to perform the segmentation of these image data (E1, E2) at least in segments, and the container processing device (1) has a control device (8) for controlling the container processing device (1) at least partially automatically, preferably fully automatically, according to the determined occupancy rate variable.

Images