Methods and apparatus for monitoring filling and / or closing equipment and / or post-processing equipment.

Abstract

The present invention provides a filling and / or closing device and / or after-processing device having an improved automatic monitoring system, and a computer program for automatically monitoring them. [Solution] Images of the transport, delivery, and / or delivery area 2 of the filling and / or sealing equipment and / or post-processing equipment are captured using the camera system 10, and based on these images, the location of the primary packaging means is determined using the AI ​​model 120, which has been trained to detect primary packaging means within the transport, delivery, and / or delivery area 2 and to classify the detected primary packaging means, and the detected primary packaging means are assigned to a class.

Description

Technical Field

[0001] The present invention relates to a method and an apparatus for monitoring filling and / or closing equipment and / or post-treatment equipment, particularly for the pharmaceutical industry. The present invention further relates to filling and / or closing equipment and / or post-treatment equipment, and a computer program for monitoring filling and / or closing equipment and / or post-treatment equipment.

[0002] Filling and / or closing equipment for the pharmaceutical industry is used to fill liquid or powdered pharmaceuticals and biopharmaceuticals into vials (ampoules), infusion bottles, cartridges, disposable syringes, or similar containers. After filling and subsequent inspection, the containers are closed as soon as possible, for example, by closures such as stoppers and / or caps. In the context of this application, the containers and closures are referred to as primary packaging means. Post-treatment equipment is used for inspection, labeling, repackaging, or other processing of the primary packaging means that are filled and usually already closed.

[0003] In this case, the filling of sensitive or dangerous products is usually carried out within an isolator. In addition, equipment without an isolator for filling or post-treatment of pharmaceuticals and / or other products is also known.

[0004] Disruptions or malfunctions in the filling and closing process flow and / or the post-treatment process flow can lead to equipment stoppages. However, particularly when small batches are processed in filling and / or closing equipment and / or post-treatment equipment, it is highly desirable to avoid downtime as much as possible. Furthermore, an unexpected stoppage of filling and / or closing equipment and / or post-treatment equipment can also result in the product being no longer usable. This applies particularly - but not exclusively - to sensitive pharmaceuticals.

Summary of the Invention

[0005] An object of the present invention is to provide a method and apparatus for automatically monitoring filling and / or closing equipment and post-processing equipment, particularly for the pharmaceutical industry. A further object is to provide filling and / or closing equipment and / or post-processing equipment having an automatic monitoring system, and a computer program for automatically monitoring filling and / or closing equipment and / or post-processing equipment.

[0006] These objectives are achieved by subject matter having the features of claims 1, 8, 13, and 14. Further advantageous embodiments arise from the dependent claims.

[0007] According to a first aspect, a method is provided for monitoring filling and / or closing equipment and / or post-processing equipment, particularly for the pharmaceutical industry. In this method, images of the transport, inbound and / or outbound areas of the filling and / or closing equipment and / or post-processing equipment are captured using a camera system, and based on these images, an artificial intelligence model (AI model), which has been trained to detect primary packaging means and classify the detected primary packaging means, is used to determine where primary packaging means are located in the images and to assign the detected primary packaging means to a class.

[0008] In the context of this application, the terms “a” and “an” are used as indefinite articles and not as counting terms. In particular, embodiments provide that a series of images, rather than a single image, are captured and evaluated for continuous monitoring of transport, inbound and / or outbound areas.

[0009] Depending on the application, a single camera system or multiple camera systems are provided in the filling and / or closing equipment and / or post-processing equipment.

[0010] By using an AI model to evaluate a single image or multiple images from a series of images, it becomes possible to automatically monitor the transport, inbound, and / or outbound areas without prior knowledge of the location and / or time of occurrence of primary packaging means within the monitoring area.

[0011] Furthermore, in this case, the AI ​​model is also used to assign the detected primary packaging method to a class.

[0012] In one embodiment, classes are predefined by experts for the classification of primary packaging means, depending on the specific application. In another embodiment, it is provided that different types of primary packaging means can be detected, where one class is assigned to each type. In yet another embodiment, detected objects that cannot be assigned to a class are marked.

[0013] Alternatively or additionally, in some embodiments, certain types of primary packaging means are classified based on their orientation. In some embodiments, in this regard, primary packaging means provided in the correct orientation, primary packaging means tilted in the direction of transport, primary packaging means tilted sideways to the direction of transport, twisted primary packaging means, and / or upside down primary packaging means are distinguished for classification purposes.

[0014] In some embodiments, the AI ​​model is an artificial neural network. In particular, in several embodiments, the AI ​​model is a deep learning artificial neural network, commonly referred to as a deep neural network.

[0015] Training data for the AI ​​model can be pre-generated for typical transport, inbound and / or outbound areas and for different primary packaging means, and the AI ​​model is trained accordingly. In advantageous embodiments, to enhance the reliability of monitoring, at least one improvement of the trained AI model is made based on training data for specific transport, inbound and / or outbound areas and specific primary packaging means, and / or taking typical environmental parameters into consideration.

[0016] In one embodiment, it is provided that the results of monitoring are recorded, particularly electronically. In this case, the data to be recorded can be appropriately specified by those skilled in the art, depending on the application. In one embodiment, the above recording is also useful for checking the monitoring using an AI model. In one embodiment, the recorded conditions, along with appropriately prepared image files, serve as training data for the AI ​​model.

[0017] In one embodiment, the output of the AI ​​model is evaluated using a rule-based algorithm for the purpose of determining disorder.

[0018] In the context of this application, a rule-based algorithm is an algorithm that performs an evaluation of determined disturbances based on rules and information that can be appropriately specified in advance by a person skilled in the art. In this regard, the above method utilizes expertise relating to specific transport, inbound and / or outbound areas, as well as specific primary packaging means, and / or other boundary conditions.

[0019] In the context of this application, the term “disturbance” is generally used to describe an irregular element in the processing flow. In some embodiments, this disturbance has already led to equipment shutdown or a disruption of the processing flow. This disturbance is detected and processed, and in particular corrected, before equipment shutdown or a disruption of the processing flow occurs and / or before the disturbance affects the subsequent processing flow. In some embodiments, the disturbance may be an irregular element in the transport, ingress, or egress of primary packaging means, such as primary packaging means in the wrong orientation, missing primary packaging means, or an excess of primary packaging means. In some embodiments, a check for the presence of primary packaging means is performed. This is performed, for example, to determine whether there is a sufficient quantity of primary packaging means in the ingress area and / or whether there is no primary packaging means in the egress area, so that primary packaging means can be deposited in this area. The unexpected presence or absence of primary packaging means is referred to as a disturbance.

[0020] When a disturbance is determined, appropriate measures may be taken depending on the application. In a simple embodiment, any determined disturbance may result in the shutdown of the equipment until the disturbance is handled manually or automatically.

[0021] In advantageous embodiments, the determined disturbances are classified, i.e., assigned to a class and / or prioritized. That is, priority values ​​are assigned to the determined disturbances using a rule-based algorithm. In some embodiments, the same rule-based algorithm used for determining the disturbances is used for classification and prioritization, in which case, in the embodiment, the determination of disturbances, classification, and prioritization are performed in a single program execution. In other embodiments, separate algorithms are provided that are executed at least partially sequentially. For the classification of disturbances, classes corresponding to specific applications are defined in advance, particularly by experts. In some embodiments, it is then specified, based on the classification, how the determined disturbances will be handled. Priority values ​​for the determined disturbances for prioritization are defined in advance, particularly, depending on the specific application, by experts. In some embodiments, it is then specified, based on the priority value, whether or not the determined disturbances will be handled, and if so, when they will be handled. Classification and / or prioritization can, depending on the context, address the determined disturbances, as well as multiple disturbances that occur particularly in overlapping timelines.

[0022] In one embodiment, only the fact that a disturbance has occurred is detected, and the disturbance's location in the real environment is not identified. In an advantageous embodiment, it is provided that the location of a determined disturbance within a transport, inbound, and / or outbound area is identified. In one embodiment, in this case, the location is defined by coordinates that allow the detected disturbance to be located within a monitoring area. In other words, the location of the disturbance is detected so that the disturbance can then be handled by an operator or automatically.

[0023] In one embodiment, the monitored transport, receiving and / or delivery area is provided to have transport and / or sorting means for primary packaging means, and in particular, the transport, receiving and / or delivery area is provided to have sorting bins and / or one or more paths for stoppers or containers, and is monitored. In one embodiment, detected primary packaging means are classified by an AI model according to their orientation. In this case, blockages in the transport, receiving and / or delivery area, missing primary packaging means, incorrectly oriented primary packaging means, and / or primary packaging means in the wrong position are detected as disturbances. In this case, incorrectly oriented primary packaging means are primary packaging means that are lying down, twisted, or supplied upside down, and these may cause errors and / or stoppages in further processing. The term "incorrectly positioned primary packaging means" is used for primary packaging means that are in a position other than the intended position. In one embodiment, blockages in the receiving area are detected indirectly, and the gaps caused by the blockage, i.e., the absence of primary packaging means at a certain position, are identified. Alternatively or additionally, in some embodiments, filled primary packaging means and unfilled primary packaging means, and / or different types of primary packaging means, are distinguished by the classification of an AI model.

[0024] Alternatively or additionally, in some embodiments, primary packaging means are provided or stacked in a disorderly or orderly manner within a matrix (nest) within a monitored transport, receiving and / or delivery area. In particular, spatially partitioned trays or nests for primary packaging means from which primary packaging means are removed or into which primary packaging means are placed are monitored. In some embodiments, in this case, primary packaging means are classified by an AI model according to their orientation. Alternatively or additionally, in some embodiments, filled and unfilled primary packaging means, and / or different types of primary packaging means, are distinguished by the AI ​​model's classification. In this case, missing primary packaging means, overlapping primary packaging means, incorrectly oriented primary packaging means, and / or primary packaging means in the wrong position are detected as disorder. Primary packaging means seated on top of other primary packaging means or nests or similar are referred to as overlapping primary packaging means.

[0025] In an advantageous embodiment, a camera system for monitoring the transport, inlet, and / or outlet area is positioned above the transport, inlet, and / or outlet area, offset from the transport, inlet, and / or outlet area, so as not to disturb the primary supply air to the transport, inlet, and / or outlet area, and the optical axis of the camera system is inclined with respect to the vertical axis. In the context of this application, primary supply air means supply air from a source or filter, particularly from a HEPA filter, in a unidirectional airflow that is at least substantially free of particles. Depending on the application, minor influences on the primary supply air are acceptable, or are classified as disturbances in the primary supply air if the influence can be determined by known measurement methods. In particular, for the handling of sensitive products in isolators or under fume hoods, contamination of the primary packaging means and components of the equipment in contact with the primary packaging means should be prevented by primary supply air that is at least substantially free of other contaminants. Accordingly, the camera system is mounted so as not to disturb the primary supply air to the transport, inlet, and / or outlet area, at least in such applications.

[0026] In one embodiment, within the feed-in and / or feed-out area, if turbulence is determined, optionally with a corresponding classification or prioritization of the turbulence, a signal is issued to the operator so that the turbulence is processed. In one embodiment, the operator manually processes the turbulence. In one embodiment, in the case of an isolator, the manual processing is carried out with the intervention of gloves.

[0027] In an advantageous embodiment, the determined turbulence is processed using a manipulator, which can be moved by a central machine controller, by a decentralized manipulator controller, and / or by a manually operable controller for the purpose of processing the determined turbulence. In one embodiment, the manipulator is arranged within the isolator housing. In one embodiment, an environmental interference contour is defined to prevent the manipulator from colliding with components of the equipment or the primary packaging means. In this case, a path plan for the autonomous movement of the manipulator is carried out such that the planned path does not intersect the interference contour. In the case of a manually operable controller, access to positions along a path that intersects the interference contour or to positions within the interference contour is blocked. In one embodiment, in this case, a virtual interference contour is also defined to prevent access to or intersection with positions that affect the safety of the processing.

[0028] In one embodiment, the manipulator comprises grippers, in particular servo grippers, vacuum grippers, pneumatic grippers and / or magnetic drive grippers. In one embodiment, different-sized objects can be gripped using the grippers, and it is possible to detect whether an object, in particular the primary packaging means, is being gripped, based on the current consumption and / or the position. In other embodiments, a tactile sensor is provided on the grippers and is used to detect whether an object is being gripped. In yet other embodiments, the monitoring is carried out using a camera system.

[0029] Depending on the application, the manipulator has other processing means instead of the gripper, for example, passive processing means such as hooks.

[0030] In certain embodiments, it is provided that within the processing area, particularly at the distal end of the gripper of the manipulator, there is no environmental interference profile provided, or only a minimal environmental interference profile is provided. In certain embodiments, in order to avoid collisions between the distal end of the gripper and the environment, in this case, a virtual distance sensor is provided at the distal end, and the virtual distance sensor is configured to sense the distance of the distal end relative to the environment in the digital images of the moving manipulator and the environment. In certain embodiments, in this case, particularly for visualization on a monitor for controlling the movement and / or for the movement of the manipulator by a manually operable controller, it is provided that the speed of the manipulator decreases as the distance decreases. When the speed decreases, the sensitivity regarding the movement of the manipulator increases. In order to approximate the digital image of the environment to the actual environment, in certain embodiments, the manipulator uses a suitable sensor, particularly a laser sensor, to sense each point of the actual environment and synchronize the digital environment with the actual environment. In order to generate a digital image of the environment, in certain embodiments, it is provided that the manipulator senses the actual environment using a suitable sensor, particularly a laser sensor.

[0031] According to a second aspect, there is provided an apparatus for monitoring filling and / or closing equipment and / or post-treatment equipment, particularly for the pharmaceutical industry. The apparatus comprises a camera system configured to capture images of the transport, feeding-in, and / or discharging areas of the filling and / or closing equipment and / or post-treatment equipment, and a computing unit comprising an artificial intelligence model (AI) model trained to detect primary packaging means within the transport, feeding-in, and / or discharging areas and to classify the detected primary packaging means into classes. The computing unit is configured to determine, based on the above-mentioned images, using the AI model, at which image positions the primary packaging means are present and to assign the detected primary packaging means to classes.

[0032] In one embodiment, the AI ​​model is trained to assign primary packaging means to a class based on their type, and / or to assign a certain type of primary packaging means to a class based on its orientation.

[0033] In one embodiment, the computing unit comprises multiple subunits operating in parallel, which may be centrally or distributedly configured, and which can, in particular, perform different processes in parallel.

[0034] The above-described apparatus may be mounted on a filling and / or closing unit and / or post-processing unit. In one embodiment, the filling and / or closing unit and / or post-processing unit comprises an isolator housing, and in one embodiment, at least the camera of the apparatus is located within the isolator housing. In another embodiment, the camera is mounted outside the isolator housing. In one embodiment, the calculation unit is part of the control means for the filling and / or closing unit and / or post-processing unit. In another embodiment, a separate calculation unit is provided, which, in an advantageous embodiment, communicates with the control means for the filling and / or closing unit and / or post-processing unit for data exchange.

[0035] In one embodiment, the computing unit is configured to evaluate the output of the AI ​​model using a rule-based algorithm for the purpose of determining disorder.

[0036] In one embodiment, when a disturbance is determined, the calculation unit sends a signal to the operator, who can respond accordingly. Alternatively or additionally, the calculation unit sends a signal to the control means of the filling and / or closing equipment and / or post-processing equipment, which can completely or partially stop the filling and / or closing equipment and / or post-processing equipment and / or slow down the processing.

[0037] In an advantageous embodiment, the arithmetic unit is configured to classify and / or prioritize the determined disturbances using a rule-based algorithm, and in particular, the arithmetic unit is configured to specify how the determined disturbances are processed based on the classification of the disturbances, and / or, based on the prioritization, whether the determined disturbances are processed, and if so, when they are processed.

[0038] In an advantageous embodiment, the optical axis of the camera system is inclined with respect to the vertical axis, and the camera system is positioned above the monitored transport, inlet, and / or outlet area, offset from the monitored transport, inlet, and / or outlet area, so as not to disturb the primary supply air to the transport, inlet, and / or outlet area. Thus, the device is also suitable for monitoring the transport, inlet, and / or outlet areas of filling and / or closing equipment and / or post-processing equipment, particularly for filling sensitive pharmaceuticals in isolators.

[0039] In one embodiment, the apparatus comprises a manipulator configured to process the determined disturbances via a central machine controller, a distributed manipulator controller, or a manually operated controller. In one embodiment, the manually operated controller is an input device known as a gamepad, having control pads, and also known for controlling computer games. In another embodiment, the manually operated controller is a mobile communication terminal such as a smartphone or tablet computer, equipped with an application for controlling the manipulator. In yet another embodiment, a conventional control device for a machine controller, having arrow keys, is provided for the movement of the manipulator. In one embodiment, control is performed remotely from the filling and / or closing equipment and / or post-processing equipment, thereby allowing actual images of the filling and / or closing equipment and / or post-processing equipment obtained using a camera system, and / or virtual images of the filling and / or closing equipment and / or post-processing equipment, to be displayed to the operator on a monitor. Alternatively or additionally, in one embodiment, it is provided that the determined disturbances can be processed automatically by the manipulator via a machine controller. In one embodiment, in this regard, it is provided that the determination and correction are performed without operator intervention. In other embodiments, automatic correction is performed after activation by the operator.

[0040] According to a third embodiment, a filling and / or closing and / or post-processing facility is provided, comprising a transport, receiving and / or delivery area and a device for monitoring the filling and / or closing and / or post-processing facility. In one embodiment, the filling and / or closing and / or post-processing facility comprises an isolator housing in which the transport, receiving and / or delivery area is located. In one embodiment, a camera system of at least the monitoring device is also located within the isolator housing. In an advantageous embodiment, the camera system is positioned above the transport, receiving and / or delivery area, offset from the transport, receiving and / or delivery area, so as not to disturb the primary supply air to the transport, receiving and / or delivery area.

[0041] According to a fourth aspect, a computer program is provided which includes the following instructions: When the program is executed by a computer unit, the instructions determine which image locations contain primary packaging means and assign the detected primary packaging means to a class, using an artificial intelligence model (AI model) that has been trained to detect primary packaging means within transport, transport, and / or delivery areas based on images of the transport, transport, and / or delivery areas of the filling and / or closing equipment and / or post-processing equipment.

[0042] In one embodiment, the computer program is installed on control means for the filling and / or closing equipment and / or post-processing equipment. In another embodiment, the computer program is installed on a separate desktop computer, notebook computer, tablet computer, or cloud server, in which case monitoring may be performed in the immediate vicinity of and / or at a distance from the filling and / or closing equipment and / or post-processing equipment.

[0043] In one embodiment, the computer program includes an instruction that, when the program is executed by the arithmetic unit, determines whether or not a disturbance exists, based on the output of the AI ​​model, using a rule-based algorithm.

[0044] Further advantages and aspects of the present invention will become apparent from the claims and from the exemplary embodiments of the invention described below with reference to the drawings. [Brief explanation of the drawing]

[0045] [Figure 1] Figure 1 schematically shows a device for monitoring a filling and / or closing equipment and / or post-processing equipment that has a feeding area. [Figure 2] Figure 2 schematically shows a filling and / or closing facility and / or post-processing facility with multiple disturbances. [Figure 3] Figure 3 schematically shows the nests within the transport, inbound, and / or outbound areas of filling and / or closing equipment and / or post-processing equipment, which have two disturbances. [Figure 4] Figure 4 schematically shows trays in the transport, receiving, and / or delivery areas of a filling and / or closing facility and / or post-processing facility with multiple disturbances. [Figure 5] Figure 5 schematically shows a device for determining and treating turbidity within a filling and / or closing facility and / or post-processing facility that has a discharge area. [Modes for carrying out the invention]

[0046] Figure 1 schematically shows a device 1 for monitoring filling and / or closing equipment and / or post-processing equipment, particularly for the pharmaceutical industry, and the device 1 comprises a transport, receiving and / or delivery area 2. The transport, receiving and / or delivery area 2 shown in Figure 1 has a plurality of linear tracks 20, 21, 22, 23 along which primary packaging means, not shown in Figure 1, in particular stoppers or filled or unfilled containers, are transported. In other embodiments, instead of tracks 20, 21, 22, 23, additional or other elements are provided by which, or along, the primary packaging means are transported, supplied, or accumulated.

[0047] The illustrated monitoring device 1 comprises a camera system 10 and a computing unit 12.

[0048] The camera system 10 is configured to capture a single image or a series of images of the transport, inlet and / or outlet area 2. In the illustrated exemplary embodiment, the optical axis 100 of the camera system 10 is inclined with respect to the vertical axis I, and the camera system 10 is positioned above the transport, inlet and / or outlet area 2, offset from the transport, inlet and / or outlet area 2, so as schematically indicated by the arrows, the primary supply air 5 coming from above toward the transport, inlet and / or outlet area 2 is not disturbed by the camera system 10.

[0049] The computing unit 12 includes an AI model 120 that has been trained to detect primary packaging means within the transport, inbound, and / or outbound area 2, and to classify the detected primary packaging means.

[0050] Images captured by the camera system 10 are analyzed by the processing unit 12, and based on these images, the AI ​​model 120 can be used to determine where the primary packaging means are located in the images and to assign the detected primary packaging means to a class.

[0051] For the classification of primary packaging means, in one embodiment, it is provided that different types of primary packaging means can be detected, where one class is assigned to each type. Alternatively or additionally, in one embodiment, a certain type of primary packaging means is classified based on its orientation. In one embodiment, in this regard, primary packaging means supplied in the correct orientation, primary packaging means lying in the direction of transport, primary packaging means lying sideways to the direction of transport, twisted primary packaging means, and / or upside-down primary packaging means are distinguished for classification.

[0052] The illustrated computation unit 12 further comprises a rule-based algorithm 122. The rule-based algorithm 122 is used to determine whether or not disorder exists based on the output of the AI ​​model 120.

[0053] In one embodiment, the determined disturbances are also classified, i.e., assigned to a class, where the classes are predefined by experts depending on the application. In another embodiment, how the determined disturbances are processed is then determined based on this classification.

[0054] Alternatively or additionally, in one embodiment, a rule-based algorithm 122 is used to prioritize the determined disturbances, i.e., a priority value is assigned to the detected disturbances. Here, the priority values ​​for the detected disturbances are predetermined by an expert, depending on the application. In one embodiment, based on this prioritization, it is determined whether the determined disturbances are processed, and if so, when they are processed. In this case, the purpose of prioritization is to avoid machine downtime.

[0055] In this case, appropriate measures can be taken depending on the specific application, the class of the detected disturbance, and / or the priority value. In the illustrated exemplary embodiment, data is transmitted to the memory unit 13, and information regarding the detected disturbance can be electronically recorded in the memory unit 13.

[0056] In one embodiment, a signal is emitted to an operator, which communicates the disturbance to the operator visually, audibly, and / or by other means, allowing the operator to take appropriate measures to deal with the disturbance. In another embodiment, the disturbance is dealt with manually by the operator. In yet another embodiment, manual dealing is performed within an isolator, via a glove.

[0057] Furthermore, in one embodiment, as schematically shown by the dashed arrows, data regarding the detected disturbances is transmitted to the machine controller 140. In one embodiment, the machine controller 140 functions as an interface for a manipulator, which is not shown in Figure 1. The manipulator can be moved by the machine controller 140, by an additional manipulator controller, or by a manually operable controller, for the purpose of processing the determined disturbances.

[0058] Figure 2 schematically shows a dispensing area 2 having multiple linearly running tracks 20, 21, 22, 23, and 24 for a primary packaging means 4 in the form of a stopper. Each of the tracks 20, 21, 22, 23, and 24 is separated laterally. The dimensions of the tracks 20, 21, 22, 23, and 24 are selected so that each primary packaging means 4 can be moved continuously along the tracks 20, 21, 22, 23, and 24. In this case, multiple disturbances within the dispensing area 2 are schematically shown in Figure 2.

[0059] The illustrated disturbances are the incorrectly oriented primary packaging means 40 and the gap 42, which is caused, for example, by the primary packaging means 43 jamming the movement of track 20. In detail, shown in the uppermost track 20 (viewed in plane in the drawing) are the primary packaging means 40 that are rotated 180 degrees or fed upside down, and the primary packaging means 41 that are tilted in the direction of the path. The gap 42 is shown in the second-to-last track 21. Shown in the middle track 22 are the primary packaging means 41 that are tilted in the direction of the path, and the primary packaging means 44 that are oriented laterally to the direction of the path. The primary packaging means 40 oriented laterally to the direction of the path is similarly shown in the second-to-last track 23. The bottommost track 24 is completely filled correctly.

[0060] The existing primary packaging means 4, 40 are detected by the computing unit 12 (see Figure 1) using the AI ​​model 120 and assigned to a class. The results can be output in a format suitable for further processing.

[0061] In one embodiment, the output of the AI ​​model is subsequently evaluated using a rule-based algorithm 122 to determine the disorder, i.e., the incorrectly oriented primary packaging means 40, 41, 44 and / or the spacing 42.

[0062] In one embodiment, the determined disturbances are further classified and prioritized using a rule-based algorithm 122. In one embodiment, the rule-based algorithm 122 may be used to determine whether the disturbance is a gap 42 caused by jamming or a primary packaging means 40, 41, 44 that is oriented incorrectly. Furthermore, when the primary packaging means are transported to the right in the plane of the drawing, in one exemplary embodiment, the disturbance marked with a circle in Figure 2 is assigned a higher priority value than the other disturbances, because the incorrectly oriented primary packaging means 41 is supplied to the subsequent processing step earlier than the other incorrectly oriented primary packaging means 40, 41, 44, and therefore the correction takes priority. However, other prioritizations are also possible. For example, in one embodiment, gap 42 is assigned a higher priority value than a defectively oriented primary packaging means 40 because primary packaging means 43 is jammed.

[0063] Figure 3 schematically shows a transport, receiving, and / or dispensing area 3 comprising a nest 30, from which primary packaging means 4 are removed from and placed into the nest 30. The primary packaging means 4 are provided or stacked in an orderly manner within a matrix. The primary packaging means 4 shown in Figure 3 are syringes, and in other embodiments, other primary packaging means 4 are provided or stacked in the form of nests. In the situation shown in Figure 3, there are two disturbances, each of which is a primary packaging means 45 stacked on top of it.

[0064] Disturbances caused by the overlapping primary packaging means 45 are detected in each case by the computing unit 12 (see Figure 1) using the AI ​​model 120 and the rule-based algorithm 122, and in some embodiments, are further classified and prioritized using the rule-based algorithm 122. Based on the class assigned to the primary packaging means by the AI ​​model, in some embodiments, the rule-based algorithm 122 can be used to detect, for example, whether disturbances have occurred due to the absence of primary packaging means or disturbances caused by overlapping primary packaging means. In some embodiments, the operating state can be taken into account; that is, the algorithm 122 can distinguish between gaps related to the processing workflow within the nest 30 and gaps related to disturbances.

[0065] Figure 4 schematically shows a transport, receiving, and / or dispensing area 6 with spatially partitioned trays 60. In the illustrated exemplary embodiment, primary packaging means 4 are provided or piled up on the trays 60 in an unkempt manner. The primary packaging means 4 shown in Figure 4 are containers in the form of vials, and in other embodiments, other primary packaging means 4 are provided or piled up on the trays 60. In the situation shown in Figure 4, there are three disturbances, two of which are fallen, i.e., oriented in the wrong direction, and the third is a primary packaging means 45 that is stacked on top of each other.

[0066] Disorders are detected by the computation unit 12 (see Figure 1) using the AI ​​model 120 and the rule-based algorithm 122. In addition, in one embodiment, the disorder is further classified and prioritized using the rule-based algorithm 122.

[0067] The disturbances shown in Figures 2-4 are for illustrative purposes only. As will be apparent to those skilled in the art, depending on the embodiment of the transport, receiving, and / or delivery areas 2, 3, and 6, further disturbances and / or other disturbances may occur, and / or less or more disturbances than those illustrated therein may occur at once.

[0068] Figure 5 schematically shows a device 1 for monitoring filling and / or closing equipment and / or post-processing equipment, the device 1 comprises a transport, receiving and / or delivery area 2, a camera system 10 and a computing unit 12, and a manipulator 14 for the automatic processing of determined disturbances.

[0069] The camera system 10 is configured to capture a single image or a series of images of the transport, inlet and / or outlet area 2. In the illustrated exemplary embodiment, the optical axis 100 of the camera system 10 is inclined with respect to the vertical axis I, and the camera system 10 is positioned above the transport, inlet and / or outlet area 2, offset from the transport, inlet and / or outlet area 2, so as schematically indicated by the arrows, the primary supply air 5 to the transport, inlet and / or outlet area 2 from above is not disturbed by the camera system 10.

[0070] The processing unit 12 is configured to detect disturbances based on images captured by the camera system, using an AI model 120 and a rule-based algorithm 122. In one embodiment, it is further provided that the detected disturbances are classified into classes using the rule-based algorithm 122, that is, that the detected disturbances are assigned to classes and prioritized, that is, that priority values ​​are assigned to the determined disturbances.

[0071] Furthermore, the calculation unit 12 is configured to determine the location of the disturbance, that is, to identify the determined location of the disturbance within the transport, inbound and / or outbound area 2. For this purpose, in the illustrated exemplary embodiment, based on a coordinate transformation 124, the location of the disturbance detected on the image plane of the camera system 10, and the dimensions and orientation of the object to be manipulated to process the disturbance, in particular the primary packaging means, are transformed into the coordinate system of the manipulator 14 and / or operator using a suitable calculation model 126.

[0072] In one embodiment, transformed data of the disturbance's location, as well as data related to the primary packaging means, such as its dimensions and / or orientation, and—if available—the classification and / or prioritization of the detected disturbance, are transmitted to the machine controller 140. In the illustrated exemplary embodiment, the machine controller 140 communicates with a calculation unit 16, which plans the path for the movement of the manipulator 14 that handles the disturbance, based on data determined by the calculation unit 12 and taking into account the interference contour 160. In the illustrated exemplary embodiment, the calculation unit 16 is formed separately from the calculation unit 12 of the monitoring system. In other embodiments, the calculation units 12 and 16 are formed together.

[0073] In one embodiment, the path through which the manipulator 14 moves to process the determined disturbance without collision is planned such that the movement path of the manipulator 14 is minimized in relation to the primary supply air 5 to the primary packaging means and / or components of the filling / closing equipment and / or post-processing equipment that are in contact with the primary packaging means.

[0074] In one embodiment, the above path is planned using an algorithm optimized for compliance with the pharmaceutical industry. In this case, the criteria for compliance with the pharmaceutical industry are selected from a group that includes, in particular, minimizing the time the manipulator is positioned above the primary packaging means; minimizing the time the manipulator is positioned above components of the filling / closing equipment and / or post-processing equipment that are in contact with the primary packaging means; optimizing the motion and / or velocity profile of the flow; minimizing the rotation of the manipulator's axis; minimizing the motion of the manipulator's processing system above the primary packaging means and / or components of the filling and / or post-processing equipment that are in contact with the primary packaging means, in particular minimizing the motion of the gripper above the primary packaging means; and minimizing the impact surface of the primary supply air.

[0075] In this case, individual criteria are weighted according to their intended use, or additional criteria are added.

[0076] In the illustrated exemplary embodiment, a monitor 18 is also present. In the illustrated exemplary embodiment, the provided monitor 18 is either part of the device 1 or communicates with the device 1 for data exchange. In one embodiment, it is provided that a planned path in the simulation environment can be visualized on the monitor 18. In other words, the monitor 18 displays a digital twin of the manipulator 14 and its real environment. In one embodiment, for interactive modification and / or approval, the planned path is visualized here on the monitor 18 before the movement of the manipulator 14 is performed. For modification of the planned path, in one embodiment, it is provided that the operator can modify individual path points, and for this purpose, in one embodiment, a touch-responsive monitor 18 is provided. Alternatively or additionally, in one embodiment, for modification, the operator may be prompted to re-plan the path using the calculation unit 16 without changing the criteria or their weightings. In another embodiment, the operator may optionally re-plan the path by changing the criteria and / or weightings. In yet another embodiment, interaction with the operator is not required for approval of the planned path. In one embodiment, the quality of the planned path is evaluated, and if the evaluation exceeds a defined threshold, no interaction with the operator is required. In another embodiment, if the value falls below the threshold, the path planning is automatically restarted from the beginning, and interaction with the operator is only required if the value falls below the threshold again. Alternatively or additionally, the actual motion of the manipulator is visualized in a simulation environment on a monitor.

[0077] In the illustrated exemplary embodiment, a manually operable control unit 142 is also provided, where the manipulator 14 can be moved by the controller 142 by an operator to handle disturbances.

[0078] In one embodiment, the path along which the manipulator 14 is moved autonomously or using a controller 142 is recorded electronically, particularly in the memory unit 13. The data recorded in relation to the movement path and cause of the manipulator 14, the primary packaging means moved by the manipulator, or similar items may be appropriately specified by those skilled in the art, depending on the application. In one embodiment, the time and / or area covered by the manipulator in the primary supply air 5 is recorded. In one embodiment, the recorded data includes a video file of the movement visualized on the monitor 18, i.e., a digital twin video file. This video file allows the operator to easily evaluate the movement performed.

[0079] The illustrated exemplary embodiment is merely an example, and many variations are possible in which an AI model 120 trained to detect disturbances within transport, ingress, and / or egress areas 2, 3, and 6 based on images of those areas is used to determine whether or not disturbances exist within those areas.

[0080] If the filling and / or closing equipment and / or post-processing equipment has an isolator housing, in one embodiment, the image is captured using a camera system located inside the isolator housing.

Claims

1. A method for monitoring filling and / or closing equipment and / or post-processing equipment, particularly for the pharmaceutical industry, wherein images of the transport, inbound and / or outbound areas (2, 3, 6) of the filling and / or closing equipment and / or post-processing equipment are captured using a camera system (10), and based on the images, an artificial intelligence model (AI model) (120) trained to detect primary packaging means within the transport, inbound and / or outbound areas (2, 3, 6) and to classify the detected primary packaging means is used to determine which image locations contain primary packaging means and to assign the detected primary packaging means to a class.

2. The method according to claim 1, characterized in that the primary packaging means (4) are classified based on their type, and / or a certain type of primary packaging means (4) is classified based on its orientation.

3. The method according to claim 1 or 2, wherein the output of the AI ​​model (120) is evaluated using a rule-based algorithm (122) for the purpose of determining disorder, and preferably the determined disorder is classified and / or prioritized, in particular how the determined disorder is processed is specified based on the classification and / or whether the determined disorder is processed—and if so, when—is processed, based on the prioritization.

4. The method according to claim 3, characterized in that the location of the determined disturbance within the transport, delivery and / or delivery area (2, 3, 6) is identified.

5. The method according to any one of claims 1 to 4, characterized in that the transport, receiving and / or sending area (2) has transport means and / or sorting means, and / or the primary packaging means (4) is provided or stacked in an unordered or orderly manner within a matrix, and the transport, receiving and / or sending area (2) is monitored.

6. The method according to any one of claims 1 to 5, characterized in that the camera system (10) is positioned above the transport, supply and / or discharge area (2, 3, 6) and offset from the transport, supply and / or discharge area (2, 3, 6) so that the primary supply air to the transport, supply and / or discharge area (2, 3, 6) is not disturbed by the camera system (10), and the optical axis (100) of the camera system (10) is inclined with respect to the vertical axis (I).

7. The method according to any one of claims 1 to 6, characterized in that the determined disturbance is processed using a manipulator (14), and the manipulator (14) can be moved by a mechanical controller (140), by a distributed manipulator controller, and / or by a manually operable controller for the purpose of processing the determined disturbance.

8. A device for monitoring filling and / or closing equipment and / or post-processing equipment, particularly for the pharmaceutical industry, the device comprising: a camera system (10) configured to capture images of transport, inbound and / or outbound areas (2, 3, 6) of the filling and / or closing equipment and / or post-processing equipment; and a computing unit (12) equipped with an artificial intelligence model (AI model) (120) trained to detect primary packaging means within the transport, inbound and / or outbound areas (2, 3, 6) and to classify the detected primary packaging means (4, 40, 41, 44, 45), wherein the computing unit (12) is configured to use the AI ​​model (120) to determine, based on the images, which image locations the primary packaging means (4, 40, 41, 44, 45) are located at and to assign the detected primary packaging means (4, 40, 41, 44, 45) to a class.

9. The apparatus according to claim 8, characterized in that the AI ​​model (120) is trained to classify primary packaging means (4, 40, 41, 44, 45) based on their type and / or to classify a certain type of primary packaging means (4) based on its orientation.

10. The apparatus according to claim 8 or 9, wherein the arithmetic unit (12) is configured to evaluate the output of the AI ​​model (120) using a rule-based algorithm (122) for the purpose of determining disorder, and preferably, the determined disorder is configured to classify and / or prioritize the determined disorder using the rule-based algorithm (122), in particular, the arithmetic unit (12) is configured to specify how the determined disorder is processed based on the classification, and / or, based on the prioritization, whether the determined disorder is processed, and if so, when it is processed.

11. The apparatus according to claim 8, 9, or 10, characterized in that the optical axis (100) of the camera system (10) is inclined with respect to the vertical axis (I), and the camera system (10) is positioned above the transport, transport and / or discharge areas (2, 3) and offset from the monitored transport, transport and / or discharge areas (2, 3) so as not to disturb the primary supply air to the transport, transport and / or discharge areas (2, 3).

12. The apparatus according to any one of claims 8 to 11, characterized in that a manipulator (14) is provided, the manipulator (14) is configured to process the determined disturbance by a central machine controller (140), a distributed manipulator controller, and / or a manually operable controller (142).

13. A filling and / or closing and / or post-processing facility comprising a transport, receiving and / or delivery area and the apparatus described in claims 8 to 12, wherein the filling and / or closing and / or post-processing facility comprises, in particular, an isolator housing in which the transport, receiving and / or delivery area (2, 3, 6) is arranged.

14. A computer program comprising instructions, wherein, when the program is executed by an arithmetic unit (12), the instructions use an artificial intelligence model (AI model) (120) trained to detect primary packaging means within the transport, transport and / or delivery areas (2, 3, 6) of the filling and / or closing equipment and / or post-processing equipment, based on images of the transport, transport and / or delivery areas (2, 3, 6), and to classify the detected primary packaging means, to determine which image locations the primary packaging means (4, 40, 41, 44, 45) are located at, and to assign the detected primary packaging means (4, 40, 41, 44, 45) to a class.

15. The computer program according to claim 14, comprising an instruction which, when the program is executed by the arithmetic unit (12), determines whether or not a disturbance exists using a rule-based algorithm (122) based on the output of the AI ​​model.

Images