Automated visual inspection of pre-filled syringes

Abstract

A system for measuring the air gap in a syringe containing a fluid, having a plunger according to the present disclosure, may include one or more processors configured to control a syringe imaging device to capture a first image of at least a portion of the syringe, with the syringe imaging device oriented at a first rotation angle around the central syringe axis of the syringe with respect to a first central imaging axis of the syringe imaging device. The one or more processors may also be configured to control the syringe imaging device to capture a second image of at least a portion of the syringe, with the syringe imaging device oriented at a second rotation angle around the central syringe axis with respect to a second central imaging axis. The one or more processors may be further configured to determine the measured air gap between the plunger and the fluid by analyzing at least the first and second images.

Description

Technical Field

[0001] Cross - reference to Related Applications Priority is claimed to U.S. Provisional Patent Application No. 63 / 447,313, filed Feb. 28, 2023, the entire content of which is hereby incorporated by reference.

[0002] This application generally relates to the inspection of containers, and more specifically to an imaging system for the automated visual inspection of pre - filled syringes.

Background Art

[0003] In certain situations, such as in quality control procedures for manufactured pharmaceuticals, it is necessary to inspect containers (e.g., pre - filled syringes, vials, cartridges, and / or their contents, etc.) for the presence of various defects (e.g., air - gap measurements, plunger depth measurements, cracks, defective seals, under - filling, over - filling, foreign objects, fibers, etc.). The acceptability of a given container or sample under applicable quality standards can depend on metrics such as the condition of the container, the presence of undesirable particles within the container, etc.

[0004] The depth of the plunger and the air - gap are two important quality parameters that can affect the functionality of the assembled product, the shelf - life, and the sterility of the product within a syringe filled with a fluid (e.g., pharmaceuticals, etc.). If a pre - filled syringe has unacceptable metrics, the syringe and its contents can be rejected and discarded.

[0005] Known measurement techniques can be classified into three general categories: manual techniques, electronic sensor-based techniques, and machine vision-based techniques. Manual techniques include handheld or manually operated devices such as calipers, depth gauges, and optical comparators. Plunger depth and air gap have traditionally been measured with calipers. Calipers are inexpensive, easy to use, fast, and versatile, but measurements using calipers place a heavy burden on the operator to ensure accuracy and consistency. Depth gauges have been considered as an option for measuring plunger depth, but they have similar drawbacks to calipers, plus the added risk of the plunger moving during measurement. In contrast, optical comparators are accurate, reproducible across users, and are widely considered the preferred standard for measuring plunger depth. Disadvantages of optical comparators include their slow speed, impending obsolescence, and potentially destructive nature due to the light intensity required to produce a silhouette image of the syringe.

[0006] Electronic sensor technology includes electronic components capable of measuring distance (e.g., confocal microscopes, optical comparators, ultrasonic sensors, etc.). While distance sensors may be acceptable for measuring plunger depth, they cannot measure air gap. Distance sensors utilize infrared or visible light beams reflected from a surface. The distance sensor detects the reflected light and, depending on the underlying measurement principle, converts the time, wavelength, or angle of reflection into a distance measurement. The plunger depth may then be calculated by subtracting the distance between the plunger and the sensor from the distance between the syringe flange and the sensor.

[0007] Machine vision technology involves acquiring images of syringes and digitally determining the air gap or plunger depth via computer algorithms. Product inspection tasks (e.g., plunger depth measurement, air gap measurement, etc.) are becoming increasingly automated, typically to handle the quantities of pre-filled syringes associated with the commercial production of pharmaceuticals. Known automated visual inspection (AVI) systems (e.g., system 100 in Figure 1) have struggled to overcome various obstacles to achieving good product fidelity without complicating the system. System 100 includes a camera 126 with a central imaging axis 128 aligned with the central syringe axis 103 of a pre-filled syringe 105, and an illumination source 131. The pre-filled syringe 105 may include a plunger 110 and a fluid 120. The camera 126 may capture a single image 101 of a portion of the pre-filled syringe 105 from a fixed predetermined rotation angle around the central syringe axis 103, with the central imaging axis 128 aligned with the central syringe axis 103, and measure the plunger depth 185 and / or the air gap 180 between the plunger 110 and the fluid 120 based on the single image 101. Measuring the air gap 180 presents a problem with the system 100 in the fluid 120, which often accumulates at the junction between the plunger 110 and the inside of the side wall 114 of the syringe 105, making it difficult to pinpoint the location of the measurement point 114. Droplets and / or bubbles at the junction between the plunger 110 and the syringe side wall 114 often block the measurement point of the air gap 180. Therefore, one of the main difficulties with measuring the air gap 180 is properly positioning the junction between the stopper and the syringe side wall 114. Furthermore, the meniscus on the upper surface of the drug solution can be asymmetrical, making accurate localization difficult when based on a single image 101. [Overview of the Initiative] [Problems that the invention aims to solve]

[0008] Therefore, an AVI system is needed to more accurately measure the air gap within pre-filled syringes. [Means for solving the problem]

[0009] The embodiments described herein relate to a fixed-position imaging system and an automated visual inspection (AVI) system incorporating fixed-position imaging.

[0010] As described herein, a computer-aided method for measuring the air gap in a syringe having a plunger and containing a fluid includes capturing a first image of at least a portion of the syringe from a first rotation angle around the central syringe axis of the syringe with respect to a first central imaging axis of the syringe imaging device. The method also includes capturing a second image of at least a portion of the syringe from a second rotation angle around the central syringe axis with respect to a second central imaging axis of the syringe imaging device. The method further includes determining the measured air gap between the plunger and the fluid by analyzing at least the first and second images using one or more processors.

[0011] A system for measuring the air gap in a syringe containing a fluid and having a plunger includes a syringe imaging device and one or more processors. One or more processors are configured to control the syringe imaging device to capture a first image of at least a portion of the syringe from a first rotation angle around the central syringe axis of the syringe with respect to a first central imaging axis of the syringe imaging device. One or more processors are also configured to control the syringe imaging device to capture a second image of at least a portion of the syringe from a second rotation angle around the central syringe axis with respect to a second central imaging axis of the syringe imaging device. One or more processors are further configured to determine the measured air gap between the plunger and the fluid by analyzing at least the first and second images.

[0012] A non-temporary computer-readable medium for storing computer-readable instructions, wherein, when executed by one or more processors, the computer-readable instructions control one or more processors to cause a syringe imaging device to capture a first image of at least a portion of the syringe from a first rotation angle around the central syringe axis of the syringe with respect to a first central imaging axis of the syringe imaging device. Further execution of the computer-readable instructions by one or more processors also causes one or more processors to control the syringe imaging device to capture a second image of at least a portion of the syringe from a second rotation angle around the central syringe axis with respect to a second central imaging axis of the syringe imaging device. Further execution of the computer-readable instructions by one or more processors further causes one or more processors to determine a measured air gap between the plunger and the fluid by analyzing at least the first and second images.

[0013] A novel AVI system is provided that allows for more accurate measurement of the air gap within pre-filled syringes.

[0014] Those skilled in the art will understand that the figures described herein are included for illustrative purposes only and do not limit the disclosure. The drawings are not necessarily to scale and instead focus on illustrating the principles of the disclosure. In some cases, various aspects of the embodiments described may be exaggerated or enlarged to facilitate understanding of the embodiments described. In the drawings, similar reference numerals throughout the various drawings refer to components that are generally functionally and / or structurally similar. [Brief explanation of the drawing]

[0015] [Figure 1] A side profile diagram of the prior art imaging system is shown. [Figure 2] An exemplary pre-filled syringe with a plunger, fluid, and air gap is shown. [Figure 3A]An exemplary automated visual inspection (AVI) system including an exemplary benchtop automated syringe inspection (DASI) system is shown. [Figure 3B] An internal view of the exemplary benchtop automated syringe inspection (DASI) system of FIG. 3A is shown. [Figure 3C] An enlarged view of an exemplary syringe rotation mechanism with a glass tube syringe jig of the exemplary benchtop automated syringe inspection (DASI) system of FIG. 3A is shown. [Figure 4A] An exemplary calibration standard is shown. [Figure 4B] An exemplary calibration standard of FIG. 4A inserted into the exemplary benchtop automated syringe inspection (DASI) system of FIGS. 3A - 3C is shown. [Figure 5] A high - level block diagram of the exemplary benchtop automated syringe inspection (DASI) system of FIGS. 3A - 3C is shown. [Figure 6] An exemplary method of calibrating a benchtop automated syringe inspection (DASI) system is shown. [Figure 7] An exemplary method of operating a benchtop automated syringe inspection (DASI) system is shown. [Figure 8] An exemplary method of generating air - gap measurement data is shown. [Figure 9] An exemplary method of generating aggregated air - gap measurement data is shown. [Figure 10] An exemplary method of generating plunger depth measurement data is shown. [Figure 11] An exemplary method of generating aggregated plunger depth measurement data is shown. [Figure 12] An exemplary automated visual inspection (AVI) system configured to implement the technology of the present disclosure is shown.

BRIEF DESCRIPTION OF THE DRAWINGS

[0016] One of ordinary skill in the art will understand that the elements in the figures are depicted for simplicity and clarity and are not necessarily drawn to scale. For example, the dimensions and / or relative positions of some of the elements in the figures may be exaggerated relative to other elements to aid in understanding the various embodiments of the present invention. Also, commonly understood elements that are useful or necessary in commercially viable embodiments are often not shown so as not to overly obscure the figures of these various embodiments. Further, it will be recognized that certain acts and / or steps may be described or shown in a particular order of occurrence, but one of ordinary skill in the art will understand that such specificity with respect to order is not actually necessary. Further, it will be recognized that certain acts and / or steps may be described or shown in a particular order of occurrence, but one of ordinary skill in the art will understand that such specificity with respect to order is not actually necessary. It will also be understood that the terms and expressions used herein, unless otherwise described with a different specific meaning, have the ordinary technical meaning as would be given by one of ordinary skill in the art to such terms and expressions, as described above.

[0017] The various concepts introduced above and discussed in more detail below can be implemented in any of many ways, and the concepts described are not limited to any particular mode of embodiment. Examples of embodiments are provided for illustrative purposes.

[0018] The systems and methods described herein measure an air gap (and optionally plunger depth and / or other characteristics) within a prefilled syringe having a plunger and containing a fluid by capturing a first image of at least a portion of the syringe with the syringe imaging device oriented at a first rotation angle about a central syringe axis and capturing a second image of at least a portion of the syringe with the syringe imaging device oriented at a second rotation angle about the central syringe axis. The systems and methods measure at least the air gap by analyzing at least the first and second images.

[0019] Figure 2 shows a typical pre-filled syringe 205 filled with fluid 220 and with a plunger 210 inserted. The plunger depth 285 is defined as the difference between the top point 207 of the syringe flange 206 and the top point 212 of the plunger 210 that does not include the small lugs or dimples 211. The top surface of the flange 206 is rarely flat, and therefore any part of the flange 206 that is determined to be furthest from the plunger 210 can be determined to be the top point 207. The air gap 280 is the measurement between the bottom 222 of the meniscus curve 221 defining the top surface of the fluid 220 and the point 214 where the bottom 213 of the plunger 210 contacts the inner surface of the syringe 205. The measurement of the air gap 280 is performed with the central syringe axis 203 of the syringe 205 oriented vertically and the syringe needle 209 pointed downward relative to the syringe flange 206, thereby allowing air to rise upward through the fluid 220 and form the air gap 280.

[0020] Figures 3A to 3C show automated visual inspection (AVI) systems 300a, b, including a desktop automated syringe inspection (DASI) system 325 configured to provide air gap measurements (e.g., measurement 280) and optionally plunger depth measurements (e.g., measurement 285) of a pre-filled syringe 205. The DASI system 325 includes at least one external port 324 configured to output aggregated air gap measurements based on the analysis of at least two different images 301a, b. The at least one external port 324 may be further configured to output aggregated plunger depth measurements based on the analysis of at least two different images 301a, b.

[0021] External port 324 outputs aggregated air gap measurement values ​​180 in a standard format (e.g., electronic batch record (EBR) system format, certified GMP Part 11 data management and storage system format, etc.). External port 324 may also output aggregated plunger depth measurement values ​​185 in a standard format. The DASI system 325 may further include a user interface 330. The user interface 330 may include a touchscreen human-machine interface (HMI) 332. A method 900 for generating aggregated air gap measurement values ​​will be described with reference to Figure 9. A method 1100 for generating aggregated plunger depth measurement values ​​will be described with reference to Figure 11. A method 700 for calibrating the DASI system will be described with reference to Figure 7.

[0022] The DASI system 325 also includes a camera 326 with a telecentric lens 327 (e.g., a 5-megapixel camera), a syringe fixture 335, a programmable logic controller (PLC) 345, and an image processing module 349. The syringe fixture 335 is designed to prevent the syringe needle 209 from bending when the pre-filled syringe 205 is inserted into the syringe fixture 335. The syringe fixture 335 may allow an operator to insert the pre-filled syringe 205 into the syringe fixture 335 with one hand.

[0023] The syringe fixture 335 may include a glass tube having an inner diameter slightly larger than the outer diameter of a given syringe (such as syringe 205). The syringe can be inserted into the glass tube so that it is held in a stable vertical position and no part of the syringe is obstructed. Thus, the air gap and plunger depth can be measured at any fill level. Additionally or alternatively, the air gap and plunger depth can be measured in syringes with a fill level of less than 0.33 ml.

[0024] The DASI system 325 further includes a motor 336 and a torque-limiting clutch 337. The motor 336 and torque-limiting clutch 337 may be configured to rotate the syringe axially. The camera 326 may be configured to capture a series of images 301a,b while the syringe rotates completely around the central syringe axis 303. The camera 326 may be configured to capture a series of images of at least 25 per revolution. The DASI system 325 may generate aggregated air gap measurements based on the analysis of at least 25 images. The analysis of at least 25 images is more likely to be accurate than the analysis of fewer images of the syringe-sidewall junction 314a,b, and even more accurate than the analysis of a single image 101. In particular, the syringe-sidewall junction 314b is at least partially obscured in image 301b. Therefore, an air gap measurement based on image 301b alone should be more likely to be incorrect than an air gap measurement based on image 301a.

[0025] As the syringe rotates axially, the plunger depth and air gap can be measured, for example, by using any suitable image processing technique (e.g., edge detection in the image, pixel position in the image, etc.). The plunger depth and air gap measurements may be transmitted to the PLC345, where each measurement for a particular syringe may be stored. If all images 301a,b are taken during the full rotation, the PLC345 may perform nonlinear filtering by removing a few extreme measurements. For example, the three largest and three smallest air gap measurements may be removed from the set of measurement data (e.g., the set of measurement data generated from the analysis of at least 25 images 301a,b). This would ensure that if a droplet covers the syringe-sidewall junction 314b, the relevant air gap measurements are not excessively large or small compared to the actual values. However, while the three largest and three smallest air gap measurements may be removed, measurements of other quantities may yield better or worse results depending on the fluid 220. After nonlinear filtering of the measurement data, the PLC345 may average the remaining measurements to generate aggregated air gap measurements.

[0026] The aggregated plunger depth measurement may also be generated using multiple measurements while the syringe is rotating, which can improve accuracy. The plunger depth measurement begins at the highest point of the syringe flange. This point can be difficult to find from only one side of the syringe. Slightly more accurate plunger depth measurements are possible by detecting the highest point in several images as the syringe rotates. Similarly, the plane defined by the upper edge of the plunger, or the base or lug of the dimple 211 on the plunger 210, is often difficult to detect accurately due to the different presentation of the dimple 211 on the plunger. By averaging several upper plunger positions, the DASI system 325 improves the accuracy of the plunger depth measurement compared to analyzing only one image.

[0027] As a quality control check for the DASI system 325, a set of images (e.g., at least 25 images) may be compared to ensure that the syringe has actually rotated. If an error occurs in the motor communication or power transmission hardware, the sets of images 301a,b may have a similar appearance to one another, since there should be no syringe movement between the images. This allows the DASI system 325 to determine, for example, that the torque limiting clutch 337 between the motor 336 and the syringe fixture 335 may be disengaged. On the other hand, if the motor has rotated the syringe properly, there should be differences in each image, particularly around the flange region.

[0028] In some embodiments, this type of check involves using a separate processor to monitor the motor encoder signal and thereby verify the movement. However, this introduces additional cost and complexity to the system. It also fails to verify that the clutch and shaft coupling between the motor and syringe are functioning correctly. Therefore, in other embodiments, already acquired images are used to provide a more robust check of proper syringe rotation.

[0029] The DASI system 325 may also include an illumination source 331. The illumination source 331 may be configured as near-infrared (NIR) light emitting a backlight. Since this is generally a non-destructive test, it is preferable to keep light exposure to a minimum so as not to impair the quality of the formulation. Exposure to light is reduced by strobe light only while the image is being acquired. This results in 20-30 or more short flashes of light. If this light flashes rapidly and continuously, it may be bothersome to the operator and may cause some seizures. Therefore, NIR light may be chosen because it is not visible to humans and causes less damage to the product than visible light.

[0030] Preferably, the hardware configuration of the DASI system 325 is compact for easy transport and self-contained so that all image acquisition and processing are performed within it. The DASI system 325 can minimize the spatial distortion and parallax encountered with standard lenses by using a high-quality optical system in the form of a telecentric lens. Furthermore, the DASI system 325 may be configured to allow syringes to be loaded into the fixture quickly and easily, so that measurements can be performed in a repeatable, easy, ergonomic, and rapid syringe insertion and removal (to minimize the time required for measuring several syringes).

[0031] Figures 4A and 4B show DASI calibration devices 400a and 400b, including a calibration standard 460 and a fixture 470. The calibration standard 460 includes the uppermost point of the syringe flange 462 and the uppermost point 412 on the plunger 410 defining a predetermined plunger depth measurement 485. The calibration standard 460 also includes the bottom 422 of a meniscus curve defining the upper surface of the fluid 420 and the point 421 where the bottom of the plunger 410 contacts the inner surface of the syringe and defines a predetermined air gap measurement 480. The calibration standard 460 can be fabricated by laser cutting a thin metal to form a two-dimensional representation of the prefilled syringe 205.

[0032] The fixture 470 is fixed in place via a base 471. The base 471 is configured to prevent the fixture from rotating. The fixture 470 also includes a calibration standard container 472. The fixture 470 is configured to prevent the calibration standard 460 from rotating when a portion of the calibration standard 460 is received into the calibration standard container 472. The fixture 470 may also include a syringe container 440 (e.g., a glass tube).

[0033] DASI300a~c can be calibrated based on an image of a calibration standard 460 inserted into a syringe container 440 and held in a fixed position via a jig 470. Further details of the method for calibrating the DASI will be explained with reference to Figure 6.

[0034] Figure 5 is a high-level block diagram of a desktop automated syringe inspection (DASI) system 500 that can implement various techniques related to training (and optionally validating and / or certifying) and / or using one or more neural networks or other non-machine learning (ML) systems to measure plunger depth and / or air gap. The DASI system 500 can also be used to test / certify non-ML AVI systems. In addition to, or as an alternative to, ML systems, the DASI system 500 may include a “computer vision” algorithm that does not use ML and instead uses predetermined rules (e.g., empty vial, underfilled, overfilled, etc.). The measurement system 525 outputs aggregated air gap measurement data to the electronic batch recording (EBR) system via external port 324 without any software modifications to the EBR system. The measurement system 525 may also output aggregated plunger depth into the electronic batch recording (EBR) system via external port 324 without any software modifications to the EBR system.

[0035] The DASI system 500 may include, for example, one or more automated visual inspection (AVI) neural networks. Once trained and qualified, the DASI system 500 may be used in production to detect defects related to containers and / or their contents. In the context of pharmaceuticals, for example, the DASI system 500 may be used to detect defects related to syringes, cartridges, vials, or other container types (e.g., damaged crimp / seal, cracks, scratches, stains, missing components, etc.), and / or defects related to the liquid or lyophilized pharmaceuticals inside the container (e.g., the presence or absence of fibers, metal particles, and / or other foreign particles, changes in product color, etc.). As used herein, “defect detection” may, depending on the embodiment, mean the classification of container images as indicating or not indicating a defect (or a particular defect category), and / or the detection of a particular object or feature (e.g., particles or cracks) relating to whether the container and / or its contents should be considered defective.

[0036] The DASI system 500 includes a visual inspection system (VIS) 545 communicatively coupled to the measurement system 525. The VIS 545 includes hardware (e.g., a light source 531, a syringe fixture 535, a syringe rotation mechanism motor 536, etc.) configured to acquire a digital image of a sample (e.g., a pre-filled syringe holding a fluid or freeze-dried substance), as well as firmware and / or software. The VIS 545 may include, for example, any of the imaging systems described herein with reference to Figures 2 to 4, or any other suitable VIS.

[0037] For the sake of clarity, this specification describes the DASI system 500 as training and validating one or more AVI neural networks using container images 301a and 301b from VIS545, and then performing AVI / defect detection using the trained / validated neural networks. However, it should be understood that this is not necessarily the case. For example, the DASI system 500 may perform training and / or validation using container images generated by several different visual inspection systems instead of, or in addition to, VIS545. Furthermore, training / validation may be performed by another system, and the DASI system 500 may then use the trained neural networks (e.g., during commercial production). In some embodiments, some or all of the container images used for training and / or validation are generated using one or more offline (e.g., laboratory-based) "mock-up stations" that closely replicate key aspects (e.g., optics, lighting, etc.) of a commercial line equipment station, thereby extending the training and / or validation library without causing excessive downtime for commercial line equipment.

[0038] The VIS545 may sequentially image each of several containers. To achieve this objective, the VIS545 may be equipped with, or operate in conjunction with, a holding means such as a turntable, a Cartesian coordinate robot, a carousel, a star wheel, and / or any other holding means, which can sequentially move each container to a suitable position for imaging, and then move the container elsewhere once imaging of the container is complete. Although not shown in Figure 5, the VIS545 may be equipped with a communication interface and a processor that enable communication with the measurement system 525. In other embodiments (e.g., laboratory-based setups), the VIS545 may be equipped with simpler holding means (e.g., a stage with holes covered by a glass plate).

[0039] The measurement system 525 may generally be configured to control / automate the operation of the VIS 545 and to receive and process images captured / generated by the VIS 545, as will be discussed further below. The measurement system 525 may be a general-purpose computer or a dedicated computing device specifically programmed to perform the operations discussed herein. As can be seen from Figure 5, the measurement system 525 comprises a user interface 532, a processing unit 546, and a memory unit 547. However, in some embodiments, the measurement system 525 includes two or more computers that are located in the same place or far apart from each other. In these distributed embodiments, the operations described herein relating to the processing unit 546 and the memory unit 547 may be divided among multiple processing units and / or memory units, respectively.

[0040] The processing unit 546 may include one or more processors, each of which may be a programmable microprocessor that executes software instructions stored in the memory unit 547 to perform some or all of the functions of the measurement system 525 described herein. The processing unit 546 may include, for example, one or more graphics processing units (GPUs) and / or one or more central processing units (CPUs). Alternatively or additionally, some of the processors in the processing unit 546 may be other types of processors (e.g., application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), etc.), and some of the functions of the computer system 604 described herein may instead be implemented in hardware.

[0041] The memory unit 547 may include one or more volatile and / or non-volatile memories. The memory unit 547 may include one or more suitable types of memory, such as read-only memory (ROM), random access memory (RAM), flash memory, solid-state drive (SSD), and hard disk drive (HDD). Collectively, the memory unit 547 may store one or more software applications, data received / used by those applications, and data output / generated by those applications.

[0042] The memory unit 547, when executed by the processing unit 546, stores software instructions for various modules that perform various functions for the purpose of training, verifying, and / or certifying one or more AVI neural networks. Specifically, in the exemplary embodiment of Figure 5, the memory unit 547 includes a measurement data generation module 549 and a visual inspection system (VIS) control module 526. In other embodiments, the memory unit 547 may omit one or more of modules 548, 549, and / or include one or more additional modules. Additionally or alternatively, one, some, or all of modules 548, 549 may be implemented by a different computer system (e.g., a remote server connected to the measurement system 525 via one or more wired and / or wireless communication networks). Furthermore, the functionality of either one of modules 548 and 549 may be divided between different software applications and / or computer systems. As just one example, in an embodiment in which the measurement system 525 accesses a web service to train and use one or more AVI neural networks, the software instructions for the measurement data generation module 549 may be stored on a remote server.

[0043] The measurement data generation module 549 includes software for training one or more AVI neural networks using images stored in the image library 530. The image library 530 may be stored in the memory unit 547 or in another local or remote memory (e.g., memory connected to a remote library server). In addition to training, module 549 may implement / run the trained AVI neural network by applying newly acquired images, for example, by VIS 545 (or another visual inspection system), to the neural network after certain preprocessing of the images may be performed as described below. In various embodiments, the AVI neural network trained and / or run by module 549 may classify the entire image (e.g., whether it is defective or not, or whether it has a specific type of defect such as a crimp blemish or a crimp defect in general), detect objects in the image (e.g., detect the location of a foreign object other than a bubble in a container image), or perform some combination of these (e.g., one neural network classifies the image and another performs object detection). As used herein, unless the context explicitly indicates a more specific use, “object detection” broadly refers to techniques for identifying specific locations of objects (e.g., particles, fibers, etc.) in an image, and / or specific locations of features of larger objects (e.g., damaged crimps or seals, cracks, or chips on a syringe or cartridge barrel, etc.), and may include, for example, techniques for segmenting container images or portions of images (e.g., pixel-by-pixel classification), or techniques for identifying objects and placing bounding boxes (or other boundary shapes) around those objects.

[0044] In embodiments where an AVI neural network detects defects in a container, the defects may be related to any suitable container feature. For example, referring to the exemplary containers in Figures 2 to 4B, a specific AVI neural network implemented by the measurement data generation module 549 may detect whether the container has cracks or dirt, whether the flange is deformed, whether the needle shield is not properly positioned, whether the plunger or piston has any defects, whether the Luer lock 528 has any defects, whether the crimp is properly positioned and / or has any defects (e.g., scratches), whether the flip cap is properly positioned and / or has defects, etc.

[0045] Module 549 may run a trained AVI neural network for the purpose of verification, certification, and / or inspection during commercial production. In one embodiment, for example, Module 549 is used solely for training and verifying an AVI neural network, which is then transferred to another computer system (for example, using another module similar to Module 549) for certification and inspection during commercial production. In some embodiments in which the measurement data generation module 549 trains / runs multiple neural networks, Module 549 includes separate software for each neural network.

[0046] Training of the AVI neural network can be performed using images from, for example, six vials, after augmenting the relevant training images by adjusting brightness, vertical mirroring, adding noise, and skewing the images, as well as skewing the bounding box (i.e., the training set can be increased fivefold). Deep learning can generally be used to detect defects in images. By using a previously trained AVI neural network, the time required to set up automated inspection recipes for new products can be further reduced. The AVI neural network of this disclosure can be used for high-mix low-volume production scenarios such as clinical operations or small-batch products, and can be implemented using state-of-the-art deep learning techniques (e.g., the measurement data generation module 549 in Figure 5).

[0047] In some embodiments, the VIS control module 548 controls / automates the operation of the VIS 545 so that container images can be generated with little or no human intervention. The VIS control module 525 can cause an imaging system at a given fixed position to acquire a container image by transmitting commands or other electronic signals to the imaging system (e.g., by generating pulses on a control line). The VIS 545 can transmit the acquired container image to a measurement system 525 that can store the image in a memory unit 547 for local processing. In alternative embodiments, the VIS 545 may be locally controlled, in which case the VIS control module 525 may have fewer functions than those described herein (e.g., only handling the reading of images from the VIS 545), or may be omitted entirely from the memory unit 547.

[0048] Figure 6 shows a method 600 for calibrating a desktop automated syringe inspection (DASI) system, which may be implemented by a processor (e.g., processing unit 546 in Figure 5) that performs at least part of the VIS control module 548 and / or measurement data generation module 549. The DASI system may be similar to, for example, the DASI system 325 in Figures 3A-3B or 500 in Figure 5. In particular, the calibration standard 400a is inserted into the syringe container 440 of the syringe rotation mechanism 338 with the central syringe axis 303 aligned with the central imaging axis 328 (block 651).

[0049] The processing unit 546 executes the VIS control module 548 to cause the processing unit 546 to receive calibration image data from, for example, the imaging device 426 (block 652). The processing unit 546 executes the measurement data generation module 549 to cause the processing unit 546 to calibrate, for example, the DASI system 325 (block 653). For example, the processing unit 546 may receive a predetermined air gap measurement value 480 and a plunger depth measurement value 485. The processing unit 546 may calibrate the DASI system 325 by comparing an air gap measurement value determined based on the analysis of the calibration image data with a predetermined air gap measurement value 480. Additionally or alternatively, the processing unit 546 may calibrate the DASI system 325 by comparing a plunger depth measurement value determined based on the analysis of the calibration image data with a predetermined plunger depth measurement value 485.

[0050] Figure 7 shows a method 700 for operating the DASI system 325, which may be implemented by the processing unit 546 executing, for example, at least part of the VIS control module 548 and / or the measurement data generation module 549. The DASI system may be similar to, for example, the DASI system 325 in Figures 3A-3B or the 500 in Figure 5. In particular, the pre-filled syringe 205 is inserted into the syringe container 340 (e.g., a glass tube) of the syringe rotation mechanism 338 with the central syringe axis 303 aligned with the central imaging axis 328 (block 751).

[0051] The processing unit 546 executes the VIS control module 548 to enable the processing unit 546 to, for example, power the illumination light source 310 (block 752). The processing unit 546 further executes the VIS control module 548 to enable the processing unit 546 to receive, for example, the first syringe image data 301a from the imaging device 326 (block 753).

[0052] The processing unit 546 may further execute the VIS control module 548 to control the syringe rotation mechanism 338 so that, for example, the syringe 205 rotates around the central syringe axis 303 (block 754). The processing unit 546 may further execute the VIS control module 548 to cause the processing unit 546 to receive, for example, a second syringe image data 301b from the imaging device 326 (block 755). The processing unit 546 may further execute the VIS control module 548 to cause the processing unit 546 to transmit, for example, a first syringe image data 301a and a second syringe image data 301b to the measurement library 530 (block 756).

[0053] The processing unit 546 may execute the measurement data generation module 549 to cause the processing unit 546 to generate measurement data (for example, the highest point 207 of the syringe flange 206, the highest point 212 on the plunger 210 that does not include small lugs, the bottom 222 of the meniscus curve 221 that defines the upper surface of the fluid 220, the point 214 where the bottom 213 of the plunger 210 contacts the inner surface of the syringe 205, air gap measurement data, plunger depth measurement data, etc.) based on the first syringe image data 301a and the second syringe image data 301b (block 757). The plunger depth measurement value 285 may be the difference between the highest point 207 of the syringe flange 206 and the highest point 212 on the plunger 210 that does not include small lugs or dimples 211. The air gap measurement 280 may be the difference between the bottom 222 of the meniscus curve 221 defining the top surface of the fluid 220 and the point 214 where the bottom 213 of the plunger 210 contacts the inner surface of the syringe 205.

[0054] The processing unit 546 can further execute the VIS control module 548 to cause the processing unit 546 to, for example, transmit measurement data from the VIS system to the measurement system 525 and / or the measurement library 530 (block 758).

[0055] Figure 8 shows a method 800 for operating a DASI system, which may be implemented by the processing unit 546 executing, for example, at least part of the VIS control module 548 and / or the measurement data generation module 549. The DASI system may be similar to, for example, the DASI system 325 in Figures 3A-3B or 500 in Figure 5. In particular, the processing unit 546 can execute the VIS control module 548 to cause the processing unit 546 to receive, for example, measurement data (block 851). The measurement data may include the bottom 222 of the meniscus curve 221 defining the top surface of the fluid 220 and the point 214 where the bottom 213 of the plunger 210 contacts the inner surface of the syringe 205.

[0056] The processing unit 546 executes the measurement data generation module 549 to cause the processing unit 546 to generate air gap measurement data based on the measurement data, for example (block 852). The first air gap measurement value 280 may be the difference between the bottom 222 of the meniscus curve 221 defining the upper surface of the fluid 220, based on the analysis of the first image 301a, and the point 214 where the bottom 213 of the plunger 210 contacts the inner surface of the syringe 205. The second air gap measurement value 280 may be the difference between the bottom 222 of the meniscus curve 221 defining the upper surface of the fluid 220, based on the analysis of the second image 301b, and the point 214 where the bottom 213 of the plunger 210 touches the inner surface of the syringe 205.

[0057] The processing unit 546 can further execute the VIS control module 548 to cause the processing unit 546 to transmit, for example, air gap measurement data from the VIS 545 to the measurement system 525 or the measurement library 530 (block 853).

[0058] Figure 9 shows a method 900 for operating a DASI system, which may be implemented by a processing unit 546 executing, for example, at least part of a visual inspection system (VIS) control module 548 and / or a measurement data generation module 549. The DASI system may be similar to, for example, the DASI system 325 in Figures 3A-3B or the 500 in Figure 5. In particular, the processing unit 546 can execute the VIS control module 548 to cause the processing unit 546 to receive, for example, air gap measurement data (block 951). The air gap measurement data may include a first air gap measurement based on the analysis of a first image 301a and a second air gap measurement based on the analysis of a second image 301b.

[0059] The processing unit 546 executes the measurement data generation module 549 to cause the processing unit 546 to generate aggregated air gap measurement data based on, for example, air gap measurement data (block 952). The aggregated air gap measurement data can be based on a first air gap measurement and a second air gap measurement. For example, the aggregated air gap measurement data may be the average of the first air gap measurement and the second air gap measurement.

[0060] The processing unit 546 can further execute the VIS control module 548 to cause the processing unit 546 to send, for example, aggregated air gap measurement data to the measurement library 530 (block 953).

[0061] Figure 10 shows a method 1000 for operating a DASI system, which may be implemented by a processing unit 546 executing, for example, at least part of a Visual Inspection System (VIS) control module 548 and / or a Measurement Data Generation Module 549. The DASI system may be similar to, for example, the DASI system 325 in Figures 3A-3B or 500 in Figure 5. In particular, the processing unit 546 can execute the VIS control module 548 to cause the processing unit 546 to receive, for example, measurement data (block 1051). The measurement data may include the uppermost point 207 of the syringe flange 206 and the uppermost point 212 on the plunger 210, which does not include small lugs.

[0062] The processing unit 546 may execute the measurement data generation module 549 to cause the processing unit 546 to generate plunger depth measurement data based on the measurement data, for example (block 1052). The first plunger depth measurement 285 may be the difference between the uppermost point 207 of the syringe flange 206 and the uppermost point 212 of the plunger 210 that does not include the small lug or dimple 211, based on the analysis of the first image 301a. The second plunger depth measurement 285 may be the difference between the uppermost point 207 of the syringe flange 206 and the uppermost point 212 of the plunger 210 that does not include the small lug or dimple 211, based on the analysis of the second image 301b.

[0063] The processing unit 546 can further execute the VIS control module 548 to cause the processing unit 546 to transmit, for example, plunger depth measurement data from the VIS 545 to the measurement system 525 or the measurement library 530 (block 1053).

[0064] Figure 11 shows a method 1100 for operating a DASI system, in which the processing unit 546 may be implemented by, for example, executing at least a portion of the VIS control module 548 and / or the measurement data generation module 549. The DASI system may be similar to, for example, the DASI system 325 in Figures 3A-3B or 500 in Figure 5. In particular, the processing unit 546 may execute the VIS control module 548 to cause the processing unit 546 to receive, for example, plunger depth measurement data (block 1151). The plunger depth measurement data may include a first plunger depth measurement based on the analysis of a first image 301a and a second plunger depth measurement based on the analysis of a second image 301b.

[0065] The processing unit 546 may execute the measurement data generation module 549 to cause the processing unit 546 to generate aggregated plunger depth measurement data based on, for example, plunger depth measurement data (block 1152). The aggregated plunger depth measurement data can be obtained based on a first plunger depth measurement and a second plunger depth measurement. For example, the aggregated plunger depth measurement data may be the average of the first plunger depth measurement and the second plunger depth measurement.

[0066] The processing unit 546 can further execute the VIS control module 548 to cause the processing unit 546 to transmit, for example, aggregate plunger depth measurement data from the VIS 545 to the measurement system 525 or the measurement library 530 (block 1153).

[0067] Figure 12 shows an AVI system 1200 including a syringe imaging device 1226 having a first camera 1226a and a second camera 1226b. The first camera 1226a includes a first central imaging axis 1228a aligned with the central syringe axis 1203 of a prefilled syringe 1205 and an illumination source 1231a. The first camera 1226a captures a first image 1201a of a portion of the prefilled syringe 1205 from a fixed predetermined rotation angle around the central syringe axis 1203, with the central imaging axis 1228a aligned with the central syringe axis 1203. The first image 1201a may include droplets and / or bubbles at the junction of the plunger 1210a and the syringe sidewall 1214a, partially blocking the measurement point of the air gap 1280a. Therefore, the system 1200 does not need to measure the plunger depth 1285a and / or air gap 1280a between the plunger 1210a and the fluid 1220a based on the first image 1201a.

[0068] The second camera 1226b includes a central imaging axis 1228b and an illumination source 1231b aligned with the central syringe axis 1203 of the prefilled syringe 1205. The second camera 1226b captures a second image 1201b of a portion of the prefilled syringe 1205 from a fixed predetermined rotation angle around the central syringe axis 1203, with the central imaging axis 1228b aligned with the central syringe axis 1203. The second image 1201b does not necessarily contain droplets and / or bubbles at the junction between the plunger 1210b and the syringe sidewall 1214b. Based on the second image 1201b, the system 1200 may measure the plunger depth 1285b and / or air gap 1280b between the plunger 1210b and the fluid 1220b.

[0069] Systems, methods, apparatus, and their components have been described in terms of exemplary embodiments, but they are not limited to these exemplary embodiments. Detailed descriptions are to be interpreted as examples only, and since it would be impractical, if not impossible, to describe all possible embodiments of the present invention, not all possible embodiments of the present invention are described. Many alternative embodiments can be carried out using either the current art or art developed after the filing date of this patent, and these should still be included within the claims defining the present invention.

[0070] Those skilled in the art will understand that various modifications, variations, and combinations of the above embodiments can be made without departing from the scope of the present invention, and that such modifications, variations, and combinations should be interpreted as being within the scope of the concept of the present invention.

Claims

1. A computer-aided method for measuring the air gap in a syringe containing a fluid, having a plunger, Using a syringe imaging device, a first image of at least a portion of the syringe is captured from a first rotation angle around the central syringe axis of the syringe with respect to a first central imaging axis of the syringe imaging device, Using the syringe imaging device, a second image of at least a portion of the syringe is captured from a second rotation angle around the central syringe axis with respect to the second central imaging axis of the syringe imaging device, The measurement of the air gap between the plunger and the fluid is determined by analyzing at least the first image and the second image using one or more processors. Methods that include...

2. The method according to claim 1, wherein the first image and the second image are captured by a first camera and a second camera of the syringe imaging device, respectively, the first camera and the second camera have fixed orientations, the first camera has a first central imaging axis, and the second camera has a second central imaging axis.

3. The method according to claim 1, wherein the first image and the second image are captured by moving a single camera of the syringe imaging device from a first rotation angle to a second rotation angle with respect to the central syringe axis, the single camera having a first central imaging axis and the second central imaging axis being the first central imaging axis.

4. The method according to claim 1, wherein the first image and the second image are captured by a single camera of the syringe imaging device and by rotating the syringe, the single camera having a first central imaging axis and the second central imaging axis being the first central imaging axis.

5. The method according to any one of claims 1 to 4, wherein the air gap measurement includes a measurement associated with the bottom of a meniscus curve associated with the upper surface of the fluid.

6. The method according to any one of claims 1 to 5, wherein the air gap measurement includes a measurement related to the intersection of the outer circumference of the bottom of the plunger and the inner surface of the syringe barrel.

7. The method according to any one of claims 1 to 6, wherein the air gap measurement includes a measurement of the difference between the bottom of a meniscus curve related to the upper surface of the fluid and the intersection of the outer circumference of the bottom of the plunger and the inner surface of the syringe barrel.

8. The method according to any one of claims 1 to 7, further comprising determining a plunger depth measurement by further analyzing at least the first image and the second image using one or more of the processors.

9. The method according to claim 8, wherein the plunger depth measurement includes a measurement of the difference between the top of the syringe flange and the top of the plunger body.

10. To provide a calibration standard having a predetermined air gap and a predetermined plunger depth, Inserting at least a portion of the calibration standard into the syringe, Using the syringe imaging device, a third image is captured of at least a portion of the syringe and at least a portion of the calibration standard. Calibrating the measuring device based on at least the third image using one or more of the aforementioned processors, The method according to any one of claims 1 to 9, further comprising:

11. A system for measuring the air gap in a syringe containing a fluid, having a plunger, Syringe imaging device, One or more processors, The syringe imaging device is controlled to capture a first image of at least a portion of the syringe from a first rotation angle around the central syringe axis of the syringe with respect to a first central imaging axis of the syringe imaging device, The syringe imaging device is controlled to capture a second image of at least a portion of the syringe from a second rotation angle around the central syringe axis with respect to the second central imaging axis of the syringe imaging device. By analyzing at least the first and second images, the measured value of the air gap between the plunger and the fluid is determined. One or more processors configured as such A system equipped with these features.

12. The system according to claim 11, wherein the first image and the second image are captured by a first camera and a second camera of the syringe imaging device, respectively, the first camera and the second camera have fixed orientations, the first camera has a first central imaging axis, and the second camera has a second central imaging axis.

13. The system according to claim 11, wherein the first image and the second image are captured by moving a single camera of the syringe imaging device from a first rotation angle to a second rotation angle with respect to the central syringe axis, the single camera having a first central imaging axis and the second central imaging axis being the first central imaging axis.

14. The system according to claim 11, wherein the first image and the second image are captured by moving a single camera of the syringe imaging device from a first rotation angle to a second rotation angle with respect to the central syringe axis, the single camera having a first central imaging axis and the second central imaging axis being the first central imaging axis.

15. The system according to any one of claims 11 to 14, further comprising a syringe rotation mechanism including a transparent tube syringe container having an inner diameter larger than the outer diameter of the syringe.

16. The system according to any one of claims 11 to 15, wherein the syringe rotation mechanism is configured to hold the syringe with the central syringe axis in a vertical position.

17. The system according to any one of claims 11 to 16, further comprising a plurality of calibration standards, each calibration standard representing at least one of a predetermined air gap or a predetermined plunger depth.

18. The system according to any one of claims 11 to 17, further comprising a data conversion device configured to convert the air gap measurement values ​​into an electronic batch record data format.

19. A non-temporary computer-readable medium for storing computer-readable instructions, wherein when an instruction is executed by one or more processors, the one or more processors... The syringe imaging device is controlled to capture a first image of at least a portion of a syringe containing a fluid, with respect to a first central imaging axis of the syringe imaging device, from a first rotation angle around the central syringe axis of the syringe. The syringe imaging device is controlled to capture a second image of at least a portion of the syringe from a second rotation angle around the central syringe axis with respect to the second central imaging axis of the syringe imaging device. By analyzing at least the first and second images, the measured value of the air gap between the plunger and the fluid is determined. Non-temporary computer-readable media.

20. The non-temporary computer-readable medium according to claim 19, wherein the air gap measurement data represents at least one of six different images, twelve different images, or twenty-four different images, each of which includes a different portion of the outer surface of the syringe, and the air gap measurement data includes different air gap measurement data based on each of the different images.

21. Further execution of the computer-readable instruction by the one or more processors is performed by the one or more processors: The air gap measurement data is filtered by removing one or more highest air gap measurements and one or more lowest air gap measurements. A non-temporary computer-readable medium according to claim 19 or 20.

22. Further execution of the computer-readable instruction by the one or more processors is performed by the one or more processors: To generate an average air gap measurement based on two or more air gap measurements, A non-temporary computer-readable medium according to claim 20 or 21.

23. Further execution of the computer-readable instruction by the one or more processors is performed by the one or more processors: Based on the syringe image data, plunger depth measurement data is generated. A non-temporary computer-readable medium according to any one of claims 19 to 22.

24. Further execution of the computer-readable instruction by the one or more processors is performed by the one or more processors: Calibration data representing at least one of a predetermined air gap measurement or a predetermined plunger depth measurement is received. Calibrate the measuring device based on the calibration data. A non-temporary computer-readable medium according to any one of claims 19 to 23.

25. The non-temporary computer-readable medium according to any one of claims 19 to 24, wherein the first image and the second image are captured by a first camera and a second camera of the syringe imaging device, respectively, the first camera and the second camera have fixed orientations, the first camera has a first central imaging axis, and the second camera has a second central imaging axis.

26. The first image and the second image are captured by moving a single camera of the syringe imaging device from a first rotation angle to a second rotation angle with respect to the central syringe axis, wherein the single camera has a first central imaging axis and the second central imaging axis is the first central imaging axis, according to any one of claims 19 to 24.

27. The first image and the second image are captured by moving a single camera of the syringe imaging device from a first rotation angle to a second rotation angle with respect to the central syringe axis, wherein the single camera has a first central imaging axis and the second central imaging axis is the first central imaging axis, according to any one of claims 19 to 24.

Images