Fixed-position imaging system for automated visual inspection

Abstract

The automated visual inspection (AVI) system of this disclosure may position a container holder such that its central container axis is optically aligned with the central image axis of a fixed-position imaging system. Once the container holder is positioned such that its central container axis is aligned with the central image axis of the fixed-position imaging system, the fixed-position imaging system may simultaneously acquire a first image of a first container and a second image of a second container. The fixed-position imaging system may acquire a first image of the first container from the same viewpoint as the fixed-position imaging system simultaneously acquires a second image of the second container. The AVI system may analyze the first and second images to inspect a first subset of the containers and a second subset of the containers for the same set of one or more features.

Description

Technical Field

[0001] Cross - reference to Related Applications This application claims priority to U.S. Provisional Patent Application No. 63 / 448,552, filed on February 27, 2023, and incorporates by reference in its entirety the content thereof into this specification.

[0002] This application generally relates to the inspection of containers, and more particularly to an imaging system for automated visual inspection.

Background Art

[0003] In certain situations, such as in the quality control procedures of manufactured pharmaceuticals, it is necessary to inspect whether various defects (e.g., cracks, poor seals, air gap measurement, plunger depth measurement, under - filling, over - filling, foreign objects, fibers, etc.) exist in containers (e.g., containers, vials, syringes, cartridges, etc., and / or their contents). The acceptability of a given container or sample under applicable quality criteria may depend on indicators such as the condition of the container, the presence of undesirable particles inside the container, etc. If the container has unacceptable indicators, the container and its contents may be rejected and discarded.

[0004]

[0005] ​In addition to the mechanical movement required to align the central container fixing axis 108 with the imaging system central axis 108, the AVI system 100 requires the camera having the mechanical movement mechanism 105 to sequentially align its central image axis 108b with the central container axis 109b so that the composite camera having the mechanical movement mechanism 105 can sequentially capture images 102b of each container 102a. Therefore, the camera having the mechanical movement mechanism 105, along with the composite optical system 106 and connecting cables 107, is subjected to associated physical stresses. The AVI system 100 often requires extensive maintenance due to the millions of associated mechanical movement cycles of the camera having the mechanical movement mechanism 105.

[0006] An imaging system is needed that reduces the mechanical movement of the associated camera, optics, and associated camera connections. [Overview of the project] [Means for solving the problem]

[0007] The embodiments described herein relate to a fixed-position imaging system and an automated visual inspection (AVI) system that incorporate fixed-position imaging.

[0008] As described herein, a system for capturing images of multiple containers includes a first fixed-position imaging system configured to capture a first image, a second fixed-position imaging system configured to capture a second image, and a transport mechanism configured to transport the containers through the first and second fixed-position imaging systems. The first fixed-position imaging system, the second fixed-position imaging system, and the transport mechanism are configured such that the first fixed-position imaging system captures a first image of a first subset of containers, and the second fixed-position imaging system simultaneously captures a second image of a second subset of containers from the same viewpoint relative to the containers.

[0009] A method for imaging multiple containers includes transporting the containers in front of a first fixed-position imaging system and a second fixed-position imaging system using a transport mechanism. The method also includes acquiring a first image of a first subset of the containers using the first fixed-position imaging system. The method further includes acquiring a second image of a second subset of the containers using the second fixed-position imaging system, wherein the acquisition of the first image and the second image are performed simultaneously and from the same viewpoint relative to the containers.

[0010] A non-temporary computer-readable medium has a stored computer-readable instruction, which, when executed by one or more processors, causes one or more processors to control a transport mechanism to transport a container in front of a first fixed-position imaging system and a second fixed-position imaging system. Further execution of the computer-readable instruction by one or more processors causes one or more processors to simultaneously (i) control the first fixed-position imaging system to acquire a first image of a first subset of the container, and (ii) control the second fixed-position imaging system to acquire a second image of a second subset of the container from the same viewpoint relative to the container.

[0011] A novel fixed-position imaging system and automated visual inspection (AVI) system incorporating a fixed-position imaging system are provided. A novel method for operating the AVI system is also provided.

[0012] Those skilled in the art will understand that the figures described herein are included for illustrative purposes only and do not limit the disclosure. The drawings are not necessarily to scale and instead focus on illustrating the principles of the disclosure. In some cases, various aspects of the embodiments described may be exaggerated or enlarged to facilitate understanding of the embodiments described. In the drawings, similar reference numerals throughout the various drawings refer to components that are generally functionally and / or structurally similar. [Brief explanation of the drawing]

[0013] [Figure 1] A known automated visual inspection (AVI) system having a mechanically positioned imaging system is shown. [Figure 2] This shows a plan view of an automated visual inspection (AVI) system in an embodiment that has a fixed-position imaging system. [Figure 3A] This shows a plan view of an automated visual inspection (AVI) system in an embodiment that has a fixed-position imaging system. [Figure 3B] Figure 3A shows a top perspective view of the automated visual inspection (AVI) system of the embodiment. [Figure 3C] Figure 3A shows an upper rear perspective view of the automated visual inspection (AVI) system of the embodiment. [Figure 4] This shows a plan view of an automated visual inspection (AVI) system in an embodiment that has a fixed-position imaging system. [Figure 5] A high-level block diagram of an automated visual inspection (AVI) system in an embodiment having a fixed-position imaging system is shown. [Figure 6] This document describes an embodiment of an automated visual inspection (AVI) system that includes a fixed-position imaging system. [Modes for carrying out the invention]

[0014] Those skilled in the art will understand that the elements in the figures are drawn for simplification and clarity and are not necessarily drawn to a specific scale. For example, the dimensions and / or relative positions of some elements in the figures may be exaggerated relative to others to help improve the understanding of the various embodiments of the invention. Also, common but well-understood elements that are useful or necessary in commercially viable embodiments are often omitted so as not to interfere too much with the illustrations of these various embodiments. Furthermore, it will be recognized that certain actions and / or processes may be described or shown in a specific order of occurrence, but those skilled in the art will understand that such specificity regarding order is not actually necessary. Furthermore, it will be recognized that certain actions and / or processes may be described or shown in a specific order of occurrence, but those skilled in the art will understand that such specificity regarding order is not actually necessary. Unless a different specific meaning is explained herein, the terms and expressions used herein also have the ordinary technical meanings that those skilled in the art would give to such terms and expressions, as described above.

[0015] The various concepts introduced above and examined in more detail below may be implemented using any of many methods, and the concepts described are not limited to any particular implementation method. Examples of implementations are provided for illustrative purposes.

[0016] The fixed-position imaging system of this disclosure may reduce or completely eliminate the complex mechanical movement of associated cameras, optics, camera connections, etc. Rather than sequentially acquiring images 102b of each container 102a, as in the known AVI system 100, the system described herein may, for example, simultaneously control a first fixed-position imaging system to acquire first images of a first subset of containers, and control a second fixed-position imaging system to acquire second images of a second subset of containers from the same viewpoint relative to the containers. The fixed-position imaging system of this disclosure may enable container inspection in milliseconds, compared to the known system 100, which performs the same operation at a much slower speed, involving complex mechanical movement. The fixed-position imaging system may eliminate the high maintenance requirements and downtime caused by various failure modes resulting from millions of complex mechanical imaging device movement cycles in the known system 100.

[0017] Where used in connection with this disclosure, “simultaneously” means “approximately simultaneously.” For example, a fixed-position imaging system may acquire the first image and the second image within a predetermined time (e.g., 50 milliseconds, 100 milliseconds, etc.). In any case, the fixed-position imaging system acquires the first image and the second image without physically moving the fixed-position imaging system or the container holder between the acquisition of the first image and the second image.

[0018] Figure 2 is a plan view of an automated visual inspection (AVI) system 200 having a fixed-position imaging system 205. The AVI system 200 includes a transport mechanism 203 configured to position a container holder 204 such that a central container fixing axis 209 aligns with the central image axis 208 of the fixed-position imaging system 205. Once the container holder 204 is positioned so that the central container fixing axis 209 aligns with the central image axis 208, the fixed-position imaging system 205 may simultaneously capture a first image 202b1 of a first subset 202a1 of containers and a second image 202b2 of a second set 202a2 of containers. The fixed-position imaging system 205 may also capture the first image 202b1 with respect to the first central image axis 208a in the same way that the fixed-position imaging system 205 simultaneously captures the second image 202b2 with respect to the second central image axis 208b. In any case, the fixed-position imaging system 205 simultaneously acquires the first image 202b1 and the second image 202b2 without any mechanical movement in between.

[0019] As shown in Figure 2, the first subset 202a1 of the containers and the second subset 202a2 of the containers may each include two containers 202. Although Figure 2 shows both the first subset 202a1 and the second subset 202a1 of the containers to include two containers 202, either the first subset 202a1 or the second subset 202a1 of the containers may include one or more containers 202. This allows the AVI system 200 to simultaneously acquire images 202b1 and 202b2 in less time than the AVI system 100 would need to mechanically align the imaging system 105 and acquire a single image 102b from a single container 102a.

[0020] The AVI system 200 may further include a lighting source 210. The lighting source 210 may be configured as a fixed-position backlight. In any case, the AVI system 200 may energize the lighting source 210 before capturing the first and second images 202b1, 202b2. The first and second images 202b1, 202b2 each include only a part of the respective container 202 (i.e., a part of the syringe flange, a part of the syringe barrel, the plunger, the air gap, and a part of the product inside the syringe), while the first and second images 202b1, 202b2 may include the entire side view of the respective containers 202a1, 202a2.

[0021] The AVI system 200 may be used, for example, to inspect the containers 202 in the final packaging area of the associated manufacturing facility. The AVI system 200 may be used to inspect the containers 202 in an inspection area inside the manufacturing facility in addition to or instead of the final packaging area. The AVI system 200 may be configured to inspect a predetermined number of containers 202 in less than 20 milliseconds, for example, compared to the AVI system 100 which takes 3 - 4 seconds (i.e., the AVI system 200 may be 100 times faster than the AVI system 100). In any case, the AVI system 200 may eliminate the millions of mechanical cycles of the AVI system 100 that require mechanical movement of the high-sensitivity optical system and millions of bends of the camera cable. Although the AVI system 200 is shown in FIG. 2 with respect to prefilled syringes, the AVI system 200 may be applied to inspections in the assembly, labeling, and packaging stages of the associated manufacturing process. Similarly, the AVI system 200 may evaluate attributes regarding combined products (e.g., hand-held autoinjectors, autoinjectors, etc.) or associated packages (e.g., the presence of labels, the position of labels, etc.). [[ID=,6]]

[0022] The AVI systems 300a - c of FIGS. 3A - 3C may be similar to the AVI system 200. The AVI systems 300a - c may include a transport mechanism 303 configured to position the container fixture 304 in a state where the central container fixing axis 309 is aligned with the central image axis 308 of the fixed - position imaging system 305.

[0023] The fixed - position imaging system 305 includes a first fixed - position imaging system 305a having a first telecentric lens 306a and a first camera connection 307a. The first fixed - position imaging system 305a may be oriented at a fixed position such that, for example, using a first mirror 340a, the first central optical axis 308a is aligned with the first central container fixing axis 309a of the first subset 302a1 of the containers. The first mirror 340a is shown in FIGS. 3A - 3B as having a planar reflecting surface oriented at an angle of 45 degrees with respect to the first central optical axis 308a, but the first mirror 340a may be oriented at an angle with respect to the first central optical axis 308a such that the first subset 302a1 of the containers is included within the field of view of the first fixed - position imaging system 305a. The first subset 302a1 of the containers may include, for example, a single container (e.g., container 102a of FIG. 1), a part of two containers (e.g., a part of containers 202a1, 202a2 of FIG. 2), or a predetermined number of containers, based on, for example, a desired container inspection speed and / or a predetermined image resolution. Instead of providing a plan view of the first subset 302a1 of the containers, the first mirror 340a may be rotated and oriented with respect to the first central optical axis 308a such that a perspective view of the first subset 302a1 of the containers is included within the field of view of the first fixed - position imaging system 305a (e.g., oriented to view the syringe flange from a predetermined viewpoint, oriented to view the vial seal from a predetermined viewpoint, etc.). The first telecentric lens 306a may include any number and type of optical elements that may be configured, for example, to align the central imaging device axis of the image sensor of the first fixed - position imaging system 305a with the first central optical axis 308a. Thus, the central imaging device axis of the image sensor may be oriented at an arbitrary fixed position with respect to the first central optical axis 308a.

[0024] The fixed-position imaging system 305 includes a second fixed-position imaging system 305b having a second telecentric lens 306b and a second camera connection 307b. The second fixed-position imaging system 305b may be oriented to a fixed position, for example using a second mirror 340b, such that the second central optical axis 308b aligns with the second subset 302a2 of the containers. The second subset 302a2 of the containers may include, for example, a single container (e.g., container 102a in Figure 1), two containers (e.g., container 202a1 or container 202a2 in Figure 2), or a predetermined number of containers, based on a desired container inspection speed and / or a predetermined image resolution. The second subset 302a2 of the containers may include more or fewer containers than the first subset 302a1 of the containers. Instead of providing a plan view of the second subset 302a2 of the container, the second mirror 340b may be rotated and oriented with respect to the second central optical axis 308b such that a perspective view of the second subset 302a2 of the container is included within the field of view of the second fixed-position imaging system 305b (for example, oriented to view the syringe flange from a given viewpoint, or oriented to view the vial seal from a given viewpoint, etc.). The second telecentric lens 306b may include any number and type of optical elements, for example, configured to align the central imaging device axis of the image sensor of the second fixed-position imaging system 305b with the second central optical axis 308b. Thus, the central imaging device axis of the image sensor may be oriented to any fixed position with respect to the second central optical axis 308b.

[0025] In any case, once the AVI systems 300a to c align the central container fixing axis 309 with the central image axis 308, the fixed-position imaging system 305 may simultaneously control the first fixed-position imaging system 305a to acquire the first image 302b1 of the first subset 302a1 of the container, and control the second fixed-position imaging system 305b to acquire the second image 302b2 of the second subset 302a2 of the container. The first fixed-position imaging system 305a may also acquire the first image 302b1 of the first subset 302a1 of the container relative to the first central image axis 308a from the same viewpoint as the second fixed-position imaging system 305b simultaneously acquires the second image 302b2 of the second set 302a2 of the container relative to the second central image axis 308b. The fixed-position imaging system 305 simultaneously acquires the first image 302b1 and the second image 320b2 without any mechanical movement in between, such as that required to sequentially acquire image 102b using the prior art AVI system 100.

[0026] The fixed-position imaging system 305a may further include an illumination source 310 (e.g., a fixed-position backlight, a backlight mounted on a rotating platform, a backlight incorporated into a container holder, etc.). The fixed-position imaging system 305 may also include a fixed-position illumination source 310 mounted in a fixed position via, for example, a bracket 311.

[0027] While the AVI system 300a is shown in Figure 3A with respect to a prefilled vial, the AVI system 200 may be applied to inspections during the assembly, labeling, and packaging stages of the relevant manufacturing process. Similarly, the AVI system 300a may evaluate attributes related to a combined product (e.g., a handheld auto-injector, an auto-injector, etc.) or the associated packaging (e.g., the presence of a label, the location of the label, etc.).

[0028] The AVI system 400 may be the same as the AVI systems 300a to c in Figures 3A to 3C or the AVI system 200 in Figure 2. The AVI system 400 may include a transport mechanism configured to position the container fixing device 404 with the central container fixing axis 409 aligned with the central image axis 408 of the fixed position imaging system 405.

[0029] The fixed-position imaging system 405 includes a first fixed-position imaging system 405a having a first telecentric lens 406a and a first camera connection 407a. The first fixed-position imaging system 405a may be oriented to a fixed position, for example, such that a first central optical axis 408a is aligned with the first central container axis 409a of the first subset 402a1 of containers, so that a first subset 402a1 of containers is included within the field of view of the first fixed-position imaging system 405a. The first subset 402a1 of containers may include, for example, a single container (e.g., container 102a in Figure 1), a portion of two containers (e.g., a portion of containers 202a1 and 202a2 in Figure 2), a side view of an entire container (e.g., side views of containers 302a1 and 302a2 in Figure 3A), or a predetermined number of containers, based on a desired container inspection speed and / or a predetermined image resolution. Instead of providing a plan view of the first subset 402a1 of the container, the first fixed-position imaging system 405a may be rotated and oriented with respect to the first central optical axis 408a such that a perspective view of the first subset 402a1 of the container is included within the field of view of the first fixed-position imaging system 405a (for example, oriented to view the vial seal from a given viewpoint, or oriented to view the syringe flange from a given viewpoint, etc.). The first telecentric lens 406a may include any number and type of optical elements, for example, configured to align the central imaging device axis of the image sensor of the first fixed-position imaging system 405a with the first central optical axis 408a. Thus, the central imaging device axis of the image sensor may be oriented to any fixed position with respect to the first central optical axis 408a.

[0030] The fixed-position imaging system 405 includes a second fixed-position imaging system 405b having a second telecentric lens 406b and a second camera connection 407b. The second fixed-position imaging system 405b may be oriented to a fixed position, for example, such that a second central optical axis 408b aligns with a second subset 402a2 of the containers. The second subset 402a2 of the containers may include, for example, a single container (e.g., container 102a in Figure 1), two containers (e.g., containers 202a1 or 202a2 in Figure 2), a side view of the entire container (e.g., side views of containers 302a1 and 302a2 in Figure 3A), or a predetermined number of containers, based on a desired container inspection speed and / or a predetermined image resolution. The second subset 402a2 of the containers may include more or fewer containers than the first subset 402a1 of the containers. Instead of providing a plan view of the second subset 402a2 of the container, the second fixed-position imaging system 405b may be rotated and oriented with respect to the second central optical axis 408b such that a perspective view of the second subset 402a2 of the container is included within the field of view of the second fixed-position imaging system 405b (e.g., oriented to view the syringe flange from a given viewpoint, or oriented to view the vial seal from a given viewpoint). The second telecentric lens 406b may include any number and type of optical elements, for example, configured to align the central imaging device axis of the image sensor of the second fixed-position imaging system 405b with the first central optical axis 408b. Thus, the central imaging device axis of the image sensor may be oriented to any fixed position with respect to the first central optical axis 408b.

[0031] In any case, once the AVI system 400 aligns the central container fixing axis 409 with the central image axis 408, the fixed-position imaging system 405 may simultaneously control the first fixed-position imaging system 405a to acquire the first image 402b1 of the first subset 402a1 of the container, and control the second fixed-position imaging system 405b to acquire the second image 402b2 of the second subset 402a2 of the container. The first fixed-position imaging system 405a may also acquire the first image 402b1 of the first subset 402a1 of the container relative to the first central image axis 408a from the same viewpoint as the second fixed-position imaging system 405b simultaneously acquires the second image 402b2 of the second set 402a2 of the container relative to the second central image axis 408b. The fixed-position imaging system 405 simultaneously acquires the first image 402b1 and the second image 402b2 without any mechanical movement in between, such as that required to sequentially acquire image 102b using the prior art AVI system 100.

[0032] The fixed-position imaging system 405 may further include an illumination source 410 (e.g., a fixed-position backlight, a backlight mounted on a rotating platform, a backlight incorporated into a container holder, etc.). The fixed-position imaging system 405 may also include a fixed-position illumination source 410 that is mounted in a fixed position via, for example, a bracket 411.

[0033] While the AVI system 400 is shown in relation to a prefilled vial in Figure 4, the AVI system 400 may also be applied to inspections during the assembly, labeling, and packaging stages of the relevant manufacturing process. Similarly, the AVI system 400 may evaluate attributes related to a combined product (e.g., a handheld auto-injector, an auto-injector, etc.) or the associated packaging (e.g., the presence of a label, the location of the label, etc.).

[0034] Figure 5 is a simplified block diagram of an embodiment of the AVI system 500, which may implement various techniques for training (and optionally validating and / or certifying) and / or using one or more neural networks or non-machine learning (ML) systems. The AVI system 500 can also be used to test / certify non-ML AVI systems. In addition to, or as an alternative to, ML systems, the AVI system 500 may include a “computer vision” algorithm that does not use ML and instead uses predetermined rules (e.g., empty vial, underfilled, overfilled, etc.).

[0035] The AVI system 500 may include one or more AVI neural networks. Once trained and qualified, the AVI system 500 may be used in production to detect defects related to containers and / or their contents. In relation to pharmaceuticals, for example, the AVI system 500 may be used to detect defects related to syringes, cartridges, vials, or other container types (e.g., damaged crimp / seal, cracks, scratches, stains, missing components, etc.) and / or defects related to liquid or lyophilized pharmaceuticals within the container (e.g., the presence or absence of fibers, metal particles, and / or other foreign particles, changes in product color, etc.). As used herein, “defect detection” may, depending on the embodiment, mean the classification of container images as showing or not showing defects (or a particular defect category), and / or the detection of specific objects or features (e.g., particles or cracks) related to whether the container and / or its contents are considered defective.

[0036] The AVI system 500 includes a visual inspection system (VIS) 505 which is communicatively coupled to a computer system 520. The VIS 505 includes hardware (e.g., an illumination source 510, a telecentric optical system 506, etc.) configured to acquire a digital image of a sample (e.g., a container holding a fluid or freeze-dried substance), as well as firmware and / or software. The VIS 505 may include, for example, any of the fixed-position imaging systems 205, 305a-c, 405 described herein with reference to Figures 2-4, or any other suitable VIS.

[0037] For the sake of clarity, the AVI system 500 is described herein as training and validating one or more AVI neural networks using container images from VIS 505, and then performing AVI / defect detection using the trained / validated neural networks. However, it should be understood that this is not necessarily the case. For example, the AVI system 500 may perform training and / or validation using container images generated by several different visual inspection systems instead of, or in addition to, VIS 505. Furthermore, training / validation may be performed by another system, and the AVI system 500 may then use the trained neural networks (e.g., during commercial production). In some embodiments, some or all of the container images used for training and / or validation are generated using one or more offline (e.g., laboratory-based) "mock stations" that closely replicate key aspects (e.g., optics, lighting, etc.) of a commercial line equipment station, thereby extending the training and / or validation library without causing excessive downtime for commercial line equipment.

[0038] VIS505 may image each of several containers simultaneously. To achieve this objective, VIS505 may include, or operate in conjunction with, a transport mechanism, a turntable, a Rectangular coordinate robot, a carousel, a star wheel, and / or any other arbitrary holding means, which can sequentially move each container to a suitable position for imaging, and then move the container elsewhere once imaging of the container is complete. Although not shown in Figure 5, VIS505 may also include a communication interface and a processor that enable communication with a computer system 520. In other embodiments (e.g., a laboratory-based setup), VIS505 may include simpler holding means (e.g., a stage with holes covered by a glass plate).

[0039] The computer system 520 may generally be configured to control / automate the operation of the VIS 505 and to receive and process images captured / generated by the VIS 505, as will be further discussed below. The computer system 520 may be a general-purpose computer or a dedicated computing device specifically programmed to perform the operations discussed herein. As can be seen in Figure 5, the computer system 520 includes a user interface 521, a processing unit 522, and a memory unit 523. However, in some embodiments, the computer system 520 includes two or more computers that are located in the same place or are located separately from each other. In these distributed embodiments, the operations described herein relating to the processing unit 522 and the memory unit 523 may be divided among multiple processing units and / or memory units, respectively.

[0040] The processing unit 522 comprises one or more processors, each of which may be a programmable microprocessor that executes software instructions stored in the memory unit 523 to perform some or all of the functions of the computer system 520 as described herein. The processing unit 522 may include, for example, one or more graphics processing units (GPUs) and / or one or more central processing units (CPUs). Alternatively, or in addition, some of the processors in the processing unit 522 may be other types of processors (e.g., application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), etc.), and some of the functions of the computer system 520 as described herein may instead be implemented in hardware.

[0041] The memory unit 523 may include one or more volatile and / or non-volatile memories. The memory unit 523 may include one or more suitable types of memory, such as read-only memory (ROM), random access memory (RAM), flash memory, solid-state drives (SSDs), hard disk drives (HDDs), etc. Collectively, the memory unit 523 may store one or more software applications, data received / used by those applications, and data output / generated by those applications.

[0042] The memory unit 523, when executed by the processing unit 522, stores software instructions for various modules that perform various functions for the purpose of training, verifying, and / or certifying one or more AVI neural networks. Specifically, in the exemplary embodiment shown in Figure 5, the memory unit 523 includes an image analysis module 525 and a visual inspection system (VIS) control module 526. In other embodiments, the memory unit 523 may omit one or more of modules 525, 526, and / or include one or more additional modules. In addition, or alternatively, one, some, or all of modules 525, 526 may be implemented by a different computer system (e.g., a remote server connected to computer system 520 via one or more wired and / or wireless communication networks). Furthermore, the functionality of either one of modules 525 and 526 may be divided between different software applications and / or computer systems. As just one example, in an embodiment in which a computer system 520 accesses a web service to train and use one or more AVI neural networks, the software instructions for the image analysis module 525 may be stored on a remote server.

[0043] The image analysis module 525 includes software for training one or more AVI neural networks using images stored in the image library 530. The image library 530 may be stored in the memory unit 523 or in another local or remote memory (e.g., memory connected to a remote library server). In addition to training, the image analysis module 525 may implement / run the trained AVI neural network by, for example, applying newly acquired images by VIS 505 (or another visual inspection system) to the neural network after certain preprocessing of the images may be performed, as discussed below. In various embodiments, the AVI neural network trained and / or run by the image analysis module 525 may classify the entire image (e.g., whether it is defective or not, or whether it has a specific type of defect such as a crimp blemish or a crimp defect in general), detect objects in the image (e.g., detect the location of a foreign object other than a bubble in a container image), or perform some combination of these (e.g., one neural network classifies the image and another performs object detection). As used herein, unless the context explicitly indicates a more specific use, “object detection” broadly refers to techniques for identifying specific locations of objects (e.g., particles, fibers, etc.) in an image, and / or specific locations of features of larger objects (e.g., damaged crimps or seals, cracks, or chips on a syringe or cartridge barrel, etc.), and may include, for example, techniques for segmenting container images or portions of images (e.g., pixel-by-pixel classification), or techniques for identifying objects and placing bounding boxes (or other boundary shapes) around those objects.

[0044] In embodiments in which an AVI neural network detects defects in a container, the defects may relate to any suitable container feature. For example, referring to the containers of the embodiments in Figures 2-4, a particular AVI neural network implemented by the image analysis module 525 may detect whether the container has cracks or stains, whether the flange is deformed, whether the needle shield is properly positioned, whether the plunger or piston is defective, whether the Luer lock is defective, whether the crimp is properly positioned and / or defective (e.g., a scratch), whether the flip cap is properly positioned and / or defective, etc.

[0045] Module 525 may run a trained AVI neural network for the purpose of verification, qualification, and / or inspection during commercial production. In one embodiment, for example, the image analysis module 525 is used solely for training and verifying an AVI neural network, which is then ported to another computer system (for example, using another module similar to module 525) for qualification and inspection during commercial production. In some embodiments in which the image analysis module 525 trains / runs multiple neural networks, the image analysis module 525 includes separate software for each neural network.

[0046] The AVI neural network may be trained using images from, for example, six vials, after augmenting the relevant training images by adjusting brightness, vertical mirroring, adding noise, and skewing the images, as well as skewing the bounding box (i.e., the training set may be multiplied by five). Deep learning may be used in general to detect defects in the images. By using a previously trained AVI neural network, the time required to set up automated inspection recipes for new products is further reduced. The AVI neural network of this disclosure may be used for high-mix low-volume production scenarios such as clinical operations or small batches of products, and may be implemented using state-of-the-art deep learning techniques (e.g., the image analysis module 525 in Figure 5).

[0047] In some embodiments, the VIS control module 526 controls / automates the operation of the VIS 505 so that container images can be generated with little or no human interaction. The VIS control module 526 may cause a predetermined fixed-position imaging system to acquire container images by transmitting commands or other electronic signals (e.g., generating pulses on a control line) to the imaging device. The VIS 505 may transmit the acquired container images to a computer system 520, which may store the images in a memory unit 523 for local processing. In alternative embodiments, the VIS 505 may be locally controlled, in which case the VIS control module 526 may have fewer functions than those described herein (e.g., only handling the reading of images from the VIS 505) or may be omitted entirely from the memory unit 523.

[0048] Figure 6 shows a method 600 for operating an automated visual inspection (AVI) system, which may be implemented by a processor (e.g., processing unit 522 in Figure 5) that performs at least a portion of the visual inspection system (VIS) control module 526 and / or the image analysis module 525. The AVI system may be similar to, for example, any one of the AVI systems 200 in Figure 2, 300a to c in Figures 3A to 3B, 400 in Figure 4, or 500 in Figure 5. In particular, the processing unit 522 may perform the VIS control module 526 to cause the processing unit 522 to align, for example, the central container fixing axes 209, 309, 409 of the container fixing devices 204, 304, 404 with the central image axes 208, 308, 408 of the fixed-position imaging systems 205, 305, 405 (block 640). The processing unit 522 may execute the VIS control module 526 to power the processing unit 522, for example, the lighting sources 210, 310, 410, and 510 (block 641).

[0049] The processing unit 522 may further execute the VIS control module 526 to cause the processing unit 522 to simultaneously capture, for example, the first images 202b1, 302b1, and 402b1 of the first subset 202a1, 302a1, and 402a1 of the containers, and the second images 202b2, 302b2, and 402b2 of the second subset 202a2, 302a2, and 402a2 of the containers (block 642). The processing unit 522 may execute the image analysis module 525 to cause the processing unit 522 to analyze, for example, the first images 202b1, 302b1, 402b1 and the second images 202b2, 302b2, 402b2, and to examine the first subset 202a1, 302a1, 402a1 and the second subset 202a2, 302a2, 402a2 of the containers for the same set of one or more features (block 643).

[0050] A method for imaging multiple containers includes transporting the containers in front of a first fixed-position imaging system and a second fixed-position imaging system using a transport mechanism. The method also includes capturing a first image of a first subset of the containers using the first fixed-position imaging system. The method further includes capturing a second image of a second subset of the containers simultaneously with the first image, from the same viewpoint relative to the containers, using the second fixed-position imaging system.

[0051] The fixed-position imaging system of this disclosure may reduce the complexity of AVI systems. The fixed-position imaging system may also reduce the lifecycle maintenance of AVI systems. Furthermore, the fixed-position imaging system may improve quality inspection by reducing vibration and lens errors.

[0052] Design challenges exist in camera connectivity and signal management in a mobile AVI system 100. The fixed-position imaging system of this disclosure does not include mobile camera connectivity.

[0053] The fixed-position imaging system may offer cost savings in design compared to the imaging system 105. The fixed-position imaging system may increase station speed compared to the AVI system 100. The fixed-position imaging system may be 100 times faster than the mechanical AVI system 100 shown in Figure 1.

[0054] Systems, methods, apparatus, and components thereof have been described in terms of exemplary embodiments, but they are not limited to these exemplary embodiments. Detailed descriptions are to be interpreted as examples only, and since it would be impractical, if not impossible, to describe all possible embodiments, not all possible embodiments of the present invention are described. Many alternative embodiments can be carried out using either the current art or art developed after the filing date of this patent, and these still fall within the scope of the claims defining the present invention.

[0055] Those skilled in the art will understand that various modifications, variations, and combinations of the above embodiments can be made without departing from the scope of the present invention, and that such modifications, variations, and combinations are to be interpreted as being within the scope of the concept of the present invention.

Claims

1. A system for capturing images of multiple containers, A first fixed-position imaging system configured to capture a first image, A second fixed-position imaging system configured to capture a second image, The system includes a transport mechanism configured to transport containers through the first and second fixed-position imaging systems, The first fixed-position imaging system, the second fixed-position imaging system, and the transport mechanism are configured such that the first fixed-position imaging system captures a first image of a first subset of the container, and the second fixed-position imaging system simultaneously captures a second image of a second subset of the container from the same viewpoint relative to the container. system.

2. The system further comprises one or more processors configured to analyze the first image and the second image to examine a first subset of the container and a second subset of the container for the same set of one or more features, The system according to claim 1.

3. The system according to claim 1 or 2, wherein each of the containers to be transported is selected from the group including vials, syringes, or cartridges.

4. The system according to any one of claims 1 to 3, wherein the first fixed-position imaging system includes a first imaging device having a first telecentric lens, and the second fixed-position imaging system includes a second imaging device having a second telecentric lens.

5. The system according to any one of claims 1 to 4, wherein the first fixed-position imaging system includes a first mirror configured to align a first central image axis with a first subset of the container, and the second fixed-position imaging system includes a second mirror configured to align a second central image axis with a second subset of the container.

6. The system according to any one of claims 1 to 5, wherein the first fixed-position imaging system includes a first mirror oriented at a 45-degree angle with respect to the first central image axis, and the second fixed-position imaging system includes a second mirror oriented at a 45-degree angle with respect to the second central image axis.

7. The system further comprises a first subset of the container, a first fixed-position imaging system, a second subset of the container, and a backlight oriented to emit light toward the second fixed-position imaging system. The system according to any one of claims 1 to 6.

8. A method for imaging multiple containers, Using a transport mechanism, the container is transported in front of the first fixed-position imaging system and the second fixed-position imaging system. Using the first fixed-position imaging system, a first image of a first subset of the container is acquired, This includes acquiring a second image of a second subset of the container using the second fixed-position imaging system, wherein the acquisition of the first image and the acquisition of the second image are performed simultaneously and from the same viewpoint relative to the container. method.

9. The system further comprises one or more processors configured to analyze the first image and the second image to examine a first subset of the container and a second subset of the container for the same set of one or more features, The method according to claim 8.

10. The method according to claim 8 or 9, wherein the first fixed-position imaging system includes a first imaging device having a first telecentric lens, and the second fixed-position imaging system includes a second imaging device having a second telecentric lens.

11. The method according to any one of claims 8 to 10, wherein the first fixed-position imaging system includes a first mirror configured to align a first central image axis with a first subset of the container, and the second fixed-position imaging system includes a second mirror configured to align a second central image axis with a second subset of the container.

12. The method according to any one of claims 8 to 11, wherein the first fixed-position imaging system includes a first mirror oriented at a 45-degree angle with respect to the first central image axis, and the second fixed-position imaging system includes a second mirror oriented at a 45-degree angle with respect to the second central image axis.

13. The method further includes using a backlight to emit light toward a first subset of the container, the first fixed-position imaging system, a second subset of the container, and the second fixed-position imaging system. The method according to any one of claims 8 to 12.

14. The method according to any one of claims 8 to 13, wherein a first subset of the containers includes two or more containers, and a second subset of the containers includes two or more containers.

15. A non-temporary computer-readable medium having computer-readable instructions to be stored, wherein, when an instruction is executed by one or more processors, the one or more processors, The transport mechanism is controlled to transport the container in front of the first fixed-position imaging system and the second fixed-position imaging system. Simultaneously, (i) the first fixed-position imaging system is controlled to capture a first image of a first subset of the container, and (ii) the second fixed-position imaging system is controlled to capture a second image of a second subset of the container from the same viewpoint relative to the container. Non-temporary computer-readable media.

16. Further execution of the computer-readable instruction by the one or more processors further involves the one or more processors: The first image and the second image are analyzed to examine the first subset and the second subset of the container for the same set of one or more features. The non-temporary computer-readable medium according to claim 15.

17. The non-temporary computer-readable medium according to claim 15 or 16, wherein the first fixed-position imaging system includes a first imaging device having a first telecentric lens, and the second fixed-position imaging system includes a second imaging device having a second telecentric lens.

18. The non-temporary computer-readable medium according to any one of claims 15 to 17, wherein the first fixed-position imaging system includes a first mirror configured to align a first central image axis with a first subset of the container, and the second fixed-position imaging system includes a second mirror configured to align a second central image axis with a second subset of the container.

19. The non-temporary computer-readable medium according to any one of claims 15 to 18, wherein the first fixed-position imaging system includes a first mirror oriented at a 45-degree angle with respect to the first central image axis, and the second fixed-position imaging system includes a second mirror oriented at a 45-degree angle with respect to the second central image axis.

20. Further execution of the computer-readable instruction by the one or more processors further involves the one or more processors: Controlling a backlight oriented to emit light toward a first subset of the container, a first fixed-position imaging system, a second subset of the container, and the second fixed-position imaging system. A non-temporary computer-readable medium according to any one of claims 15 to 19.

Images