Assembly line with integrated electronic visual inspection

Abstract

To provide a new technique for product assembly that can reduce errors and increase the efficiency of package assembly.SOLUTION: A plurality of imagers acquires images in visual fields, analyzes the images to determine the number and / or orientation of one or more articles in each visual field, the one or more articles including two or more appearances of a first pattern, determines that the two or more appearances of the first pattern are aligned on the same axis in each visual field, determines the number and / or orientation of one or more articles in each visual field, on the basis of the determination that the two or more appearances of the first pattern are aligned on the same axis in each visual field, and generates acceptance inspection signals on the basis of the determination of the number and / or orientation of one or more articles in each visual field.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present invention relates to an assembly line incorporating electronic visual inspection. [Background technology]

[0002] Assembling a product containing multiple items, such as pharmaceutical packaging, is a complex task. Assembly can proceed in one or more stages, with items being placed within the product at each stage. Errors can occur at each stage by failing to include the correct item in the product, including too many of the correct items in the product, and / or including the wrong item in the product. Products ultimately shipped with errors result in lost revenue, increased customer complaints, and lost time addressing customer complaints. In the case of pharmaceutical packaging, one unintended consequence of improper packaging is that clinicians or patients may be reluctant to use the medication contained within the improperly assembled package. This may be particularly true for medications administered parenterally, e.g., subcutaneously, intramuscularly, intravenously, intraocularly, or by inhalation. Even if an improperly assembled package is returned to the manufacturer by the clinician or patient, regulatory agencies such as the U.S. Food and Drug Administration will not allow the medication to be repackaged, resulting in a Notice of Event (NOE). Such an NOE can trigger an investigation, increased costs, and potentially a loss of competitiveness. Summary of the Invention [Problem to be solved by the invention]

[0003] It would therefore be desirable to develop new techniques for product assembly that overcome these and other limitations of the prior art and enhance it by reducing errors and increasing the efficiency of package assembly. [Means for solving the problem]

[0004] It should be understood that both the following general description and the following detailed description are exemplary and explanatory only and not limiting. Methods and systems are disclosed for acquiring a first image of a tray, determining the presence or absence of one or more first patterns in the first image, determining a rotation of each of the one or more first patterns in the first image, and performing an action based on the presence or absence and rotation of the one or more first patterns in the first image.

[0005] Additional advantages will be set forth in part in the description which follows, or may be learned by practice. These advantages will be realized and attained by means of the elements and combinations particularly pointed out in the appended claims. [Brief explanation of the drawings]

[0006] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments and, together with the description, serve to explain the principles of the method and system. [Figure 1] Exemplary System. [Figure 2] An illustrative image of the object. [Figure 3A] An illustrative image of the object. [Figure 3B] An illustrative image of the object. [Figure 4A] An illustrative image of the object. [Figure 4B] An illustrative image of the object. [Figure 5A] An illustrative image of the object. [Figure 5B] An illustrative image of the object. [Figure 6A] An illustrative image of the object. [Figure 6B] An illustrative image of the object. [Figure 7A] An illustrative image of the object. [Figure 7B] An illustrative image of the object. [Figure 8A] An illustrative image of the object. [Figure 8B]An illustrative image of the object. [Figure 9] 1 illustrates an exemplary embodiment of an exemplary system. [Figure 10] 1 is a flowchart illustrating an exemplary method. [Figure 11] Exemplary Operating Environment. DETAILED DESCRIPTION OF THE INVENTION

[0007] Before the present methods and systems are disclosed and described, it is to be understood that the present methods and systems are not limited to particular methods, components, or implementations. It is further understood that the terminology used herein is for the purpose of describing particular embodiments only and is not intended to be limiting.

[0008] Furthermore, as used in this specification and the appended claims, the singular forms "a," "an," and "the" include plural referents unless the context clearly dictates otherwise. Ranges may be expressed herein as ranging from "about" one particular value and / or to "about" another particular value. When such a range is expressed, another embodiment includes from the one particular value and / or to the other particular value. Similarly, when values ​​are expressed as approximations, by use of the antecedent "about," it is understood that the particular value forms another embodiment. It will be further understood that the endpoints of each range have meaning in relation to the other endpoint, and independently of the other endpoint.

[0009] "Optional" or "optionally" means that the subsequently described event or circumstance may or may not occur, and that the description includes cases where the event or circumstance occurs and cases where the event or circumstance does not occur.

[0010] Throughout the description and claims of this specification, the term "comprise" and variations of terms such as "comprising" and "comprises" mean "including, but not limited to," and are not intended to exclude, for example, other components, integers, or steps. "Exemplary" means "an example of" and is not intended to convey an indication of a preferred or ideal embodiment. "Etc." is used for descriptive purposes, not limiting.

[0011] Disclosed are components that can be used to implement the disclosed methods and systems. These and other components are disclosed herein, and when combinations, subsets, interactions, groups, etc. of these components are disclosed, it is understood that specific reference to each of the various individual, collective, combinations, and permutations thereof may not be explicitly disclosed, but each is specifically contemplated and described herein for all methods and systems. This applies to all aspects of the present application, including, but not limited to, steps in the disclosed methods. Thus, where there are various additional steps that may be performed, it is understood that these additional steps can each be performed with any specific embodiment or combination of embodiments of the disclosed methods.

[0012] The present method and system may be more readily understood by reference to the following detailed description of the preferred embodiments and the examples included therein, as well as the drawings and their accompanying description.

[0013] As will be appreciated by those skilled in the art, the present methods and systems may take the form of an entirely hardware embodiment, an entirely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present methods and systems may take the form of a computer program product on a computer-readable storage medium having computer-readable program instructions (e.g., computer software) embodied in the storage medium. More specifically, the present methods and systems may take the form of web-implemented computer software. Any suitable computer-readable storage medium may be utilized, including a hard disk, a CD-ROM, an optical storage device, or a magnetic storage device.

[0014] Embodiments of the present methods and systems are described below with reference to block diagrams and flowchart illustrations of methods, systems, apparatuses, and computer program products. It will be understood that each block of the block diagrams and flowchart illustrations, and combinations of blocks in the block diagrams and flowchart illustrations, respectively, can be implemented by computer program instructions. These computer program instructions can be loaded into a general-purpose computer, special-purpose computer, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute on the computer or other programmable data processing apparatus, create means for performing the functions specified in one or more blocks of the flowchart.

[0015] These computer program instructions can also be stored in a computer-readable memory that can instruct a computer or other programmable data processing apparatus to function in a particular manner to produce an article of manufacture that includes computer-readable instructions for performing the functions specified in one or more blocks of the flowchart, such that the instructions stored in the computer-readable memory instruct the article of manufacture that includes computer-readable instructions for performing the functions specified in one or more blocks of the flowchart. The computer program instructions can also be loaded into a computer or other programmable data processing apparatus to perform a series of operational steps on the computer or other programmable apparatus, creating a computer-implemented process such that the instructions executing on the computer or other programmable apparatus provide steps for performing the functions specified in one or more blocks of the flowchart.

[0016] Thus, the blocks in the block diagrams and flowchart diagrams represent combinations of means for performing the specified functions, combinations of steps for performing the specified functions, and program instruction means for performing the specified functions. It will also be understood that each block of the block diagrams and flowchart diagrams, and combinations of blocks in the block diagrams and flowchart diagrams, can be implemented by a dedicated hardware-based computer system that performs the specified functions or steps, or a combination of dedicated hardware and computer instructions.

[0017] FIELD OF THE DISCLOSURE The present disclosure relates to improving computer functionality related to manufacturing and product assembly. 1 is a block diagram illustrating various aspects of an exemplary system 100 in which the present methods and systems may operate. Those skilled in the art will understand that functional descriptions are provided herein and that each function may be performed by software, hardware, or a combination of software and hardware.

[0018] In one aspect, the system 100 may include a conveyor belt 101. The conveyor belt 101 may include one or more cleats 102. The one or more cleats 102 may be made of rubber or a similar material for attachment to the conveyor belt 101. The one or more cleats 102 may be raised or otherwise extend over the surface of the conveyor belt 101. The one or more cleats 102 may include a front cleat and a rear cleat based on the direction of travel 103. The front cleats and the rear cleats may be relative to objects placed on the belt such that the front cleat is in front of the object with respect to the direction of travel 103 and the rear cleat is behind the object with respect to the direction of travel 103. Thus, a front cleat for a first object may be a rear cleat for a second object that is in front of the first object, and so on. One or more objects 104 may be placed on the conveyor belt 101. In one embodiment, the one or more objects 104 can include one or more pre-assembled products. For example, the one or more objects 104 can include a tray. The tray can be configured to hold one or more items. The one or more items can be related to a medical treatment. For example, the one or more items can include one or more syringes, an autoinjector, one or more needles for a syringe, one or more containers of a medication, one or more pamphlets or instruction sets, combinations thereof, etc.

[0019] In one embodiment, the set of instructions provides information about how to use and administer the medication, hi another embodiment, the instructions are drug labeling approved by a regulatory agency such as the U.S. Food and Drug Administration.

[0020] In one embodiment, the medicament is a solid formulation. In another embodiment, the medicament is a liquid formulation. In another embodiment, the medicament is a gel formulation. In one embodiment, the agent is formulated for oral administration. In another embodiment, the agent is formulated for parenteral administration. In another embodiment, the agent is formulated for subcutaneous administration. In another embodiment, the agent is formulated for intramuscular administration. In another embodiment, the agent is formulated for intravenous administration. In another embodiment, the agent is formulated for inhaled administration. In another embodiment, the agent is formulated for intraocular administration.

[0021] In one embodiment, the agent comprises a small molecule active ingredient. In another embodiment, the agent comprises a biologic. In another embodiment, the agent comprises a peptide or polypeptide active ingredient. In one embodiment, the drug comprises vascular endothelial growth factor (VEGF) derivative active ingredient.In another embodiment, the drug comprises aflibercept described in one or more of United States Patent No. 7,070,959; United States Patent No. 7,303,746; United States Patent No. 7,303,747; United States Patent No. 7,306,799; United States Patent No. 7,374,757; United States Patent No. 7,374,758; United States Patent No. 7,531,173; United States Patent No. 7,608,261; United States Patent No. 7,972,598; United States Patent No. 8,029,791; United States Patent No. 8,092,803; United States Patent No. 8,343,737; United States Patent No. 8,647,842 (each of which is incorporated by reference in its entirety).

[0022] The conveyor belt 101 may pass over a drive roll that may be driven by a stepper motor 105. The use of the stepper motor 105 allows for precise positioning of one or more objects 104 relative to the cameras 106, 107, and 108. The length of each of the one or more objects 104 may be expressed as a precise number of motor steps. The conveyor belt 101 may move precisely forward or backward to move each of the one or more objects 104 into the fields of view 109, 110, and 111 associated with the cameras 106, 107, and 108. A programmable logic controller (PLC) 112 (which may comprise a computing device, PLC, or other controller / processor) may be configured to cause the stepper motor 105 to take any number of steps in either direction to move the one or more objects 104 into the fields of view 109, 110, and 111.

[0023] In one aspect, camera 106, camera 107, and / or camera 108 can be configured to scan, decode, read, detect, image, capture, and / or interpret visual codes. In some aspects, camera 106, camera 107, and / or camera 108 can be configured to process laser, linear, or area imaging. For example, in one aspect, camera 106, camera 107, and / or camera 108 can include imagers for scanning, reading, and decoding one-dimensional or two-dimensional bar codes. Camera 106, camera 107, and / or camera 108 can include any imager, bar code scanner, or visual code scanner capable of extracting information from visual codes consistent with disclosed embodiments. In certain aspects, camera 106, camera 107, and / or camera 108 can be configured to process scanned bar codes, images, and other data. Camera 106, camera 107, and / or camera 108 may include one or more depth cameras for capturing, processing, sensing, observing, modeling, detecting, and interacting with a three-dimensional environment. In certain aspects, camera 106, camera 107, and / or camera 108 may recognize and detect the depth and color of objects within field of view 109, field of view 110, and field of view 111, respectively. Camera 106, camera 107, and / or camera 108 may also provide other camera and video recorder functions, such as taking pictures, recording video, streaming images or other data, storing data in an image buffer, etc. These functions may or may not include depth information. In conjunction with hardware and / or software processes consistent with disclosed embodiments, camera 106, camera 107, and / or camera 108 may determine the size, orientation, and visual characteristics of one or more objects 104. Camera 106, camera 107 and / or camera 108 may include or embody any camera known to those skilled in the art that is capable of processing the processes disclosed herein.Camera 106, camera 107, and / or camera 108 may include appropriate hardware and software components (e.g., circuitry, software instructions, etc.) for sending signals and information to and from pass / fail controller 113 to execute processes consistent with the disclosed embodiments. Pass / fail controller 113 may include a computing device, PLC, or other controller / processor. Camera 106, camera 107, and / or camera 108 may output images and / or one or more notifications to monitor 114, monitor 115, and monitor 116, respectively.

[0024] Positioning of one or more objects 104 within field of view 109, field of view 110, and field of view 111 can occur upon startup of system 100 or can be adjusted while system 100 is in use. One or more of camera 106, camera 107, and / or camera 108 can be used to ensure proper positioning of conveyor belt 101. For example, camera 107 can be configured to generate an image of an area within field of view 110. Camera 107 can determine the position of one or more cleats 102 in the image. In one embodiment, camera 107 can determine the position of a front cleat. Camera 107 can compare the determined position of one or more cleats 102 in the image to a reference position. If the determined position is equal to the reference position, no adjustment of conveyor belt 101 is necessary. If the determined position is not equal to the reference position, camera 107 can determine an offset based on the difference between the determined position and the reference position. The offset can be determined in a measure of distance, e.g., millimeters, centimeters, inches, etc., and / or the offset can be determined as a number of steps. The camera 107 can send a signal to the PLC 112 to actuate the stepper motor 105 to move the conveyor belt 101 forward or backward by the offset.

[0025] During operation, system 100 can be configured to evaluate a current assembly state of one or more objects 104 and take one or more actions based on the current assembly state. As each of one or more objects 104 advances along conveyor belt 101, the one or more objects 104 are positioned within field of view 109, field of view 110, and field of view 111 of camera 106, camera 107, and / or camera 108, respectively. While FIG. 1 illustrates only three cameras, it is specifically contemplated that fewer than three or more than three cameras may be used. It is further contemplated that conveyor belt 101 may be configured to have four or more depicted objects 104 positioned thereon, regardless of the number of cameras. As one or more objects 104 advance along conveyor belt 101, one or more articles may be assembled into one or more objects 104 by a human operator or a robot.

[0026] When one or more objects 104 are each within the field of view of one of the cameras, the camera can generate an image of the object 104 within the field of view associated with that camera. For example, camera 106 can generate an image of an area within field of view 109, camera 107 can generate an image of an area within field of view 110, and camera 108 can generate an image of an area within field of view 111. Camera 106, camera 107, and / or camera 108 can each analyze their corresponding images. Analyzing the images can include determining the presence or absence of one or more patterns. The one or more patterns can include text patterns, number patterns, symbol patterns, and combinations thereof. For example, a text pattern can include any character string such as "filter needle." A number pattern can include any number sequence such as "6941518." A symbol pattern can include any symbol sequence such as "●□□◆." In one aspect, camera 106, camera 107, and / or camera 108 can utilize optical character recognition (OCR) to "read" the one or more patterns. In another embodiment, camera 106, camera 107 and / or camera 108 may be configured not to utilize OCR, but rather to simply recognize one or more patterns as particular patterns.

[0027] In one embodiment, one or more patterns may be embodied on one or more articles that are assembled into one or more objects 104. In one embodiment, at least a portion of one or more articles may include one or more associated patterns. Thus, if camera 106, camera 107, and / or camera 108 determine the presence of one or more patterns, the presence of the one or more patterns indicates the presence of an article associated with the particular pattern. For example, if camera 106 determines the presence of a "filter needle" in an image taken of an area within field of view 109, a conclusion may be drawn that an article associated with the pattern "filter needle" is present on object 104 within field of view 109. Camera 106, camera 107, and / or camera 108 may be configured to determine the presence or absence of multiple patterns in a single image. For example, camera 106 may determine the presence of "filter needle" and "filter needle" in an image taken of an area within field of view 109. The presence of both patterns may indicate the presence of an article associated with the pattern "filter needle," appearing twice, on object 104 within field of view 109.

[0028] Each of the articles that can be assembled into one or more objects 104 can be associated with one or more patterns that indicate the presence or absence of a particular number of articles. For example, an article may be embodied with only one occurrence of a particular pattern. If camera 106, camera 107, and / or camera 108 determines that a particular pattern only occurs once, a conclusion can be drawn that only one article is present. However, if camera 106, camera 107, and / or camera 108 determines that a particular pattern occurs two or more times, a conclusion can be drawn that multiple articles are present. In another example, an article may be embodied with two occurrences of a particular pattern. If camera 106, camera 107, and / or camera 108 determines that a particular pattern only occurs two times, a conclusion can be drawn that only one article is present. However, if camera 106, camera 107, and / or camera 108 determines that a particular pattern occurs one or more than two times, a conclusion can be drawn that multiple articles are present. In further embodiments, the article may be embodied with a range of particular patterns. For example, the article may be embodied with one or two occurrences of the particular pattern. If camera 106, camera 107, and / or camera 108 determines that the particular pattern occurs one or two times, a conclusion can be drawn that only one article is present. However, if camera 106, camera 107, and / or camera 108 determines that the particular pattern occurs three or more times, a conclusion can be drawn that multiple articles are present.

[0029] Camera 106, camera 107, and / or camera 108 can each be configured to analyze the entire image or one or more specific regions of the image. FIG. 2 shows an example image 200 of object 104. Object 104 can include a tray 201 configured to store one or more items. One or more items can be assembled into tray 201 such that at least a portion of the one or more items is present in the one or more specific regions. Tray 201 can include one or more regions, e.g., region 202, region 203, and region 204. Region 202, region 203, and region 204 can each be associated with an area where one or more patterns should be present when the item is present in tray 201. For example, region 202 can be associated with the position of a vial cap of a vial when assembled into tray 201, region 203 can be associated with the position of one or more syringes and / or one or more needles when assembled into tray 201, and region 204 can be associated with the position of one or more pamphlets when assembled into tray 201. Camera 106, camera 107, and / or camera 108 can each be configured to analyze one or more assigned regions of image 200. For example, camera 106 can be assigned to analyze region 202 and region 203, camera 107 can be assigned to analyze region 203, and camera 108 can be assigned to analyze region 203 and region 204. Any combination of assigned regions is contemplated. Furthermore, camera 106, camera 107, and / or camera 108 can each be configured to determine the presence or absence of one or more assigned patterns in the assigned regions.For example, camera 106 may be assigned to determine the presence or absence of vial caps in region 202 and the presence or absence of a first pattern (including the number of occurrences of the first pattern) in region 203, camera 107 may be assigned to determine the presence or absence of a second pattern (including the number of occurrences of the second pattern) in region 203, and camera 108 may be assigned to determine the presence or absence of a third pattern (including the number of occurrences of the third pattern) in region 203 and the presence or absence of a fourth pattern (including the number of occurrences of the fourth pattern) in region 204. Any combination of assigned patterns and assigned regions is contemplated.

[0030] Returning to FIG. 1 , each of the one or more objects 104 can be configured to include a specific number of each of the one or more items. The presence of the specific number of each item indicates that the one or more objects 104 are correctly assembled. The presence of anything other than the specific number of each item indicates that the one or more objects 104 are incorrectly assembled. Camera 106, camera 107, and / or camera 108 can each be configured to perform an independent evaluation of the objects 104 within a corresponding field of view. If the camera determines that the specific number of items that the camera is configured to detect are present, the camera can issue a PASS signal to pass / fail controller 113. If the camera determines that the specific number of items that the camera is configured to detect are not present, the camera can issue a FAIL signal to pass / fail controller 113. If camera 106, camera 107, and / or camera 108 each issues a PASS signal to pass / fail controller 113, pass / fail controller 113 can provide a signal to PLC 112 to cause stepper motor 105 to advance conveyor belt 101 to advance one or more objects 104 and position them under the field of view of the next camera. Pass / fail controller 113 can further send a notification to each of monitors 114-116 to display a PASS notification. If one or more of camera 106, camera 107, and / or camera 108 issues a FAIL signal to pass / fail controller 113, pass / fail controller 113 does not provide a signal to PLC 112 to advance stepper motor 105. Pass / fail controller 113 can further send a notification to monitor 114-116 associated with the camera issuing the FAIL signal to display a FAIL notification. An operator (eg, human or robot) located on the monitor 114-116 displaying the FAIL notification can take corrective action to remedy the FAIL condition.For example, if a FAIL signal is issued as a result of a missing item, an operator can replace the missing item, so that the camera that previously made the FAIL determination can re-generate and re-analyze an image, determine that the item is now present, and issue a PASS signal to pass / fail controller 113. In another example, if a FAIL signal is issued as a result of one or more extra items, an operator can remove the one or more extra items, so that the camera that previously made the FAIL determination can re-generate and re-analyze an image, determine that the required number of items is now present, and issue a PASS signal to pass / fail controller 113.

[0031] In further embodiments, analysis of images by camera 106, camera 107, and / or camera 108 can include not only determining the presence or absence of one or more patterns, but also determining the rotation of the multiple patterns. In one embodiment, the multiple patterns can be embodied on one or more articles that are assembled into one or more objects 104 along a particular axis. In one embodiment, at least a portion of the one or more articles can include multiple associated patterns along a particular axis. Thus, if camera 106, camera 107, and / or camera 108 determine the presence of multiple patterns along a particular axis, the presence of multiple patterns along the particular axis indicates the presence of an article associated with the specific pattern along the particular axis. For example, if camera 106 determines the presence of "filter needle" and "filter needle" along the same axis (e.g., 30°, 60°, 90°, 120°, 180°, etc.) in an image captured of an area within field of view 109, a conclusion can be drawn that an article associated with the patterns "filter needle" and "filter needle" along the same axis is present on object 104 within field of view 109. Camera 106, camera 107, and / or camera 108 can be configured to determine the rotation of multiple patterns in a single image. For example, camera 106 can determine the presence of "filter needle" and "filter needle" along a first axis and the presence of "syringe needle" and "syringe needle" along a second axis in an image captured of an area within field of view 109. The presence of both patterns along two different axes can indicate that an article associated with two occurrences of the pattern "filter needle" along the first axis is present on object 104, and that an article associated with two occurrences of the pattern "syringe needle" along the second axis is also present on object 104. As a further example, camera 106 can determine the presence of "filter needle" and "filter needle" along a first axis and the presence of "filter needle" along a second axis in an image captured of an area within field of view 109. The presence of both patterns along two different axes can indicate that two occurrences of an article associated with the pattern "filter needle" are present on object 104.

[0032] Each of the articles that can be assembled into one or more objects 104 can be associated with one or more patterns that are embodied along a particular axis, indicating the presence or absence of a particular number of the articles. For example, an article may be embodied with a particular pattern appearing twice along a particular axis. If camera 106, camera 107, and / or camera 108 determines that a particular pattern appears only twice along a particular axis, a conclusion can be drawn that only one article is present. However, if camera 106, camera 107, and / or camera 108 determines that a particular pattern appears along multiple axes, a conclusion can be drawn that multiple articles are present.

[0033] 3A and 3B show exemplary images 300 and 303 of tray 201 containing item 301 and item 302. For example, item 301 may be a vial, and item 302 may be a filter needle. Any of cameras 106, 107, and / or 108 generating image 300 may determine that a vial cap is present in region 202. The presence of a single vial cap indicates the presence of item 301. Cameras 106, 107, and / or 108 generating image 300 may determine that two occurrences of pattern "Text A" are present in region 203. In one embodiment, the two occurrences of pattern "Text A" may indicate the presence of one or more instances of item 302, and camera 106, camera 107, and / or camera 108 may generate a PASS or FAIL signal, as appropriate. In another embodiment, depending on the pattern configuration on article 302 (e.g., a single instance of article 302 can have either a single occurrence of "Text A" or dual occurrences of "Text A"), camera 106, camera 107, and / or camera 108 can determine whether "Text A" and "Text A" appear on the same axis. If "Text A" and "Text A" appear on the same axis, camera 106, camera 107, and / or camera 108 can determine that a single instance of article 302 is present, and camera 106, camera 107, and / or camera 108 can generate a PASS or FAIL signal, as appropriate. If "Text A" and "Text A" appear on different axes, camera 106, camera 107, and / or camera 108 can determine that multiple instances of article 302 are present, and camera 106, camera 107, and / or camera 108 can generate a PASS or FAIL signal, as appropriate. In one embodiment, the axis determination can be used to verify that any number of items 302 are present and generate a PASS or FAIL signal based on the expected number of instances of item 302 versus the determined number of instances of item 302.

[0034] 4A shows an example image 400 of tray 201 containing two instances of item 301 and item 302. Camera 106, camera 107, and / or camera 108 generating image 400 can determine that three occurrences of a pattern (“Text A”) are present within region 203. In one embodiment, three occurrences of the pattern “Text A” can indicate the presence of one or more instances of item 302, and camera 106, camera 107, and / or camera 108 can generate a PASS or FAIL signal, as appropriate. In another embodiment, depending on the pattern configuration on item 302 (e.g., a single instance of item 302 can have one occurrence of “Text A,” double occurrences of “Text A,” or triple occurrences of “Text A”), camera 106, camera 107, and / or camera 108 can determine whether the three occurrences of “Text A” appear on the same axis. 4A, two occurrences of "Text A" appear on the same axis and one occurrence of "Text A" appears on a different axis. Thus, camera 106, camera 107, and / or camera 108 can determine that multiple instances of item 302 are present, and camera 106, camera 107, and / or camera 108 can generate a PASS or FAIL signal as appropriate. In one aspect, the axis determination can be used to verify that any number of items 302 are present, and a PASS or FAIL signal can be generated based on the expected number of instances of item 302 versus the determined number of instances of item 302.

[0035] 4B shows an example image 401 of tray 201 containing item 301, one instance of item 302, and one instance of item 402. In one embodiment, camera 106, camera 107, and / or camera 108 generating image 400 can determine that there are two occurrences of a first pattern ("Text A") and one occurrence of a second pattern ("Text B") within region 203. In one embodiment, the two occurrences of pattern "Text A" can indicate the presence of one or more instances of item 302, and camera 106, camera 107, and / or camera 108 can generate a PASS or FAIL signal, as appropriate. In another embodiment, depending on the pattern configuration on article 302 (e.g., a single instance of article 302 can have one occurrence of “Text A,” two occurrences of “Text A,” or three occurrences of “Text A”), camera 106, camera 107, and / or camera 108 can determine whether two occurrences of “Text A” appear on the same axis. As shown in FIG. 4B , two occurrences of “Text A” appear on the same axis. Thus, camera 106, camera 107, and / or camera 108 can determine that multiple instances of article 302 are present, and camera 106, camera 107, and / or camera 108 can generate a PASS or FAIL signal as appropriate. However, a single occurrence of pattern “Text B” can indicate that an article that should not be in tray 201 has been placed in tray 201 at this stage of the assembly process. Thus, camera 106, camera 107, and / or camera 108 can generate a FAIL signal based on the presence of a pattern that is not intended to be present.

[0036] In another embodiment, camera 106, camera 107, and / or camera 108 generating image 400 can determine that pattern "Text B" is present and can disregard the presence of pattern "Text A" (or anything else, as desired). In one embodiment, a single occurrence of pattern "Text B" can indicate the presence of one instance of article 302, and camera 106, camera 107, and / or camera 108 can generate a PASS signal.

[0037] 5A shows an example image 500 of tray 201 containing one instance of item 301, item 302, and item 501. Camera 106, camera 107, and / or camera 108 generating image 500 can be configured to ignore the vial cap in region 202 and to ignore the presence of pattern "Text A" in region 203. Instead, camera 106, camera 107, and / or camera 108 generating image 400 can determine that two occurrences of another pattern ("Text B") are present in region 203. In one embodiment, the two occurrences of pattern "Text B" can indicate the presence of one or more instances of item 501, and camera 106, camera 107, and / or camera 108 can generate a PASS or FAIL signal, as appropriate. In another embodiment, depending on the pattern configuration on the article 501 (e.g., a single instance of the article 501 can have one occurrence of "Text B," two occurrences of "Text B," or three occurrences of "Text B"), the camera 106, the camera 107, and / or the camera 108 can determine whether two occurrences of "Text B" appear on the same axis. As shown in FIG. 5A, two occurrences of "Text B" appear on the same axis. Thus, the camera 106, the camera 107, and / or the camera 108 can determine that one instance of the article 501 is present, and the camera 106, the camera 107, and / or the camera 108 can generate a PASS or FAIL signal, as appropriate. In one embodiment, the axis determination can be used to confirm the presence of any number of the article 501, and a PASS or FAIL signal can be generated based on the expected number of instances of the article 501 versus the determined number of instances of the article 501. Figure 5B shows an example image 503 of tray 201 containing item 301, item 302, and one instance of item 501. Figure 5B is similar to Figure 5A, except that Figure 5B shows that the pattern "Text B" appears twice along the same axis, but at a different angle than the axis in Figure 5A.

[0038] 6A shows an example image 600 of tray 201 containing two instances of item 301 and item 501. Camera 106, camera 107, and / or camera 108 generating image 600 can determine that four occurrences of pattern "Text B" are present within region 203. In one embodiment, the four occurrences of pattern "Text B" can indicate the presence of one or more instances of item 501, and camera 106, camera 107, and / or camera 108 can generate a PASS or FAIL signal, as appropriate. In another embodiment, depending on the pattern configuration on item 501 (e.g., a single instance of item 501 can have one occurrence of "Text B," two occurrences of "Text B," three occurrences of "Text B," or four occurrences of "Text B"), camera 106, camera 107, and / or camera 108 can determine the axis along which the four occurrences of "Text B" appear. 5A , two occurrences of "Text B" appear on a first axis and the other two occurrences of "Text B" appear on a second axis. Thus, because the two sets of "Text B" appear on different axes, camera 106, camera 107, and / or camera 108 can determine that multiple instances of item 501 are present, and camera 106, camera 107, and / or camera 108 can generate a PASS or FAIL signal as appropriate. In one aspect, the axis determination can be used to verify that any number of items 501 are present, and a PASS or FAIL signal can be generated based on the expected number of instances of item 501 versus the determined number of instances of item 501.

[0039] 6B shows an example image 601 of tray 201 containing two instances of item 301, item 302, and item 501. Camera 106, camera 107, and / or camera 108 generating image 601 can determine that three occurrences of pattern "Text B" are present within region 203. In one embodiment, the three occurrences of pattern "Text B" can indicate the presence of one or more instances of item 501, and camera 106, camera 107, and / or camera 108 can generate a PASS or FAIL signal, as appropriate. In another embodiment, depending on the pattern configuration on item 501 (e.g., a single instance of item 501 can have one occurrence of "Text B," two occurrences of "Text B," three occurrences of "Text B," or four occurrences of "Text B"), camera 106, camera 107, and / or camera 108 can determine the axis along which the three occurrences of "Text B" appear. 6B, two occurrences of "Text B" appear on a first axis and one occurrence of "Text B" appears on a second axis. Thus, because two sets of "Text B" appear on different axes, camera 106, camera 107, and / or camera 108 can determine that multiple instances of item 501 are present, and camera 106, camera 107, and / or camera 108 can generate a PASS or FAIL signal as appropriate. In one aspect, the axis determination can be used to verify that any number of items 501 are present, and a PASS or FAIL signal can be generated based on the expected number of instances of item 501 versus the determined number of instances of item 501.

[0040] 7A shows an example image 700 of tray 201 including item 301, item 302, item 501, one instance of item 701, and one instance of item 702. Camera 106, camera 107, and / or camera 108 generating image 700 can be configured to ignore the vial cap in region 202 and to ignore the presence of patterns “Text A” and “Text B” in region 203. Instead, camera 106, camera 107, and / or camera 108 generating image 700 can determine that two occurrences of another pattern (“Text D”) are present in region 203. In one embodiment, the two occurrences of pattern “Text D” can indicate the presence of one or more instances of item 701, and camera 106, camera 107, and / or camera 108 can generate a PASS or FAIL signal, as appropriate. In another embodiment, depending on the pattern configuration on the article 701 (e.g., a single instance of the article 701 can have one occurrence of "Text D," two occurrences of "Text D," or three occurrences of "Text D"), the camera 106, the camera 107, and / or the camera 108 can determine whether two occurrences of "Text D" appear on the same axis. As shown in FIG. 7A, two occurrences of "Text D" appear on the same axis. Thus, the camera 106, the camera 107, and / or the camera 108 can determine that one instance of the article 701 is present, and the camera 106, the camera 107, and / or the camera 108 can generate a PASS or FAIL signal, as appropriate. In one embodiment, the axis determination can be used to confirm that any number of the article 701 is present, and a PASS or FAIL signal can be generated based on the expected number of instances of the article 701 versus the determined number of instances of the article 701. In the same image 700, camera 106, camera 107 and / or camera 108 may determine that there are two occurrences of another pattern ("text C") within region 204.In one embodiment, two occurrences of the pattern "Text C" can indicate the presence of one or more instances of article 702, and camera 106, camera 107, and / or camera 108 can generate a PASS or FAIL signal, as appropriate. In another embodiment, depending on the pattern configuration on article 702 (e.g., a single instance of article 702 can have one occurrence of "Text C," two occurrences of "Text C," or three occurrences of "Text C"), camera 106, camera 107, and / or camera 108 can determine whether two occurrences of "Text C" appear on the same axis. As shown in FIG. 7A , two occurrences of "Text C" appear on the same axis. Thus, camera 106, camera 107, and / or camera 108 can determine that one instance of article 702 is present, and camera 106, camera 107, and / or camera 108 can generate a PASS or FAIL signal, as appropriate. In one embodiment, the axis determination can be used to verify the presence of any number of items 702 and generate a PASS or FAIL signal based on the expected number of instances of item 702 versus the determined number of instances of item 702. Figure 7B shows an example image 703 of tray 201 containing item 301, item 302, item 501, one instance of item 701, and one instance of item 702. Figure 7B is similar to Figure 7A, except that Figure 7B shows that pattern "Text D" appears twice along the same axis but at a different angle from the axis in Figure 7A, and similarly, pattern "Text C" appears twice along the same axis but at a different angle from the axis in Figure 7A.

[0041] 8A shows an example image 800 of tray 201 containing item 301, item 302, item 501, two instances of item 701, and one instance of item 702. Camera 106, camera 107, and / or camera 108 generating image 800 can determine that three occurrences of pattern "Text D" are present within region 203. In one embodiment, the three occurrences of pattern "Text D" can indicate the presence of one or more instances of item 701, and camera 106, camera 107, and / or camera 108 can generate a PASS or FAIL signal, as appropriate. In another embodiment, depending on the pattern configuration on item 701 (e.g., a single instance of item 701 can have one occurrence of "Text D," a double occurrence of "Text D," a triple occurrence of "Text D," or a quadruple occurrence of "Text D"), camera 106, camera 107, and / or camera 108 can determine the axis along which the three occurrences of "Text D" appear. As shown in FIG. 8A , two occurrences of “Text D” appear on a first axis and one occurrence of “Text D” appears on a second axis. Therefore, because two sets of “Text D” appear on different axes, camera 106, camera 107, and / or camera 108 can determine that multiple instances of article 701 are present, and camera 106, camera 107, and / or camera 108 can generate a PASS or FAIL signal as appropriate. In one embodiment, the axis determination can be used to verify that any number of articles 701 are present, and a PASS or FAIL signal can be generated based on the expected number of instances of article 701 versus the determined number of instances of article 701. FIG. 8B is similar to FIG. 8A , except that FIG. 8B shows that the pattern “Text D” appears twice along the first axis and once along the second axis, but the first and second axes are at a different angle than the axes in FIG. 8A .

[0042] 1 , camera 106, camera 107, and camera 108 can each independently determine both the presence or absence of one or more patterns in the image and the rotation of each of the one or more patterns in the image of object 104. Camera 106, camera 107, and camera 108 can each perform an action based on the presence, presence, and rotation of the one or more patterns in the image. If the camera determines that the correct number of items are present in the image of object 104 based on the presence of the patterns and the rotation of the patterns, the action can include sending a PASS signal to pass / fail controller 113. If the camera determines that the incorrect number of items are present in the image of object 104 based on the presence of the patterns and the rotation of the patterns, the action can include sending a FAIL signal to pass / fail controller 113. If camera 106, camera 107, and / or camera 108 each issues a PASS signal to pass / fail controller 113, pass / fail controller 113 can provide a signal to PLC 112 to cause stepper motor 105 to advance conveyor belt 101 to advance one or more objects 104 and position them under the field of view of the next camera. Pass / fail controller 113 can further send a notification to each of monitors 114-116 to display a PASS notification. If one or more of camera 106, camera 107, and / or camera 108 issues a FAIL signal to pass / fail controller 113, pass / fail controller 113 does not provide a signal to PLC 112 to advance stepper motor 105. Pass / fail controller 113 can further send a notification to monitor 114-116 associated with the camera issuing the FAIL signal to display a FAIL notification. An operator (eg, human or robot) located on the monitor 114-116 displaying the FAIL notification can take corrective action to remedy the FAIL condition.

[0043] In another embodiment, one or more of cameras 106, 107, and 108 can count the number of one or more objects 104. For example, as one or more objects 104 pass one of cameras 106, 107, and 108, the camera can increment a tally of the one or more objects 104 imaged by the camera. In a further embodiment, multiple empty locations may be interspersed among one or more objects 104. For example, in certain situations, one or more of cameras 106, 107, and 108 may not have an object 104 within their corresponding field of view. The conveyor belt 101 may have a pattern embodied thereon (e.g., a "no tray" pattern) at a location where an object 104 would otherwise be located. Cameras 106, 107, and 108 can identify the pattern and issue a PASS signal to encourage the advancement of the conveyor belt 101.

[0044] FIG. 9 illustrates an exemplary embodiment of system 100 showing the positioning of cameras 106, 107, and 108 relative to conveyor belt 101. FIG. 9 further illustrates the positioning of monitors 114-116. Stepper motor 105 is shown at one end of conveyor belt 101. One or more of PLC 112 and / or pass / fail controller 113 can be housed in housing 901. One or more dispensers 902 can be configured to store one or more items that are accessed during assembly into one or more objects. System 100 can include one or more emergency stop (“E-Stop”) buttons 903. E-Stop button 903 can be activated at any time to temporarily halt operation of system 100 for any reason. E-Stop button 903 can be reset and system 100 can be restarted (e.g., by an operator or technician who determines it is safe to do so). System 100 can include one or more optoswitches 904. Optoswitch 904 can be actuated ("tripped") by placing a finger or thumb on the saddle-like structure of optoswitch 904. This action interrupts the optical signal path and triggers a switching condition. Optoswitch 904 can be used to accept visual inspection during a "manual trigger" mode and to start / resume belt movement during an "autonomous" (or "automatic") mode.

[0045] System 100 can include a key switch mechanism 905, which can be used to toggle between an "autonomous" mode and a "manual trigger" mode. Under normal operation, regardless of the mode, a first operator station can include an operator loading trays onto conveyor belt 101. In one embodiment, these trays can be pre-loaded with pre-filled capped vials. In manual trigger mode, at a second operator station, an operator can load filter needle tips onto the tray. After this operation, camera 106 inspects the tray for the appropriate item. At a third operator station, an injection needle tip can be added to the tray. Camera 107 then inspects the tray for the appropriate item. At a fourth operator station, an operator loads empty blister-packaged syringes onto the tray. A fifth operator then loads a Physician Insert (PI) onto the tray. After the PI is loaded, camera 108 inspects the tray for completion. Once the tray has passed this final station, the filled tray exits the conveyor belt 101 for boxing.

[0046] In automatic mode, trays move automatically down the conveyor belt 101. The system 100 can maintain a dwell time (e.g., 1-5 seconds) before the conveyor belt 101 moves to the next position. This movement occurs only when all three inspection cameras (e.g., camera 106, camera 107, and camera 108) pass the tray being inspected by the corresponding camera ("pass"). A problem at any inspection station can cause the conveyor belt 101 to go into a "red light" state ("fail"), at which point an operator can either resolve the problem or remove the tray from the conveyor belt 101 (each camera can advance the conveyor belt 101 if the tray is not within its field of view). Advancement of the conveyor belt 101 can depend on all cameras detecting a "pass" tray configuration. Display screens (e.g., monitors 114-116) at each camera station can display the associated camera's video stream with an overlaid "pass," "fail," or "no job" status, depending on the inspection results. The online status of the camera can be reset from the monitors 114-116 as needed during operation.

[0047] In one embodiment shown in FIG. 10 , a method 1000 is disclosed that includes acquiring a first image of a tray at 1010. The method 1000 may include determining the presence or absence of one or more first patterns in the first image at 1020. The one or more first patterns may include text patterns, numeric patterns, symbol patterns, and combinations thereof. The method 1000 may include determining a rotation of each of the one or more first patterns in the first image at 1030. The method 1000 may include performing an action based on the presence or absence and rotation of the one or more first patterns in the first image at 1040. In one embodiment, each step of the method 1000 may be performed by a computing device, a camera (with processing capabilities), or a combination thereof. In some embodiments, multiple computing devices and / or cameras may be used to perform the method 1000. For example, multiple cameras may be used, with a first camera performing steps 1010, 1020, and 1030 while a second camera performs step 1040. In another embodiment, method 1000 can be repeated with each of several cameras and / or computing devices as the trays progress along the assembly line. For example, steps 1010, 1020, 1030, and 1040 can be performed by a first camera for a particular pattern, and then steps 1010, 1020, 1030, and 1040 can be performed again by a second camera for another particular pattern. Still further, one or more substeps described herein can be performed by designated cameras and / or computing devices.

[0048] Determining the presence or absence of the one or more first patterns in the first image can include determining the presence of one or two of the one or more first patterns, and determining the rotation of each of the one or more first patterns in the first image can include determining that one or two of the one or more first patterns are on a first axis. Performing an action based on the presence or absence and rotation of the one or more first patterns in the first image can include generating a pass test signal and advancing a belt on which the tray is disposed. Determining the presence or absence of the one or more first patterns in the first image can include determining the presence of three or more of the one or more first patterns. Performing an action based on the presence or absence and rotation of the one or more first patterns in the first image can include generating a fail test signal and notifying an operator that a first item associated with the one or more first patterns should be removed from the tray. Determining the presence or absence of the one or more first patterns in the first image can include determining the presence of two of the one or more first patterns, and determining the rotation of each of the one or more first patterns in the first image can include determining that two of the one or more first patterns are not on the same axis. Performing an action based on the presence or absence and rotation of the one or more first patterns in the first image can include generating a fail inspection signal and notifying an operator that the first article associated with the one or more first patterns should be removed from the tray.

[0049] The method 1000 may further include acquiring a second image of the tray, determining the presence or absence of one or more second patterns in the second image, determining a rotation of each of the one or more second patterns in the second image, and performing an action based on the presence or absence and rotation of the one or more second patterns in the second image. The one or more second patterns may include a text pattern, a numeric pattern, a symbol pattern, and combinations thereof. Determining the presence or absence of the one or more second patterns in the second image may include determining the presence of one or two of the one or more second patterns, and determining the rotation of each of the one or more second patterns in the second image may include determining that one or two of the one or more second patterns are on a second axis. Performing an action based on the presence or absence and rotation of the one or more second patterns in the second image may include generating a pass test signal and advancing a belt on which the tray is disposed. Determining the presence or absence of the one or more second patterns in the second image may include determining the presence of three or more of the one or more second patterns. Performing an action based on the presence, absence, and rotation of the one or more second patterns in the second image can include generating a fail inspection signal and notifying an operator that a second item associated with the one or more second patterns should be removed from the tray. Determining the presence or absence of the one or more second patterns in the second image can include determining the presence of two of the one or more second patterns, and determining the rotation of each of the one or more second patterns in the second image can include determining that two of the one or more second patterns are not on the same axis.Performing an action based on the presence, absence, and rotation of the one or more second patterns in the second image may include generating a fail inspection signal and notifying an operator that the second item associated with the one or more second patterns should be removed from the tray.

[0050] The method 1000 may further include determining a position of the cleat in the first image, comparing the determined position of the cleat in the first image with a reference position, determining that the determined position differs from the reference position, determining an offset based on the difference between the determined position and the reference position, and sending a signal to a belt controller to adjust the distance and advance the belt with the tray positioned thereon by the offset. The offset may be one of a negative value, a positive value, or a zero value. In one aspect, determining the offset based on the difference between the determined position and the reference position and sending a signal to the belt controller to adjust the distance and advance the belt with the tray positioned thereon by the offset can be performed by one or more cameras. For example, a single camera can be designated for determining the offset. The offset determination can be performed after each movement of the belt.

[0051] The method 1000 may further include repeatedly acquiring first images of the trays, determining the presence or absence of one or more first patterns in the first images, determining a rotation of each of the one or more first patterns in the first images, and performing an action based on the presence or absence and rotation of the one or more first patterns in the first images for each of the multiple trays.

[0052] Method 1000 may further include counting the number of the plurality of trays, with some empty tray positions interspersed among the plurality of trays. Method 1000 may further include counting the number of empty tray positions. Determining the presence or absence of one or more first patterns in the first image may include determining a no-tray pattern. Performing an action based on the presence or absence and rotation of one or more second patterns in the first image may include advancing a belt on which the no-tray pattern is disposed.

[0053] In an exemplary embodiment, the present methods and systems may be implemented on a computer 1101 as shown in FIG. 11 and described below. By way of example, camera 106, camera 107, camera 108, PLC 112, and / or pass / fail controller 113 (or components thereof) of FIG. 1 may be computer 1101 as shown in FIG. 11. Similarly, the disclosed methods and systems may utilize one or more computers to perform one or more functions at one or more locations. FIG. 2 is a block diagram illustrating an exemplary operating environment 1100 for implementing the disclosed methods. This exemplary operating environment 1100 is merely an example of an operating environment and is not intended to suggest any limitation as to the scope of use or functionality of the operating environment architecture. Neither should the operating environment 1100 be interpreted as having any dependency or requirement regarding any one or combination of components illustrated in the exemplary operating environment 1100.

[0054] The present method and system may be operational with numerous other general-purpose or special-purpose computing system environments or configurations. Examples of well-known computing systems, environments, and / or configurations that may be suitable for use with the present system and method include, but are not limited to, personal computers, server computers, laptop devices, and multiprocessor systems. Further examples include set-top boxes, programmable consumer electronics, network PCs, programmable logic controllers (PLCs), minicomputers, mainframe computers, distributed computing environments that include any of the above systems or devices, and the like.

[0055] The processing of the disclosed methods and systems can be performed by software components. The disclosed systems and methods can be described in the general context of computer-executable instructions, such as program modules, executed by one or more computers or other devices. Generally, program modules include computer code, routines, programs, objects, components, data structures, etc. that perform particular tasks or implement particular abstract data types. The disclosed methods can also be practiced in grid-based, distributed computing environments where tasks are performed by remote processing devices that are linked through a communications network. In a distributed computing environment, program modules can be located in local and / or remote computer storage media, including memory storage devices.

[0056] Additionally, those skilled in the art will appreciate that the systems and methods disclosed herein may be implemented via a general-purpose computing device in the form of a computer 1101. The computer 1101 may include one or more components, such as one or more processors 1103, a system memory 1112, and a bus 1113 that couples various components of the computer 1101, including the one or more processors 1103, to the system memory 1112. With multiple processors 1103, the system may utilize parallel computing.

[0057] Bus 1113 may include one or more of several possible types of bus structures, such as a memory bus, a memory controller, a peripheral bus, an accelerated graphics port, and a processor or local bus using any of a variety of bus architectures. Bus 1113, and all buses specified in this description, may also be implemented over wired or wireless network connections.

[0058] The computer 1101 typically includes a variety of computer-readable media. Exemplary readable media may be any available media accessible by the computer 1101, and are intended to include, by way of example and not limitation, both volatile and nonvolatile media, removable and non-removable media. The system memory 1112 may include computer-readable media in the form of volatile memory, such as random access memory (RAM), and / or non-volatile memory, such as read-only memory (ROM). The system memory 1112 typically includes data, such as image analysis data 1107, and / or program modules, such as an operating system 1105 and image analysis software 1106, that are accessible to and / or operated by the one or more processors 1103.

[0059] In another aspect, computer 1101 may also include other removable / non-removable, volatile / non-volatile computer storage media. Mass storage device 1104 may provide non-volatile storage of computer code, computer-readable instructions, data structures, program modules, and other data for computer 1101. For example, mass storage device 1104 may be a hard disk, a removable magnetic disk, a removable optical disk, a magnetic cassette or other magnetic storage device, a flash memory card, a CD-ROM, a digital versatile disk (DVD) or other optical storage device, random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), etc.

[0060] Any number of program modules may be stored on the mass storage device 1104, including, by way of example, an operating system 1105 and image analysis software 1106. One or more (or some combination) of the operating system 1105 and the image analysis software 1106 may include programming elements and the image analysis software 1106. Image analysis data 1107 may also be stored on the mass storage device 1104. The image analysis data 1107 may be stored in any one or more databases known in the art. Examples of such databases include DB2®, Microsoft® Access, Microsoft® SQL Server, Oracle®, mySQL, PostgreSQL, etc. The database may be centralized or distributed across multiple locations within the network 1115.

[0061] In another aspect, a user can enter commands and information into computer 1101 through input devices (not shown). Examples of such input devices include, but are not limited to, keyboards, pointing devices (e.g., computer mice, remote controls), microphones, joysticks, scanners, touch-enabled devices such as touch screens, tactile input devices such as gloves and other body coverings, and motion sensors. These and other input devices can be connected to one or more processors 1103 through a human-machine interface 1102 coupled to bus 1113, but may also be connected by other interface and bus structures, such as, but not limited to, a parallel port, a game port, an IEEE 1394 port (also known as a Firewire port), a serial port, a network adapter 1108, and / or a universal serial bus (USB).

[0062] In yet another aspect, a display device 1111 can also be connected to the bus 1113 via an interface, such as a display adapter 1109. It is contemplated that the computer 1101 can have multiple display adapters 1109, and that the computer 1101 can have multiple display devices 1111. For example, the display device 1111 can be a monitor, an LCD (liquid crystal display), a light-emitting diode (LED) display, a television, a smart lens, smart glass, and / or a projector. In addition to the display device 1111, other output peripheral devices can include components such as speakers (not shown) and a printer (not shown), which can be connected to the computer 1101 via the input / output interface 1110. The steps and / or results of any of these methods can be output to an output device in any form. Such output can be any form of visual representation, including, but not limited to, text, graphics, animation, audio, tactile, etc. The display 1111 and the computer 1101 can be part of a single device or separate devices.

[0063] In one aspect, computer 1101 can be coupled to system 100 via input / output interface 1110. Computer 1101 can be configured to monitor and store data. Computer 1101 can be configured to store images captured by cameras connected to system 100, store data related to pass / fail statistics generated during inspections occurring in the system, etc. Computer 1101 can also be used as a programming interface to one or more smart devices (e.g., smart cameras) and / or embedded logic controllers that require customized firmware to operate. Computer 1101 can be used to generate, troubleshoot, upload, and store iterations of this software or firmware.

[0064] The computer 1101 can operate in a networked environment using logical connections to one or more remote computing devices 1114a, b, c. By way of example, the remote computing devices 1114a, b, c can be personal computers, computing stations (e.g., workstations), portable computers (e.g., laptops, mobile phones, tablet devices), smart devices (e.g., smartphones, smart watches, activity trackers, smart apparel, smart accessories), security and / or monitoring devices, servers, routers, network computers, peer devices, edge devices or other common network nodes, etc. The logical connections between the computer 1101 and the remote computing devices 1114a, b, c can be through a network 1115, such as a local area network (LAN) and / or a general wide area network (WAN). Such network connections may be through a network adapter 1108. The network adapter 1108 can be implemented in both wired and wireless environments. Such network environments are commonplace and common in homes, offices, enterprise-wide computer networks, intranets, and the Internet. In one aspect, the network adapter 1108 can be configured to provide power to one or more connected devices (e.g., a camera). For example, the network adapter 1108 can comply with the Power over Ethernet (PoE) standard, or the like.

[0065] For purposes of illustration, application programs and other executable program components, such as the operating system 1105, are illustrated herein as separate blocks, with the understanding that such programs and components may reside at various times in different storage components of the computing device 1101 and be executed by one or more processors 1103 of the computer 1101. An implementation of the image analysis software 1106 may be stored on or transmitted across some form of computer-readable media. Any of the disclosed methods may be performed by computer-readable instructions embodied on a computer-readable medium. A computer-readable medium may be any available medium that can be accessed by a computer. By way of example and not limitation, computer-readable media may include “computer storage media” and “communications media.” “Computer storage media” may include volatile and nonvolatile, removable and non-removable media implemented in any method or technology for storage of information, such as computer-readable instructions, data structures, program modules, or other data. Exemplary computer storage media may include RAM, ROM, EEPROM, flash memory or other memory technology, CD-ROM, digital versatile disk (DVD) or other optical storage, magnetic cassettes, magnetic tape, magnetic disk storage or other magnetic storage devices, or any other medium usable to store the desired information and accessible by a computer.

[0066] The methods and systems may employ artificial intelligence (AI) techniques such as machine learning and iterative learning, including, but not limited to, expert systems, case-based reasoning, Bayesian networks, behavior-based AI, neural networks, fuzzy systems, evolutionary computing methods (e.g., genetic algorithms), swarm intelligence (e.g., ant algorithms), and hybrid intelligent systems (e.g., expert inference rules generated through neural networks or production rules from statistical learning).

[0067] The disclosed method and system have been implemented and tested, and the results compared to a standard, manual-only, operator-driven assembly line process. The table below shows that the disclosed method and system outperforms the standard, manual-only, operator-driven assembly line process.

[0068] [Table 1] While the present methods and systems have been described in terms of preferred embodiments and specific examples, the scope is not intended to be limited to the particular embodiments described, as the embodiments herein are intended in all respects to be illustrative and not restrictive.

[0069] Unless otherwise expressly stated, any method described herein is in no way intended to be construed as requiring that its steps be performed in a particular order. Accordingly, unless a method claim actually recites the order in which its steps should be followed or it is otherwise specifically stated in the claims or description that the steps are to be limited to a particular order, no order is intended to be implied in any way. This holds true for any possible implicit basis for interpretation, including matters of logic regarding the sequence or operational flow of steps, simple meaning derived from grammatical construction or punctuation, or the number or type of embodiments described in the specification.

[0070] It will be apparent to those skilled in the art that various modifications and variations can be made without departing from the scope or spirit of the invention. Other embodiments will be apparent to those skilled in the art from consideration of the specification and practice disclosed herein. It is intended that the specification and examples be considered as exemplary only, with a true scope and spirit of the invention being indicated by the following claims.

Claims

1. 1. A system comprising: Belt and a plurality of imagers, Acquire images within each field of view analyzing the images to determine the number and / or orientation of one or more articles within each field of view, the one or more articles including two or more occurrences of the first pattern; determining that two or more occurrences of the first pattern are aligned on the same axis within each field of view; determining a number and / or orientation of one or more articles in each field of view based at least in part on determining that two or more occurrences of the first pattern are aligned on the same axis in each field of view; a plurality of imagers configured to generate a pass inspection signal based on determining the number and / or orientation of one or more articles within each field of view; a processor coupled to each of the plurality of imagers, receiving a pass test signal from each of the plurality of imagers; a processor configured to advance the belt based on receiving a pass inspection signal from each of the plurality of imagers; A system comprising:

2. 10. The system of claim 1, The plurality of imagers further comprises: determining that two or more occurrences of the first pattern are not aligned on the same axis within each field of view; configured to generate a fail test signal based on two or more occurrences of the first pattern not being aligned on the same axis within each field of view; a processor coupled to each of the plurality of imagers; receiving a fail test signal; The system is configured to prevent movement of the belt based on receiving a failing inspection signal from one or more of the plurality of imagers.

3. 10. The system of claim 1, The system, wherein the first pattern includes a text pattern, a number pattern, a symbol pattern, and combinations thereof.

4. 10. The system of claim 1, At least one of the plurality of imagers further comprises: Determine the location of the cleats within each field of view; comparing the determined position of the cleat in each field of view to a reference position; determining that the determined position differs from the reference position; determining an offset based on a difference between the determined position and the reference position; The system is configured to send a signal to the processor to adjust the distance and advance the belt by the offset.

5. 10. The system of claim 1, the belt includes a plurality of spaced apart cleats; the length of each cleat is substantially perpendicular to the length of the belt; one or more fields of view are located between adjacent cleats of the plurality of cleats; At least one of the plurality of imagers further comprises: Determine the location of the cleats within each field of view; comparing the determined position of the cleat in each field of view to a reference position; determining that the determined position differs from the reference position; determining an offset based on a difference between the determined position and the reference position; The system is configured to send a signal to the processor to adjust the distance and advance the belt by the offset.

6. 10. The system of claim 1, The system, wherein at least one of the plurality of imagers is further configured to count the number of empty tray positions.

7. 10. The system of claim 1, At least one of the plurality of imagers is further configured to determine that one or more first patterns in the image include a no-tray pattern.

8. 10. The system of claim 1, The processor advancing the belt only after receiving a pass inspection signal from each of the plurality of imagers; The system is configured to prevent the belt from advancing until another pass test signal is received.

9. 10. The system of claim 1, The processor further comprises: advancing the belt only after receiving a pass inspection signal from each of the plurality of imagers; Do not advance the belt until another pass signal is received. The system is configured to notify an operator if the belt does not advance.

10. 10. The system of claim 1, The system, wherein the one or more articles include one or more syringes, syringe needles, or autoinjectors.

11. 1. A method comprising: acquiring a first image of the tray with each of the plurality of imagers; For each of the plurality of imagers, analyzing the images to determine a number and / or orientation of one or more articles within each field of view, the one or more articles including two or more occurrences of the first pattern; determining, for each of the plurality of imagers, that two or more occurrences of the first pattern are aligned on the same axis within each field of view; for each of the plurality of imagers, determining a number and / or orientation of one or more articles within each field of view based at least in part on determining that two or more occurrences of the first pattern are aligned on the same axis within each field of view; generating a pass inspection signal for each of the plurality of imagers based on determining the number and / or orientation of the one or more articles within each field of view; The method advances a belt containing the tray based on receiving a pass inspection signal from each of a plurality of imagers.

12. The method of claim 11 further comprises: determining, for one or more of the plurality of imagers, that two or more occurrences of the first pattern are not aligned on the same axis within each field of view; generating, for one or more of the plurality of imagers, a fail inspection signal based on two or more occurrences of the first pattern not being aligned on the same axis within each field of view; receiving a failing inspection signal from one or more of the plurality of imagers; The method includes not moving the belt based on receiving a failing inspection signal from one or more of the plurality of imagers.

13. 12. The method of claim 11, The method, wherein the first pattern includes a text pattern, a number pattern, a symbol pattern, and combinations thereof.

14. 12. The method of claim 11, The first pattern includes a text pattern, a number pattern, a symbol pattern, and combinations thereof; The method further comprises: determining, for each of the plurality of imagers, that two or more occurrences of the text pattern, the number pattern, the symbol pattern, and combinations thereof are aligned on the same axis; A method for generating a pass test signal based on determining that two or more occurrences of a text pattern, a number pattern, a symbol pattern, and combinations thereof are aligned on the same axis.

15. 12. The method of claim 11, A method wherein two or more occurrences of the first pattern are positioned about one or more syringes, syringe needles, or autoinjectors.

16. 12. The method of claim 11, Capturing a first image of the tray includes: determining, for each of the plurality of imagers, a location of a cleat within each field of view; comparing, for each of the plurality of imagers, the determined location of the cleat within each field of view to a reference location; determining, for each of a plurality of imagers, that the determined position differs from the reference position; determining an offset for each of the plurality of imagers based on a difference between the determined position and the reference position; For each of a plurality of imagers, a signal is sent to a processor to adjust the distance and advance the belt by the offset.

17. 12. The method of claim 11, determining, for each of the plurality of imagers, a location of a cleat within each field of view from a plurality of spaced apart cleats, the length of each cleat being substantially perpendicular to the length of the belt, and the field of view being located between adjacent cleats of the plurality of cleats; comparing, for each of the plurality of imagers, the determined location of the cleat within each field of view to a reference location; determining, for each of a plurality of imagers, that the determined position differs from the reference position; determining an offset for each of the plurality of imagers based on a difference between the determined position and the reference position; For each of a plurality of imagers, a signal is sent to a processor to adjust the distance and advance the belt by the offset.

18. The method of claim 11 further comprises: The method counts the number of empty tray positions for each of the plurality of imagers after acquiring a first image of the tray.

19. 12. The method of claim 11, Determining the number and / or orientation of the one or more articles includes: The method determines, for each of a plurality of imagers, that two or more occurrences of the first pattern include a no tray pattern.

20. 12. The method of claim 11, Determining the number and / or orientation of the one or more articles includes: The method includes determining the number and / or orientation of one or more articles while the belt is stationary.

Images