Assembly line incorporating electronic visual inspection

Abstract

To provide a new technique for product assembly that can reduce errors and increase the efficiency of package assembly.SOLUTION: A plurality of imagers acquires images in visual fields, analyzes the images to determine the number and / or orientation of one or more articles in each visual field, the one or more articles including two or more appearances of a first pattern, determines that the two or more appearances of the first pattern are aligned on the same axis in each visual field, determines the number and / or orientation of one or more articles in each visual field, on the basis of the determination that the two or more appearances of the first pattern are aligned on the same axis in each visual field, and generates acceptance inspection signals on the basis of the determination of the number and / or orientation of one or more articles in each visual field.SELECTED DRAWING: Figure 1

Description

Technical Field

[0001] The present invention relates to an assembly line incorporating electronic visual inspection.

Background Art

[0002] The assembly of products containing multiple articles, such as pharmaceutical packaging, is a complex task. Assembly can be carried out in one or more stages using the articles to be placed within the product at each stage. Errors can occur at each stage by misplacing the correct articles in the product, overloading the product with too many of the correct articles, and / or inserting incorrect articles into the product. Products shipped in a state where an error has ultimately occurred result in loss of revenue, increased customer complaints, and loss of time in dealing with customer complaints. In the case of pharmaceutical packages, one unintended result of improper packaging is that clinicians or patients may be reluctant to use the pharmaceuticals contained within the improperly assembled package. This can be particularly true for pharmaceuticals administered parenterally, for example subcutaneously, intramuscularly, intravenously, intravitreally, or by inhalation. Even if an improperly assembled package is returned by a clinician or patient to the manufacturer, regulatory agencies such as the US Food and Drug Administration do not permit repackaging of pharmaceuticals, resulting in a Notice of Event (NOE). Such NOEs trigger investigations, increase costs, and potentially result in loss of competitiveness.

Summary of the Invention

Problems to be Solved by the Invention

[0003] Therefore, it would be desirable to develop new technologies for product assembly that overcome these and other limitations of the prior art, reduce errors, and enhance it by increasing the efficiency of package assembly.

Means for Solving the Problems

[0004] The following general description and the embodiments for carrying out the invention are both illustrative and descriptive, and should be understood as not limiting. A method and system are disclosed for acquiring a first image of a tray, determining the presence or absence of one or more first patterns in the first image, determining the rotation of each of the one or more first patterns in the first image, and performing an action based on the presence or absence and rotation of the one or more first patterns in the first image.

[0005] Further advantages are partially described below or can be acquired through practice. These advantages will be realized and achieved by the elements and combinations specifically indicated in the attached claims. [Brief explanation of the drawing]

[0006] The accompanying drawings incorporated herein and constituting part of this specification illustrate embodiments and, together with their descriptions, serve to explain the principles of the Method and System. [Figure 1] An exemplary system. [Figure 2] An illustrative image of an object. [Figure 3A] An illustrative image of an object. [Figure 3B] An illustrative image of an object. [Figure 4A] An illustrative image of an object. [Figure 4B] An illustrative image of an object. [Figure 5A] An illustrative image of an object. [Figure 5B] An illustrative image of an object. [Figure 6A] An illustrative image of an object. [Figure 6B] An illustrative image of an object. [Figure 7A] An illustrative image of an object. [Figure 7B] An illustrative image of an object. [Figure 8A] An illustrative image of an object. [Figure 8B]An illustrative image of an object. [Figure 9] An exemplary embodiment of an exemplary system. [Figure 10] A flowchart illustrating an exemplary method. [Figure 11] Exemplary operating environment. [Modes for carrying out the invention]

[0007] Before the disclosure and description of this method and system, it should be understood that this method and system is not limited to any particular method, component, or embodiment. Furthermore, it should be understood that the terms used herein are for the purpose of describing a particular embodiment and are not intended to limit it.

[0008] Furthermore, when used herein and in the appended claims, the singular forms “a,” “an,” and “the” encompass multiple references unless otherwise explicitly indicated in the context. Ranges may be expressed herein as a range from “approximately” a certain value to and / or “approximately” another specific value. When such ranges are expressed, other embodiments include that certain value to and / or other specific values. Similarly, when values ​​are expressed as approximate values ​​using the preceding phrase “approximately,” it is understood that a certain value forms another embodiment. It will also be understood that the endpoint of each range has a meaning related to and independent of the endpoint of the other.

[0009] "Optional" or "optionally" means that the events or circumstances described below may or may not occur, and that the description includes both cases in which such events or circumstances occur and cases in which they do not occur.

[0010] Throughout this description and claims, the terms “comprise” and variations such as “comprising” and “comprises” mean “including, but not limited to,” and are not intended to exclude, for example, other components, integers, or steps. “Exemplary” means “an example of,” and is not intended to convey any suggestion of a preferred or ideal embodiment. “Etc.” is used for descriptive purposes only, not limiting purposes.

[0011] Components that can be used to carry out the disclosed methods and systems are disclosed herein. These and other components are disclosed herein, and where combinations, subsets, interactions, groups, etc., of these components are disclosed, specific references to each of the various individual, collective, combinations and substitutions thereof are not necessarily expressly disclosed, but each is intended and described herein for all methods and systems. This applies to all aspects of this application, including but not limited to steps in the disclosed methods. Therefore, where there are various additional steps that can be carried out, each of these additional steps may be carried out using any particular embodiment or combination of embodiments of the disclosed methods.

[0012] This method and system can be more readily understood by referring to the following detailed description of preferred embodiments, the examples and drawings contained therein, and the descriptions preceding and following them.

[0013] As will be understood by those skilled in the art, the method and system can take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware aspects. Further, the method and system can take the form of a computer program product in a computer-readable storage medium having computer-readable program instructions (e.g., computer software) embodied therein. More specifically, the method and system can take the form of computer software implemented on the web. Any suitable computer-readable storage medium including a hard disk, CD-ROM, optical storage device, or magnetic storage device can be utilized.

[0014] Embodiments of the method and system are described below with reference to block diagrams and flow chart diagrams of methods, systems, apparatuses, and computer program products. It will be understood that each block of the block diagrams and flow chart diagrams, as well as combinations of blocks in the block diagrams and flow chart diagrams, can be implemented by computer program instructions. These computer program instructions are loaded onto a general-purpose computer, a special-purpose computer, or other programmable data processing apparatus to create means for implementing the functions specified in one or more blocks of the flow chart by instructions executed by the computer or other programmable data processing apparatus, thereby manufacturing a machine.

[0015] These computer program instructions can also be stored in a computer-readable memory that directs a computer or other programmable data processing apparatus to function in a particular manner so as to produce an article of manufacture including computer-readable instructions for performing the functions specified in one or more blocks of the flowchart. The computer program instructions can also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus, and to generate a process that is executed by the computer to provide steps for performing the functions specified in one or more blocks of the flowchart.

[0016] Thus, it is understood that the blocks of the block diagrams and flowchart diagrams support combinations of means for performing the specified functions, combinations of steps for performing the specified functions, and program instruction means for performing the specified functions. It will also be understood that each block of the block diagrams and flowchart diagrams, and combinations of blocks in the block diagrams and flowchart diagrams, can be implemented by dedicated hardware-based computer systems that perform the specified functions or steps, or by combinations of dedicated hardware and computer instructions.

[0017] The present disclosure relates to improvements in computer functionality related to manufacturing and product assembly. FIG. 1 is a block diagram showing various aspects of an exemplary system 100 in which the present method and system can operate. Those skilled in the art will understand that functional descriptions are provided herein, and that each function can be performed by software, hardware, or a combination of software and hardware.

[0018] In one embodiment, the system 100 may include a conveyor belt 101. The conveyor belt 101 may include one or more anti-slip surfaces 102. One or more anti-slip surfaces 102 may be made of rubber or a similar material for attachment to the conveyor belt 101. One or more anti-slip surfaces 102 may be raised or otherwise extend on the surface of the conveyor belt 101. One or more anti-slip surfaces 102 may include front anti-slip surfaces and rear anti-slip surfaces based on the direction of movement 103. The front anti-slip surfaces and rear anti-slip surfaces may be relative to an object placed on the belt such that the front anti-slip surface is in front of the object with respect to the direction of movement 103, and the rear anti-slip surface is behind the object with respect to the direction of movement 103. Thus, a front anti-slip surface for a first object may be a rear anti-slip surface for a second object in front of the first object, and so on. One or more objects 104 may be placed on the conveyor belt 101. In one embodiment, one or more objects 104 may include one or more assembled products. For example, one or more objects 104 may include a tray. The tray may be configured to hold one or more articles. One or more articles may be related to a treatment. For example, one or more articles may include one or more syringes, auto-injectors, one or more syringe needles, one or more containers of pharmaceuticals, one or more sets of pamphlets or instructions, or a combination thereof.

[0019] In one embodiment, the instruction manual contains information on how to use and administer the drug. In another embodiment, the instruction manual is a drug label approved by a regulatory agency such as the U.S. Food and Drug Administration.

[0020] In one embodiment, the drug is a solid dosage form. In another embodiment, the drug is a liquid dosage form. In yet another embodiment, the drug is a gel dosage form. In one embodiment, the drug is formulated for oral administration. In another embodiment, the drug is formulated for parenteral administration. In another embodiment, the drug is formulated for subcutaneous administration. In another embodiment, the drug is formulated for intramuscular administration. In another embodiment, the drug is formulated for intravenous administration. In another embodiment, the drug is formulated for inhalation administration. In another embodiment, the drug is formulated for intraocular administration.

[0021] In one embodiment, the drug comprises a small molecule active ingredient. In another embodiment, the drug comprises a biological preparation. In yet another embodiment, the drug comprises a peptide or polypeptide active ingredient. In one embodiment, the drug comprises an active ingredient of vascular endothelial growth factor (VEGF). In another embodiment, the drug comprises aflibercept as described in one or more of U.S. Patent Nos. 7,070,959; 7,303,746; 7,303,747; 7,306,799; 7,374,757; 7,374,758; 7,531,173; 7,608,261; 7,972,598; 8,029,791; 8,092,803; 8,343,737; and 8,647,842 (each of which is incorporated by reference in its entirety).

[0022] The conveyor belt 101 can pass over a drive roll which can be driven by a stepping motor 105. The use of the stepping motor 105 enables the precise positioning of one or more objects 104 relative to cameras 106, 107, and 108. The length of each of the one or more objects 104 can be expressed as the exact number of motor steps. The conveyor belt 101 can move precisely forward or backward to move each of the one or more objects 104 into the fields of view 109, 110, and 111 associated with cameras 106, 107, and 108. A programmable logic controller (PLC) 112 (PLC 112 may comprise a computing device, PLC, or other controller / processor) can be configured to cause the stepping motor 105 to perform any number of steps in either direction to move one or more objects 104 into the fields of view 109, 110, and 111.

[0023] In one embodiment, cameras 106, 107, and / or 108 may be configured to scan, decode, read, detect, image, capture, and / or interpret visual codes. In some embodiments, cameras 106, 107, and / or 108 may be configured to process laser, linear, or area imaging. For example, in one embodiment, cameras 106, 107, and / or 108 may be imagers for scanning, reading, and decoding one-dimensional or two-dimensional barcodes. Cameras 106, 107, and / or 108 may be any imagers, barcode scanners, or visual code scanners capable of extracting information from visual codes consistent with the disclosed embodiments. In certain embodiments, cameras 106, 107, and / or 108 may be configured to process scanned barcodes, images, and other data. Cameras 106, 107, and / or 108 may include one or more depth cameras for capturing, processing, detecting, observing, modeling, detecting, and interacting with a three-dimensional environment. In certain embodiments, cameras 106, 107, and / or 108 may recognize and detect the depth and color of objects within fields 109, 110, and 111, respectively. Cameras 106, 107, and / or 108 may also provide other camera and video recorder functions such as taking photographs, recording video, streaming images or other data, and storing data in an image buffer. These functions may or may not include depth information. In relation to hardware and / or software processes consistent with the disclosed embodiments, cameras 106, 107, and / or 108 may determine the size, orientation, and visual characteristics of one or more objects 104. Cameras 106, 107, and / or 108 may include or embody any cameras known to those skilled in the art that are capable of performing the processes disclosed herein.Cameras 106, 107, and / or 108 may include appropriate hardware and software components (e.g., circuits, software instructions, etc.) for transmitting signals and information to and from the pass / fail controller 113 to perform a process consistent with the disclosed embodiments. The pass / fail controller 113 may include a computing device, a PLC, or other controller / processor. Cameras 106, 107, and / or 108 may output images and / or one or more notifications to monitors 114, 115, and 116, respectively.

[0024] The positioning of one or more objects 104 within fields 109, 110, and 111 can be performed when the system 100 is started and can be adjusted during use of the system 100. One or more of cameras 106, 107, and / or 108 may be used to ensure proper positioning of the conveyor belt 101. For example, camera 107 may be configured to produce an image of an area within field 110. Camera 107 can determine the position of one or more anti-slip devices 102 in the image. In one embodiment, camera 107 can determine the position of the front anti-slip device. Camera 107 can compare the determined position of one or more anti-slip devices 102 in the image to a reference position. If the determined position is equal to the reference position, the conveyor belt 101 does not need to be adjusted. If the determined position is not equal to the reference position, camera 107 can determine an offset based on the difference between the determined position and the reference position. The offset can be determined in terms of distance, such as millimeters, centimeters, inches, etc., and / or the offset can be determined as several steps. Camera 107 can send a signal to PLC 112 to move the conveyor belt 101 forward or backward by the amount of the offset by operating the stepping motor 105.

[0025] During operation, the system 100 can be configured to evaluate the current assembly state of one or more objects 104 and take one or more actions based on the current assembly state. As each of the one or more objects 104 moves forward on the conveyor belt 101, each of the one or more objects 104 is positioned within the fields of view 109, 110, and 111 of camera 106, camera 107, and / or camera 108, respectively. Although Figure 1 illustrates only three cameras, it is particularly conceivable that fewer than three or four or more cameras could be used. It is further conceivable that the conveyor belt 101 can be configured to have four or more of the illustrated objects 104 positioned on it, regardless of the number of cameras. As the one or more objects 104 move along the conveyor belt 101, one or more articles can be assembled into one or more objects 104 by a human operator or robot.

[0026] When one or more objects 104 are each within the field of view of one of the cameras, the camera can generate an image of the objects 104 within the field of view related to that camera. For example, camera 106 can generate an image of a region within field of view 109, camera 107 can generate an image of a region within field of view 110, and camera 108 can generate an image of a region within field of view 111. Cameras 106, 107, and / or camera 108 can each analyze their corresponding images. Image analysis may include determining the presence or absence of one or more patterns. One or more patterns may include text patterns, numeric patterns, symbol patterns, and combinations thereof. For example, a text pattern may include any string of characters such as "filter needle". A numeric pattern may include any sequence of numbers such as "6941518". A symbol pattern may include any sequence of symbols such as "●□□◆". In one embodiment, cameras 106, 107, and / or camera 108 may utilize optical character recognition (OCR) to "read" one or more patterns. In another embodiment, cameras 106, 107, and / or 108 may be configured not to utilize OCR, but rather to simply recognize one or more patterns as a specific pattern.

[0027] In one embodiment, one or more patterns can be embodied on one or more articles that are assembled into one or more objects 104. In one embodiment, at least a portion of one or more articles may include one or more associated patterns. Therefore, when cameras 106, 107, and / or 108 determine the presence of one or more patterns, the presence of one or more patterns indicates the presence of articles associated with a particular pattern. For example, if camera 106 determines the presence of a "filter needle" in an image of an area within the field of view 109, it can be concluded that an article associated with the "filter needle" pattern exists in object 104 within the field of view 109. Cameras 106, 107, and / or 108 can be configured to determine the presence or absence of multiple patterns in a single image. For example, camera 106 can determine the presence of "filter needle" and "filter needle" in an image of an area within the field of view 109. The presence of both patterns can indicate that an article associated with the twice-appearing "filter needle" pattern exists in object 104 within the field of view 109.

[0028] Each article that can be assembled into one or more objects 104 can be associated with one or more patterns indicating the presence or absence of a specific number of articles. For example, an article may be materialized by a particular pattern appearing only once. If cameras 106, 107, and / or 108 determine that a particular pattern appears only once, it can be concluded that there is only one article. However, if cameras 106, 107, and / or 108 determine that a particular pattern appears two or more times, it can be concluded that there are multiple articles. In another embodiment, an article may be materialized by a particular pattern appearing twice. If cameras 106, 107, and / or 108 determine that a particular pattern appears only twice, it can be concluded that there is only one article. However, if cameras 106, 107, and / or 108 determine that a particular pattern appears one or three or more times, it can be concluded that there are multiple articles. In further embodiments, an article may be materialized in conjunction with a specific pattern within a range. For example, an article may be materialized by the appearance of a specific pattern one or two times. If cameras 106, 107, and / or 108 determine that the specific pattern occurs once or twice, it can be concluded that there is only one article. However, if cameras 106, 107, and / or 108 determine that the specific pattern occurs three or more times, it can be concluded that there are multiple articles.

[0029] Cameras 106, 107, and / or 108 can each be configured to analyze the entire image or one or more specific regions of the image. Figure 2 shows an exemplary image 200 of object 104. Object 104 may include a tray 201 configured to hold one or more articles. One or more articles can be assembled into the tray 201 such that at least a portion of the articles are present in one or more specific regions. The tray 201 may include one or more regions, for example, region 202, region 203, and region 204. Regions 202, 203, and 204 can each be associated with regions where one or more patterns should be present when articles are present in the tray 201. For example, region 202 can be associated with the position of the vial cap of the vial when assembled on tray 201, region 203 can be associated with the position of one or more syringes and / or one or more needles when assembled on tray 201, and region 204 can be associated with the position of one or more pamphlets when assembled on tray 201. Cameras 106, 107, and / or 108 can each be configured to analyze one or more assigned regions of image 200. For example, camera 106 may be assigned to analyze regions 202 and 203, camera 107 may be assigned to analyze region 203, and camera 108 may be assigned to analyze regions 203 and 204. Any combination of assigned regions is possible. Furthermore, cameras 106, 107, and / or 108 can each be configured to determine the presence or absence of one or more assigned patterns in the assigned regions.For example, camera 106 may be assigned to determine the presence or absence of vial caps in region 202 and the presence or absence of a first pattern (including the number of occurrences of the first pattern) in region 203; camera 107 may be assigned to determine the presence or absence of a second pattern (including the number of occurrences of the second pattern) in region 203; and camera 108 may be assigned to determine the presence or absence of a third pattern (including the number of occurrences of the third pattern) in region 203 and the presence or absence of a fourth pattern (including the number of occurrences of the fourth pattern) in region 204. Any combination of assigned patterns and assigned regions is possible.

[0030] Returning to Figure 1, each of the one or more objects 104 can be configured to contain a specific number of each of the one or more articles. The presence of a specific number of each article indicates that the one or more objects 104 are correctly assembled. The presence of anything other than a specific number of each article indicates that the one or more objects 104 are incorrectly assembled. Cameras 106, 107, and / or 108 can each be configured to perform independent evaluations of the objects 104 in their respective fields of view. If the camera determines that a specific number of articles configured to be detected are present, the camera can send a PASS signal to the pass / fail controller 113. If the camera determines that a specific number of articles configured to be detected are not present, the camera can send a FAIL signal to the pass / fail controller 113. If cameras 106, 107, and / or 108 each send a PASS signal to the pass / fail controller 113, the pass / fail controller 113 may send a signal to the PLC 112 to move the conveyor belt 101 forward using the stepping motor 105 to move one or more objects 104 forward and place them under the field of view of the next camera. The pass / fail controller 113 may further send notifications to each of the monitors 114-116 to display the PASS notification. If one or more of cameras 106, 107, and / or 108 send a FAIL signal to the pass / fail controller 113, the pass / fail controller 113 does not send a signal to the PLC 112 to move the stepping motor 105 forward. The pass / fail controller 113 may further send notifications to the monitors 114-116 associated with the camera that sent the FAIL signal to display the FAIL notification. Operators (e.g., humans or robots) positioned on monitors 114-116 displaying FAIL notifications can perform corrective actions to improve the FAIL condition.For example, if a FAIL signal is issued as a result of a missing item, the operator can replace the missing item, and as a result, the camera that made the previous FAIL determination can regenerate and reanalyze the image, determine that the item is now present, and issue a PASS signal to the pass / fail controller 113. In another embodiment, if a FAIL signal is issued as a result of one or more extra items, the operator can remove the one or more extra items, and as a result, the camera that made the previous FAIL determination can regenerate and reanalyze the image, determine that the required number of items are now present, and issue a PASS signal to the pass / fail controller 113.

[0031] In a further embodiment, the analysis of images by cameras 106, 107, and / or 108 may include not only the determination of the presence or absence of one or more patterns, but also the determination of the rotation of multiple patterns. In one embodiment, multiple patterns may be embodied on one or more articles that are assembled into one or more objects 104 along a particular axis. In one embodiment, at least a portion of one or more articles may include multiple associated patterns along a particular axis. Therefore, if cameras 106, 107, and / or 108 determine the presence of multiple patterns along a particular axis, the presence of multiple patterns along a particular axis indicates the presence of articles associated with a particular pattern along that axis. For example, if camera 106 determines the presence of "filter needles" and "filter needles" along the same axis (e.g., 30°, 60°, 90°, 120°, 180°, etc.) in an image of a region within the field of view 109, it can be concluded that articles associated with the patterns "filter needles" and "filter needles" along the same axis exist in object 104 within the field of view 109. Cameras 106, 107, and / or 108 can be configured to determine the rotation of multiple patterns in a single image. For example, camera 106 can determine the presence of "filter needle" and "filter needle" along a first axis, and the presence of "syringe needle" and "syringe needle" along a second axis, in an image of a region within the field of view 109. The presence of both patterns along two different axes can indicate that an item associated with two occurrences of the "filter needle" pattern along the first axis exists in object 104, and an item associated with two occurrences of the "syringe needle" pattern along the second axis also exists in object 104. As a further example, camera 106 can determine the presence of "filter needle" and "filter needle" along a first axis, and the presence of "filter needle" along a second axis, in an image of a region within the field of view 109. The presence of both patterns along two different axes can indicate that an item associated with two occurrences of the "filter needle" pattern exists in object 104.

[0032] Each article that can be assembled into one or more objects 104 can be associated with one or more patterns that are embodied along a specific axis, indicating the presence or absence of a specific number of articles. For example, an article may be embodied by a specific pattern appearing twice along a specific axis. If cameras 106, 107, and / or 108 determine that a specific pattern appears only twice along a specific axis, it can be concluded that there is only one article. However, if cameras 106, 107, and / or 108 determine that a specific pattern appears along multiple axes, it can be concluded that there are multiple articles.

[0033] Figures 3A and 3B show exemplary images 300 and 303 of a tray 201 containing articles 301 and 302. For example, article 301 may be a vial and article 302 may be a filter needle. Any of the cameras 106, 107, and / or 108 generating image 300 can determine that a vial cap is present in region 202. The presence of a single vial cap indicates the presence of article 301. The cameras 106, 107, and / or 108 generating image 300 can determine that there are two occurrences of the pattern "text A" in region 203. In one embodiment, two occurrences of the pattern "text A" may indicate the presence of one or more instances of article 302, and cameras 106, 107, and / or 108 may generate a PASS or FAIL signal as needed. In another embodiment, depending on the pattern configuration on article 302 (for example, a single instance of article 302 may have either one occurrence of "text A" or two occurrences of "text A"), cameras 106, 107, and / or 108 can determine whether "text A" and "text A" appear on the same axis. If "text A" and "text A" appear on the same axis, cameras 106, 107, and / or 108 can determine that a single instance of article 302 exists, and cameras 106, 107, and / or 108 can generate a PASS signal or a FAIL signal as necessary. If "text A" and "text A" appear on different axes, cameras 106, 107, and / or 108 can determine that multiple instances of article 302 exist, and cameras 106, 107, and / or 108 can generate a PASS signal or a FAIL signal as necessary. In one embodiment, axis determination can be used to confirm the existence of any number of articles 302, and a PASS signal or FAIL signal can be generated based on the predicted number of instances of article 302 versus the determined number of instances of article 302.

[0034] Figure 4A shows an exemplary image 400 of tray 201 containing two instances of article 301 and article 302. Cameras 106, 107, and / or 108, which generate the image 400, can determine that there are three occurrences of the pattern ("text A") within region 203. In one embodiment, three occurrences of the pattern "text A" may indicate the presence of one or more instances of article 302, and cameras 106, 107, and / or 108 may generate a PASS or FAIL signal as needed. In another embodiment, depending on the pattern configuration on article 302 (for example, a single instance of article 302 may have one occurrence of "text A", two occurrences of "text A", or three occurrences of "text A"), cameras 106, 107, and / or 108 may determine whether the three occurrences of "text A" appear on the same axis. As shown in Figure 4A, two occurrences of "text A" lie on the same axis, while one occurrence of "text A" lies on a different axis. Therefore, cameras 106, 107, and / or 108 can determine that multiple instances of article 302 exist, and cameras 106, 107, and / or 108 can generate a PASS or FAIL signal as needed. In one embodiment, axis determination can be used to confirm the existence of any number of articles 302, and a PASS or FAIL signal can be generated based on the predicted number of instances of article 302 versus the determined number of instances of article 302.

[0035] Figure 4B shows an exemplary image 401 of tray 201 containing article 301, one instance of article 302, and one instance of article 402. In one embodiment, cameras 106, 107, and / or 108 generating the image 400 can determine that there are two occurrences of a first pattern ("text A") and one occurrence of a second pattern ("text B") within region 203. In one embodiment, two occurrences of the pattern "text A" may indicate the presence of one or more instances of article 302, and cameras 106, 107, and / or 108 may generate a PASS signal or a FAIL signal as needed. In another embodiment, depending on the pattern configuration on article 302 (for example, a single instance of article 302 may have one occurrence of "text A", two occurrences of "text A", or three occurrences of "text A"), cameras 106, 107, and / or 108 can determine whether two occurrences of "text A" appear on the same axis. As shown in Figure 4B, two occurrences of "text A" appear on the same axis. Thus, cameras 106, 107, and / or 108 can determine that multiple instances of article 302 exist, and cameras 106, 107, and / or 108 can generate a PASS signal or a FAIL signal as necessary. However, a single occurrence of the pattern "text B" may indicate that an article that should not be in tray 201 is placed in tray 201 at this stage of the assembly process. Thus, cameras 106, 107, and / or 108 can generate a FAIL signal based on the presence of a pattern that is not intended to be present.

[0036] In another embodiment, cameras 106, 107, and / or 108, which generate the image 400, can determine that pattern "text B" is present and ignore the presence of pattern "text A" (or any other as necessary). In one embodiment, a single occurrence of pattern "text B" may indicate the presence of one instance of article 302, and cameras 106, 107, and / or 108 may generate a PASS signal.

[0037] Figure 5A shows an exemplary image 500 of tray 201 containing articles 301, 302, and one instance of article 501. Cameras 106, 107, and / or 108 that generate image 500 can be configured to ignore vial caps in region 202 and ignore the presence of pattern “text A” in region 203. Alternatively, cameras 106, 107, and / or 108 that generate image 400 can determine that there are two occurrences of another pattern ("text B") in region 203. In one embodiment, two occurrences of pattern “text B” may indicate the presence of one or more instances of article 501, and cameras 106, 107, and / or 108 can generate a PASS or FAIL signal as needed. In another embodiment, depending on the pattern configuration on article 501 (for example, a single instance of article 501 may have one occurrence of "text B", two occurrences of "text B", or three occurrences of "text B"), cameras 106, 107, and / or 108 can determine whether two occurrences of "text B" appear on the same axis. As shown in Figure 5A, two occurrences of "text B" appear on the same axis. Thus, cameras 106, 107, and / or 108 can determine that one instance of article 501 exists, and cameras 106, 107, and / or 108 can generate a PASS signal or a FAIL signal as needed. In one embodiment, axis determination can be used to confirm the existence of any number of articles 501 and generate a PASS signal or a FAIL signal based on the predicted number of instances of article 501 versus the determined number of instances of article 501. Figure 5B shows an exemplary image 503 of tray 201 containing one instance of articles 301, 302, and 501. Figure 5B is similar to Figure 5A, except that Figure 5B shows that the pattern "text B" appears twice along the same axis, but at a different angle than the axis in Figure 5A.

[0038] Figure 6A shows an exemplary image 600 of tray 201 containing two instances of article 301 and article 501. Cameras 106, 107, and / or 108, which generate the image 600, can determine that there are four occurrences of the pattern "text B" within region 203. In one embodiment, four occurrences of the pattern "text B" may indicate the presence of one or more instances of article 501, and cameras 106, 107, and / or 108 may generate a PASS or FAIL signal as needed. In another embodiment, depending on the pattern configuration on article 501 (for example, a single instance of article 501 may have one occurrence of "text B", two occurrences of "text B", three occurrences of "text B", or four occurrences of "text B"), cameras 106, 107, and / or 108 may determine the axis on which the four occurrences of "text B" appear. As shown in Figure 5A, the two occurrences of "text B" appear on the first axis, and the other two occurrences of "text B" appear on the second axis. Therefore, since the two sets of "text B" appear on different axes, cameras 106, 107, and / or 108 can determine that there are multiple instances of article 501, and cameras 106, 107, and / or 108 can generate a PASS signal or a FAIL signal as needed. In one embodiment, axis determination can be used to confirm the existence of any number of articles 501 and generate a PASS signal or a FAIL signal based on the predicted number of instances of article 501 versus the determined number of instances of article 501.

[0039] Figure 6B shows an exemplary image 601 of tray 201 containing article 301, article 302, and two instances of article 501. Cameras 106, 107, and / or 108, which generate the image 601, can determine that there are three occurrences of the pattern "text B" within region 203. In one embodiment, three occurrences of the pattern "text B" may indicate the presence of one or more instances of article 501, and cameras 106, 107, and / or 108 may generate a PASS or FAIL signal as needed. In another embodiment, depending on the pattern configuration on article 501 (for example, a single instance of article 501 may have one occurrence of "text B", two occurrences of "text B", three occurrences of "text B", or four occurrences of "text B"), cameras 106, 107, and / or 108 may determine the axis on which the three occurrences of "text B" appear. As shown in Figure 6B, two occurrences of "text B" appear on the first axis, and one occurrence of "text B" appears on the second axis. Therefore, since two sets of "text B" appear on different axes, cameras 106, 107, and / or 108 can determine that multiple instances of article 501 exist, and cameras 106, 107, and / or 108 can generate a PASS signal or a FAIL signal as needed. In one embodiment, axis determination can be used to confirm the existence of any number of articles 501 and generate a PASS signal or a FAIL signal based on the predicted number of instances of article 501 versus the determined number of instances of article 501.

[0040] Figure 7A shows an exemplary image 700 of tray 201 containing article 301, article 302, article 501, one instance of article 701, and one instance of article 702. Cameras 106, 107, and / or 108 generating the image 700 can be configured to ignore vial caps in region 202 and ignore the presence of patterns “text A” and “text B” in region 203. Alternatively, cameras 106, 107, and / or 108 generating the image 700 can determine that there are two occurrences of another pattern (“text D”) in region 203. In one embodiment, two occurrences of pattern “text D” may indicate the presence of one or more instances of article 701, and cameras 106, 107, and / or 108 may generate a PASS or FAIL signal as needed. In another embodiment, depending on the pattern configuration on article 701 (for example, a single instance of article 701 may have one occurrence of "text D", two occurrences of "text D", or three occurrences of "text D"), cameras 106, 107, and / or 108 can determine whether two occurrences of "text D" appear on the same axis. As shown in Figure 7A, two occurrences of "text D" appear on the same axis. Thus, cameras 106, 107, and / or 108 can determine that one instance of article 701 exists, and cameras 106, 107, and / or 108 can generate a PASS signal or a FAIL signal as needed. In one embodiment, axis determination can be used to confirm the existence of any number of articles 701 and generate a PASS signal or a FAIL signal based on the predicted number of instances of article 701 versus the determined number of instances of article 701. In the same image 700, cameras 106, 107, and / or 108 can determine that there are two different occurrences of the pattern ("text C") within region 204.In one embodiment, two occurrences of the pattern "text C" can indicate the presence of one or more instances of article 702, and cameras 106, 107, and / or 108 can generate a PASS or FAIL signal as needed. In another embodiment, depending on the pattern configuration on article 702 (for example, a single instance of article 702 may have one occurrence of "text C", two occurrences of "text C", or three occurrences of "text C"), cameras 106, 107, and / or 108 can determine whether two occurrences of "text C" appear on the same axis. As shown in Figure 7A, two occurrences of "text C" appear on the same axis. Therefore, cameras 106, 107, and / or 108 can determine that one instance of article 702 exists, and cameras 106, 107, and / or 108 can generate a PASS or FAIL signal as needed. In one embodiment, axis determination can be used to confirm the presence of any number of articles 702 and to generate a PASS or FAIL signal based on the predicted number of instances of article 702 versus the determined number of instances of article 702. Figure 7B shows an exemplary image 703 of tray 201 containing article 301, article 302, article 501, one instance of article 701, and one instance of article 702. Figure 7B is similar to Figure 7A, except that the pattern "text D" appears twice along the same axis but at a different angle than the axis in Figure 7A, and similarly the pattern "text C" appears twice along the same axis but at a different angle than the axis in Figure 7A.

[0041] Figure 8A shows an exemplary image 800 of tray 201 containing article 301, article 302, article 501, two instances of article 701, and one instance of article 702. Cameras 106, 107, and / or 108, which generate the image 800, can determine that there are three occurrences of the pattern "text D" within region 203. In one embodiment, three occurrences of the pattern "text D" may indicate the presence of one or more instances of article 701, and cameras 106, 107, and / or 108 may generate a PASS or FAIL signal as needed. In another embodiment, depending on the pattern configuration on article 701 (for example, a single instance of article 701 may have one occurrence of "text D", two occurrences of "text D", three occurrences of "text D", or four occurrences of "text D"), cameras 106, 107, and / or 108 may determine the axis on which the three occurrences of "text D" appear. As shown in Figure 8A, two occurrences of "text D" appear on the first axis, and one occurrence of "text D" appears on the second axis. Therefore, since two sets of "text D" appear on different axes, cameras 106, 107, and / or 108 can determine that multiple instances of article 701 exist, and cameras 106, 107, and / or 108 can generate a PASS or FAIL signal as needed. In one embodiment, axis determination can be used to confirm the existence of any number of articles 701 and generate a PASS or FAIL signal based on the predicted number of instances of article 701 versus the determined number of instances of article 701. Figure 8B is similar to Figure 8A, except that in Figure 8B, the pattern "text D" appears twice along the first axis and once along the second axis, but the first and second axes are at different angles than the axes in Figure 8A.

[0042] Returning to Figure 1, cameras 106, 107, and 108 can each independently determine both the presence or absence of one or more patterns in the image and the rotation of each of the one or more patterns in the image of object 104. Cameras 106, 107, and 108 can each perform actions based on the presence and rotation of one or more patterns in the image. If the camera determines, based on the presence and rotation of the patterns, that the correct number of items are present in the image of object 104, the action may include sending a PASS signal to the pass / fail controller 113. If the camera determines, based on the presence and rotation of the patterns, that the wrong number of items are present in the image of object 104, the action may include sending a FAIL signal to the pass / fail controller 113. If cameras 106, 107, and / or 108 each send a PASS signal to the pass / fail controller 113, the pass / fail controller 113 may provide a signal to the PLC 112 to move the stepping motor 105 forward the conveyor belt 101 and move one or more objects 104 forward and place them under the field of view of the next camera. The pass / fail controller 113 may further send notifications to each of the monitors 114-116 to display the PASS notification. If one or more of cameras 106, 107, and / or 108 send a FAIL signal to the pass / fail controller 113, the pass / fail controller 113 does not provide a signal to the PLC 112 to move the stepping motor 105 forward. The pass / fail controller 113 may further send notifications to the monitors 114-116 associated with the camera that sent the FAIL signal to display the FAIL notification. Operators (e.g., humans or robots) positioned on monitors 114-116 displaying FAIL notifications can perform corrective actions to improve the FAIL condition.

[0043] In another embodiment, one or more of cameras 106, 107, and 108 can count the number of one or more objects 104. For example, when one or more objects 104 pass through one of cameras 106, 107, and 108, the camera can increment the tally of one or more objects 104 being imaged by the camera. In a further embodiment, there may be multiple empty spaces scattered among one or more objects 104. For example, in certain situations, one or more of cameras 106, 107, and 108 may not have any objects 104 in their corresponding field of view. The conveyor belt 101 may have a pattern embodied on it (e.g., a "no tray" pattern) at positions where objects 104 would otherwise be placed. Cameras 106, 107, and 108 can identify the pattern and issue a PASS signal to contribute to the forward movement of the conveyor belt 101.

[0044] Figure 9 shows an exemplary embodiment of system 100 illustrating the positioning of cameras 106, 107, and 108 relative to a conveyor belt 101. Figure 9 further illustrates the positioning of monitors 114-116. A stepping motor 105 is shown at one end of the conveyor belt 101. One or more PLCs 112 and / or pass / fail controllers 113 can be housed in the housing 901. One or more dispensers 902 can be configured to hold one or more articles accessed during assembly into one or more objects. System 100 may include one or more emergency stop ("E-Stop") buttons 903. The E-Stop buttons 903 can be activated at any time to temporarily halt the operation of system 100 for any reason. The E-Stop buttons 903 can be reset and system 100 can be restarted (for example, by an operator or technician who deems it safe to do so). System 100 may include one or more optoswitches 904. The optoswitch 904 can be activated ("tripped") by placing a finger or thumb on the saddle-like structure of the optoswitch 904. This action interrupts the optical signal path and triggers a switching condition. The optoswitch 904 can be used to accept visual inspection during "manual trigger" mode and to start / restart belt movement during "autonomous" (or "automatic") mode.

[0045] System 100 may include a key switch mechanism 905. The key switch mechanism 905 can be used to toggle between "autonomous" mode and "manual trigger" mode. Under normal operation, regardless of the mode, the first operator station may include an operator who places trays on the conveyor belt 101. In one embodiment, these trays may be pre-filled with capped vials. In manual trigger mode, at the second operator station, an operator may place filter needle tips on the tray. After this operation, camera 106 inspects the tray for the appropriate items. At the third operator station, injection needle tips may be added to the tray. Camera 107 then inspects the tray for the appropriate items. At the fourth operator station, an operator places empty blister-packed syringes on the tray. Then, a fifth operator places a Physician Insert (PI) on the tray. After the PI is placed, camera 108 inspects whether the tray has been completed. Once the trays have passed this final station, the full trays leave conveyor belt 101 for boxing.

[0046] In automatic mode, the tray automatically moves down the conveyor belt 101. The system 100 can maintain a dwell time (e.g., 1-5 seconds) before the conveyor belt 101 moves to the next position. This movement only occurs when all three inspection cameras (e.g., camera 106, camera 107, and camera 108) pass the tray being inspected by the corresponding camera ("pass"). If a problem occurs at any inspection station, the conveyor belt 101 may enter a "red light" state ("fail"), at which point the operator can resolve the problem or remove the tray from the conveyor belt 101 (each camera can advance the conveyor belt 101 if the tray is not in its field of view). Advancement of the conveyor belt 101 may depend on all cameras detecting a "pass" tray configuration. Display screens at each camera station (e.g., monitors 114-116) can display the video streams from the relevant cameras, superimposed with the "pass," "fail," or "no job" status depending on the inspection result. The camera's online status can be reset from monitors 114-116 as needed during operation.

[0047] In one embodiment shown in Figure 10, a method 1000 is disclosed, which includes, in 1010, acquiring a first image of a tray. Method 1000 may include, in 1020, determining the presence or absence of one or more first patterns in the first image. One or more first patterns may include text patterns, numeric patterns, symbol patterns, and combinations thereof. Method 1000 may include, in 1030, determining the rotation of each of the one or more first patterns in the first image. Method 1000 may include, in 1040, performing an action based on the presence and rotation of one or more first patterns in the first image. In one embodiment, each step of Method 1000 may be performed by a computing device, a camera (with processing capabilities), or a combination thereof. In some embodiments, multiple computing devices and / or cameras may be used to perform Method 1000. For example, multiple cameras may be used, with a first camera performing steps 1010, 1020, and 1030 while a second camera performs step 1040. In another embodiment, as the tray moves along the assembly line, method 1000 can be repeated with each of several cameras and / or computing devices. For example, steps 1010, 1020, 1030 and 1040 can be performed by a first camera for a particular pattern, and then steps 1010, 1020, 1030 and 1040 can be performed again by a second camera for another particular pattern. Furthermore, one or more substeps described herein can be performed by designated cameras and / or computing devices.

[0048] Determining the presence or absence of one or more first patterns in a first image may include determining the presence of one or two of the one or more first patterns, and determining the rotation of each of the one or more first patterns in the first image may include determining that one or two of the one or more first patterns are on a first axis. Performing an action based on the presence or absence and rotation of one or more first patterns in a first image may include generating a pass signal and advancing a belt on which a tray is placed. Determining the presence or absence of one or more first patterns in a first image may include determining the presence of three or more of the one or more first patterns. Performing an action based on the presence or absence and rotation of one or more first patterns in a first image may include generating a fail signal and notifying an operator that a first article associated with one or more first patterns should be removed from the tray. Determining the presence or absence of one or more first patterns in a first image may include determining the presence of two of the one or more first patterns, and determining the rotation of each of the one or more first patterns in the first image may include determining that two of the one or more first patterns are not on the same axis. Performing an action based on the presence and rotation of one or more first patterns in the first image may include generating a rejection signal and notifying the operator that a first article associated with one or more first patterns should be removed from the tray.

[0049] Method 1000 may further include acquiring a second image of the tray, determining the presence or absence of one or more second patterns in the second image, determining the rotation of each of the one or more second patterns in the second image, and performing an action based on the presence or absence and rotation of the one or more second patterns in the second image. The one or more second patterns may include text patterns, numeric patterns, symbol patterns, and combinations thereof. Determining the presence or absence of one or more second patterns in the second image may include determining the presence of one or two of the one or more second patterns, and determining the rotation of each of the one or more second patterns in the second image may include determining that one or two of the one or more second patterns are on a second axis. Performing an action based on the presence or absence and rotation of one or more second patterns in the second image may include generating a pass / fail signal and advancing a belt on which the tray is placed. Determining the presence or absence of one or more second patterns in the second image may include determining the presence of three or more of the one or more second patterns. Performing an action based on the presence and rotation of one or more second patterns in a second image may include generating a rejection signal and notifying the operator that a second item associated with one or more second patterns should be removed from the tray. Determining the presence or absence of one or more second patterns in a second image may include determining the presence of two of the one or more second patterns, and determining the rotation of each of the one or more second patterns in a second image may include determining that two of the one or more second patterns are not on the same axis.Performing an action based on the presence and rotation of one or more second patterns in a second image may include generating a rejection signal and notifying the operator that a second article associated with one or more second patterns should be removed from the tray.

[0050] Method 1000 may further include determining the position of the anti-slip surface in a first image, comparing the determined position of the anti-slip surface in the first image with a reference position, determining if the determined position differs from the reference position, determining an offset based on the difference between the determined position and the reference position, and sending a signal to a belt controller to adjust the distance and advance the belt on which the tray is placed by the offset. The offset may be a negative value, a positive value, or zero. In one embodiment, determining the offset based on the difference between the determined position and the reference position, sending a signal to a belt controller to adjust the distance, and advancing the belt on which the tray is placed by the offset can be performed by one or more cameras. For example, a single camera can be designated to determine the offset. The offset determination can be performed after each movement of the belt.

[0051] Method 1000 may further include acquiring a first image of a repeating tray, determining the presence or absence of one or more first patterns in the first image, determining the rotation of each of the one or more first patterns in the first image, and performing an action for each of the multiple trays based on the presence or absence and rotation of one or more first patterns in the first image.

[0052] Method 1000 may further include counting the number of trays, where some empty tray positions are scattered among the trays. Method 1000 may further include counting the number of empty tray positions. Determining the presence or absence of one or more first patterns in the first image may include determining the no-tray pattern. Performing an action based on the presence and rotation of one or more second patterns in the first image may include advancing the belt where the no-tray pattern is located.

[0053] In exemplary embodiments, the method and system can be implemented on a computer 1101 as shown in Figure 11 and described below. For example, the cameras 106, 107, 108, PLC 112, and / or pass / fail controller 113 (or their components) in Figure 1 can be replaced by the computer 1101 as shown in Figure 11. Similarly, the disclosed method and system can utilize one or more computers to perform one or more functions in one or more locations. Figure 2 is a block diagram of an exemplary operating environment 1100 for implementing the disclosed method. This exemplary operating environment 1100 is merely an example of an operating environment and is not intended to imply any limitations on the scope of use or functionality of the operating environment architecture. Nor should the operating environment 1100 be construed as having any dependencies or requirements on any one or combination of components shown in the exemplary operating environment 1100.

[0054] This method and system may operate with a number of other general-purpose or dedicated computing system environments or configurations. Examples of well-known computing systems, environments, and / or configurations suitable for use with this system and method include, but are not limited to, personal computers, server computers, laptop devices, and multiprocessor systems. Further examples include set-top boxes, programmable consumer electronics, network PCs, programmable logic controllers (PLCs), minicomputers, mainframe computers, and distributed computing environments that include any of the systems or devices described above.

[0055] The methods and systems disclosed can be executed by software components. The disclosed systems and methods can be described in the general context of computer executable instructions, such as program modules, executed by one or more computers or other devices. Generally, a program module includes computer code, routines, programs, objects, components, data structures, etc., that perform a specific task or implement a specific abstract data type. The disclosed methods can also be implemented in a grid-based distributed computing environment where tasks are executed by remote processing devices linked over a communication network. In a distributed computing environment, program modules may reside on local and / or remote computer storage media, including memory storage devices.

[0056] Furthermore, those skilled in the art will understand that the systems and methods disclosed herein can be implemented via a general-purpose computing device in the form of a computer 1101. The computer 1101 may include one or more processors 1103, a system memory 1112, and one or more components such as a bus 1113 that connects various components of the computer 1101, including the one or more processors 1103, to the system memory 1112. In the case of multiple processors 1103, the system can utilize parallel computing.

[0057] Bus 1113 may include one or more of several possible types of bus structures, such as a memory bus, memory controller, peripheral bus, accelerated graphics port, and a processor or local bus using any of the various bus architectures. Bus 1113 and all buses specified in this description may also be implemented via wired or wireless network connections.

[0058] Computer 1101 typically includes various computer-readable media. Illustrative computer-readable media can be any available media accessible by computer 1101, and are intended to include, but are not limited to, both volatile and non-volatile media, and both removable and non-removable media. System memory 1112 may include computer-readable media in the form of volatile memory such as random-access memory (RAM) and / or non-volatile memory such as read-only memory (ROM). System memory 1112 may typically include data such as image analysis data 1107, and / or program modules such as an operating system 1105 and image analysis software 1106 that are accessible to and / or run by one or more processors 1103.

[0059] In another embodiment, the computer 1101 may also include other removable / non-removable, volatile / non-volatile computer storage media. The mass storage device 1104 can provide a non-volatile storage device for computer code, computer-readable instructions, data structures, program modules, and other data for the computer 1101. For example, the mass storage device 1104 may be a hard disk, a removable magnetic disk, a removable optical disk, a magnetic cassette or other magnetic storage device, a flash memory card, a CD-ROM, a digital general-purpose disk (DVD) or other optical storage device, random access memory (RAM), read-only memory (ROM), electrically erasable and programmable read-only memory (EEPROM), etc.

[0060] If desired, any number of program modules, including, for example, an operating system 1105 and image analysis software 1106, can be stored in the mass storage device 1104. One or more (or a combination of) of the operating system 1105 and the image analysis software 1106 may include programming elements and the image analysis software 1106. Image analysis data 1107 can also be stored in the mass storage device 1104. Image analysis data 1107 can be stored in one or more databases known in the art. Examples of such databases include DB2®, Microsoft® Access, Microsoft® SQL Server, Oracle®, MySQL, PostgreSQL, etc. The database may be centralized or distributed across multiple locations within the network 1115.

[0061] In another embodiment, a user can input commands and information to the computer 1101 via an input device (not shown). Examples of such input devices include, but are not limited to, keyboards, pointing devices (e.g., computer mouse, remote control), microphones, joysticks, scanners, touch-enabled devices such as touchscreens, tactile input devices such as gloves and other body coverings, and motion sensors. These and other input devices may be connected to one or more processors 1103 via a human-machine interface 1102 coupled to bus 1113, but may also be connected by other interfaces and bus structures, but are not limited to, parallel ports, game ports, IEEE 1394 ports (also known as FireWire ports), serial ports, network adapters 1108, and / or Universal Serial Bus (USB).

[0062] In yet another embodiment, the display device 1111 can also be connected to the bus 1113 via an interface such as a display adapter 1109. The computer 1101 may have multiple display adapters 1109, and the computer 1101 may have multiple display devices 1111. For example, the display device 1111 may be a monitor, an LCD (liquid crystal display), an LED display, a television, a smart lens, smart glass, and / or a projector. In addition to the display device 1111, other output peripheral devices may include components such as a speaker (not shown) and a printer (not shown) that can be connected to the computer 1101 via an input / output interface 1110. Any step and / or result of these methods can be output to an output device in any form. Such output can be any form of visual representation, including but not limited to text, graphics, animations, sound, and haptics. The display 1111 and the computer 1101 may be part of a single device or separate devices.

[0063] In one embodiment, the computer 1101 can be coupled to the system 100 via an input / output interface 1110. The computer 1101 can be configured to monitor and store data. For example, the computer 1101 can be configured to store images acquired by a camera connected to the system 100, and to store data related to pass / fail statistics generated during inspections occurring in the system. The computer 1101 can also be used as a programming interface to one or more smart devices (e.g., smart cameras) and / or embedded logic controllers that require customized firmware to run on it. The computer 1101 can be used to generate, troubleshoot, upload, and store iterations of this software or firmware.

[0064] Computer 1101 can operate in a network environment using logical connections to one or more remote computing devices 1114a, b, c. For example, remote computing devices 1114a, b, c may be personal computers, computing stations (e.g., workstations), portable computers (e.g., laptops, mobile phones, tablet devices), smart devices (e.g., smartphones, smartwatches, activity trackers, smart apparel, smart accessories), security and / or monitoring devices, servers, routers, network computers, peer devices, edge devices, or other common network nodes. The logical connection between computer 1101 and the remote computing devices 1114a, b, c can be made via a network 1115 such as a local area network (LAN) and / or a general wide area network (WAN). Such a network connection may also be made via a network adapter 1108. The network adapter 1108 can be implemented in both wired and wireless environments. Such network environments are customary and common in homes, offices, enterprise-scale computer networks, intranets, and the internet. In one embodiment, the network adapter 1108 can be configured to supply power to one or more connected devices (e.g., cameras). For example, the network adapter 1108 can comply with the Power over Ethernet (PoE) standard, etc.

[0065] For illustrative purposes, application programs and other executable program components, such as the operating system 1105, are shown herein as separate blocks, but such programs and components may reside in different storage components of the computing device 1101 at different points in time and be recognized as being executed by one or more processors 1103 of the computer 1101. Embodiments of the image analysis software 1106 may be stored in or transmitted through any form of computer-readable medium. Any of the disclosed methods may be executed by computer-readable instructions embodied in the computer-readable medium. The computer-readable medium may be any available medium accessible by a computer. Not intended to be limiting, but as an example, the computer-readable medium may include “computer storage medium” and “communication medium.” The “computer storage medium” may include volatile and non-volatile, removable and non-removable media implemented in any method or technique for storing information such as computer-readable instructions, data structures, program modules or other data. Exemplary computer storage media may include RAM, ROM, EEPROM, flash memory or other memory technologies, CD-ROM, digital general-purpose disk (DVD) or other optical storage devices, magnetic cassettes, magnetic tapes, magnetic disk storage devices or other magnetic storage devices, or any other media that can be used to store desired information and are accessible by a computer.

[0066] This method and system may employ artificial intelligence (AI) techniques such as machine learning and iterative learning. Examples of such techniques include, but are not limited to, expert systems, case-based reasoning, Bayesian networks, behavior-based AI, neural networks, fuzzy systems, evolutionary computation methods (e.g., genetic algorithms), swarm intelligence (e.g., Ant algorithms), and hybrid intelligent systems (e.g., expert reasoning rules generated through neural networks or production rules from statistical learning).

[0067] The disclosed methods and systems have been implemented and tested, and the results have been compared to standard manual-only, operator-driven assembly line processes. The table below shows that the disclosed methods and systems are superior to standard manual-only, operator-driven assembly line processes.

[0068] [Table 1] While the methods and systems described herein are described in reference to preferred embodiments and specific examples, the embodiments described herein are intended to be illustrative and not restrictive in any way, and their scope is not intended to be limited to the specific embodiments described herein.

[0069] Unless otherwise specified, no method described herein is ever intended to be construed as requiring its steps to be performed in a particular order. Accordingly, unless a method claim actually lists the order in which its steps should be followed, or unless otherwise specifically stated in the claims or description that those steps should be limited to a particular order, no order is ever implied in any way. This holds true for any possible implicit basis for interpretation, including logical issues concerning the arrangement or flow of steps, simple meanings derived from grammatical structure or punctuation, and the number or type of embodiments described in the specification.

[0070] It will be apparent to those skilled in the art that various modifications and variations can be made without departing from the scope or spirit of the invention. Other embodiments will be obvious to those skilled in the art by considering this specification and the practices disclosed herein. This specification and the examples are for illustrative purposes only, and the true scope and spirit of the invention are intended to be shown by the following claims.

Claims

1. It is a system, belt and, Multiple imagers, Acquire images within each field of view, The image is analyzed to determine the number and / or orientation of one or more items within each field of view. It is determined that two or more occurrences of the first pattern of one or more items appear on the same axis within each field of view, The number of one or more items in each field of view is determined based on the fact that at least two occurrences of the first pattern appear on the same axis within each field of view, Multiple imagers configured to generate a pass / fail signal based on a comparison between the determined number of one or more items in each field of view and the predicted number of those items, A processor coupled to each of multiple imagers, The system receives pass / fail signals from each of the multiple imagers. A processor configured to advance the belt based on receiving pass / fail signals from each of multiple imagers. A system equipped with these features.

2. In the system described in claim 1, Multiple imagers, further, The number of one or more items in each field of view is determined based on whether two or more occurrences of the first pattern of one or more items appear on different axes within each field of view. It is configured to generate a fail inspection signal based on a comparison between the determined number of one or more items within each field of view and the predicted number of those items. The processor is coupled to each of the multiple imagers. Receiving a failure inspection signal, A system configured to prevent the belt from moving based on the reception of a failure inspection signal from one or more of several imagers.

3. In the system described in claim 1, The first pattern is a system that includes text patterns, number patterns, symbol patterns, and combinations thereof.

4. In the system described in claim 1, At least one of the multiple imagers is further, Determine the position of the anti-slip material within each field of view. The determined positions of the anti-slip material within each field of view are compared to the reference position. It is determined that the determined position is different from the reference position, The offset is determined based on the difference between the determined position and the reference position. A system configured to transmit a signal to the processor to adjust the distance and advance the belt by the amount of the offset.

5. In the system described in claim 1, The belt includes multiple anti-slip elements spaced apart from each other. The length of each anti-slip strip is, in effect, perpendicular to the length of the belt. One or more fields of view are located between adjacent anti-slip surfaces among multiple anti-slip surfaces. At least one of the multiple imagers is further, Determine the position of the anti-slip material within each field of view. The determined positions of the anti-slip material within each field of view are compared to the reference position. It is determined that the determined position is different from the reference position, The offset is determined based on the difference between the determined position and the reference position. A system configured to transmit a signal to the processor to adjust the distance and advance the belt by the amount of the offset.

6. In the system described in claim 1, A system in which at least one of several imagers is further configured to count the number of empty tray positions.

7. In the system described in claim 1, A system in which at least one of several imagers is further configured to determine trayless patterns.

8. In the system described in claim 1, The processor is The belt is only advanced after receiving a pass / fail signal from each of the multiple imagers. A system configured to prevent the belt from moving forward until it receives another pass inspection signal.

9. In the system described in claim 1, The processor further, The belt is only advanced after receiving a pass / fail signal from each of the multiple imagers. A system configured to prevent the belt from moving forward until it receives another pass inspection signal.

10. In the system described in claim 1, One or more articles are a system comprising one or more syringes, syringe needles, or auto-injectors.

11. It is a method, Each of the multiple imagers acquires a first image of the tray. For each of the multiple imagers, the image is analyzed to determine the number and / or orientation of one or more objects within each field of view. For each of the multiple imagers, it is determined that two or more occurrences of the first pattern of one or more items appear on the same axis within each field of view. For each of the multiple imagers, the number of one or more items in each field of view is determined based on the fact that at least two occurrences of the first pattern appear on the same axis within each field of view. For each of the multiple imagers, a pass / fail signal is generated based on a comparison between the determined number of one or more items in each field of view and the predicted number of those items. A method for advancing a belt containing a tray based on receiving pass / fail signals from each of multiple imagers.

12. The method according to claim 11 further, Based on whether two or more occurrences of a first pattern of one or more items appear on different axes within each field of view for one or more of the multiple imagers, the number of one or more items in each field of view is determined. For one or more of the multiple imagers, a rejection signal is generated based on a comparison between the determined number of one or more items in each field of view and the predicted number of those items. Receiving a failure inspection signal from one or more of the multiple imagers, A method for preventing the belt from moving based on the reception of a failure inspection signal from one or more of multiple imagers.

13. In the method according to claim 11, The first pattern is a method that includes text patterns, number patterns, symbol patterns, and combinations thereof.

14. In the method according to claim 11, The first pattern includes text patterns, number patterns, symbol patterns, and combinations thereof. The aforementioned method further, For each of the multiple imagers, it is determined whether two or more occurrences of text patterns, number patterns, symbol patterns, and combinations thereof appear on the same axis within each field of view. A method for determining the number of one or more items in each field of view based on whether two or more occurrences of text patterns, number patterns, symbol patterns, and combinations thereof appear on the same axis within each field of view.

15. In the method according to claim 11, A method in which two or more occurrences of the first pattern are associated with one or more syringes, syringe needles, or autoinjectors.

16. In the method according to claim 11, To obtain the first image of the tray, For each of the multiple imagers, the position of the anti-slip material within each field of view is determined. For each of the multiple imagers, the determined position of the anti-slip surface within each field of view is compared to the reference position. For each of the multiple imagers, it is determined that the determined position is different from the reference position. For each of the multiple imagers, an offset is determined based on the difference between the determined position and the reference position. A method comprising sending a signal to a processor for each of several imagers to adjust the distance and advance the belt by the amount of the offset.

17. In the method according to claim 11, For each of the multiple imagers, the position of each anti-slip surface within each field of view is determined from multiple anti-slip surfaces spaced apart from each other, and the length of each anti-slip surface is substantially perpendicular to the length of the belt, and the field of view is located between adjacent anti-slip surfaces among the multiple anti-slip surfaces. For each of the multiple imagers, the determined position of the anti-slip surface within each field of view is compared to the reference position. For each of the multiple imagers, it is determined that the determined position is different from the reference position. For each of the multiple imagers, an offset is determined based on the difference between the determined position and the reference position. A method comprising sending a signal to a processor for each of several imagers to adjust the distance and advance the belt by the amount of the offset.

18. The method according to claim 11 further, A method for obtaining a first image of a tray and then counting the number of empty tray positions for each of multiple imagers.

19. In the method according to claim 11, Determining the number and / or orientation of one or more articles is: A method including determining a trayless pattern for each of multiple imagers.

20. In the method according to claim 11, Determining the number and / or orientation of one or more articles is: A method comprising determining the number and / or orientation of one or more articles while the belt is stationary.

Images