Seal defect detection system and method

Abstract

Methods and systems for detecting defects of a sample holder in real time during manufacturing. Such systems utilize one or more external sensors to detect characteristics or parameters of sample clips and / or manufacturing processes from one or more data sets. The external sensor includes any one of an RGB camera, an infrared camera, a high resolution optical camera, and an ultrasonic microphone, or a combination thereof. The automation system obtains a data set from an external sensor and compares the data set to a baseline of the sample holder and / or the manufacturing process so that defects can be determined from variations from the baseline. Such methods may utilize feature extraction and spectroscopic analysis to identify features or characteristics for comparison to a baseline. The machine learning model may be used to determine an algorithm from a data set of an acceptable sample cartridge and a data set from an external sensor associated with a cartridge defect.

Description

Technical Field

[0001] This application claims priority to U.S. Provisional Patent Application No. 63 / 374,313, filed on September 1, 2022, the entire content of which is incorporated herein by reference.

[0002] This application is generally related to the following concurrently filed U.S. non-provisional patent applications: "Faulty Unit Detection System and Methods" (Attorney Docket No. 85430-1406037-018410US) and "Transfer Learning Methods and Models Facilitating Defect Detection" (Attorney Docket No. 85430-1406033-018310US), the entire contents of which are incorporated herein by reference for all purposes.

[0003] The present invention generally relates to the field of manufacturing of biological devices, and more particularly to defect detection of sample cartridges for analyzing fluid samples. Background Art

[0004] In recent years, significant developments have been made in the field of biological detection devices for facilitating manipulation of fluid samples within sample cartridges for biological detection of fluid samples via nucleic acid amplification testing (NAAT). Notable developments in this field include the sample cartridges of Cepheid. The construction and operation of this type of cartridge can be further understood by referring to U.S. Patent No. 6,374,684, entitled "Fluid Control and Processing System," and U.S. Patent No. 8,048,386, entitled "Fluid Processing and Control." Although these sample cartridges have made great progress in the prior art, as with any precision instrument, there are still certain challenges in the manufacturing of sample cartridges. In particular, the assembly of multiple components in a precision pressurized instrument may occasionally result in defects, causing sample cartridge leakage or inability to maintain the internal pressure required for successful operation.

[0005] ​Traditional sample cassette manufacturing systems involve a series of manufacturing processes and steps, where certain processes may introduce defects into the sample cassette, such as causing the sample cassette to leak or fail to maintain internal pressure, i.e., what is commonly referred to as seal failures. Certain steps, such as welding a lid device to the cassette body, or sealing reagents into the cassette with a membrane, may introduce seal defects that are difficult to detect. Current defect detection methods include a variety of seal testing methods and visual inspections. However, these methods typically utilize destructive testing methods and / or sometimes fail to identify all defects. In addition, these methods usually conduct spot checks on several cassettes within a batch. Once unacceptable defects are detected, the entire batch is scrapped to prevent defective cassettes from reaching the end users. Moreover, due to human error, it is also difficult to maintain consistency with current detection methods.

[0006] Therefore, there is a great need in the art for improved defect detection methods to enhance consistency in the manufacturing process and reduce unnecessary waste. In particular, a non-destructive detection method is needed that does not require extensive testing after product manufacturing and can avoid human errors associated with visual inspection methods. Summary of the Invention

[0007] In one aspect, the present invention relates to a method for detecting defects in a sample cartridge. The method may include the following steps: during a sample cartridge manufacturing process, obtaining one or more data sets from one or more external sensors; comparing the one or more data sets with a baseline data set of the manufacturing process and / or the sample cartridge associated with an acceptable sample cartridge; and identifying a defect in the sample cartridge based on the variation between the one or more data sets and the baseline. The defect may be determined in real time during sample cartridge manufacturing by an automated method. In certain embodiments, obtaining the one or more data sets includes obtaining a plurality of images by one or more RGB cameras and / or infrared (IR) cameras, wherein the one or more data sets include thermal imaging images. In certain embodiments, the identification of the defect utilizes an algorithm derived by a machine learning model based on a plurality of data sets associated with an acceptable sample cartridge and a plurality of data sets associated with cartridge defects for differential analysis. In certain embodiments, the method includes extracting features corresponding to sample cartridge features from one or more images. In certain embodiments, the one or more data sets include a plurality of consecutive images obtained during the manufacturing process, which may include images from multiple positions or different perspectives. In certain embodiments, the algorithm matches data from different data sources and provides time series data processing to facilitate defect detection. In certain embodiments, the one or more data sets are associated with cartridge defects using an algorithm determined by a machine learning model, which may include any one of deep learning, supervised learning, and unsupervised learning. In certain embodiments, the automated method is performed during a manufacturing / assembly process, such as a reagents-on-board automated line (ROBAL), to detect any defects in the cartridge, lid, or membrane seal in real time by the method.

[0008] In some embodiments, the manufacturing process includes ultrasonically welding a lid device to a clip body of a sample clip. In some embodiments, the manufacturing process includes ultrasonically welding internal clip components within the clip body during manufacturing. In certain embodiments, the manufacturing process includes heat-sealing a film to the top of a lid device on a clip body of a sample clip. In certain embodiments, the manufacturing process includes sealing the film (by heat-sealing or ultrasonic welding) to isolate a chamber containing a reagent within the clip before the lid device is secured to the clip body. The one or more external sensors may include an RGB camera, an infrared camera, or a combination thereof, and are disposed at one or more locations along a sample clip production line. In certain embodiments, the one or more external sensors include an ultrasonic microphone. In such an embodiment, the one or more data sets further include an ultrasonic audio spectrum compared to a baseline ultrasonic audio spectrum of a successful weld, the characteristics of which may be peaks and / or variations in the ultrasonic audio spectrum. In other embodiments, the one or more external sensors include a high-resolution camera for obtaining high-resolution optical images. In such an embodiment, the variation may be a deviation between an extracted feature and a corresponding feature of a baseline high-resolution image.

[0009] In another aspect, the present invention relates to a system for detecting defects in a sample clip. The system may include: one or more external sensors disposed at or near a manufacturing process station of a sample clip production line, wherein the one or more external sensors include any one of an RGB camera, an infrared camera, or a combination thereof; and a processing unit operatively coupled to the one or more external sensors and communicatively connected to a control unit of an automated manufacturing device of the production line. Instructions for performing automated defect detection of a sample clip are recorded on the processing unit, which may include the steps of: obtaining one or more data sets from the one or more external sensors during a sample clip manufacturing process; comparing the one or more data sets with a data set of a baseline associated with an acceptable sample clip for the manufacturing process and / or the sample clip; and identifying a defect in the sample clip based on a variation between the one or more data sets and the baseline. The processing unit may be configured to determine defect detection in real time during sample clip manufacturing.

[0010] In some embodiments, the processing unit is further configured to obtain one or more data sets including: acquiring images through one or more RGB cameras and / or infrared cameras, wherein the one or more data sets are one or more thermal imaging images. The processing unit may be further configured to: perform mutation identification using an algorithm derived by machine learning based on multiple data sets associated with acceptable sample cartridges and multiple data sets associated with cartridge defects. In some embodiments, the processing unit may be further configured to extract one or more features corresponding to standard features of the sample cartridge from the one or more images. The one or more data sets may include multiple consecutive images that can be acquired from different positions and / or perspectives during the manufacturing process. In some embodiments, the manufacturing process includes ultrasonically welding a lid device onto the cartridge body of the sample cartridge. In some embodiments, the manufacturing process includes ultrasonically welding internal components of the cartridge inside the cartridge body during manufacturing. In some embodiments, the manufacturing process includes isolating or sealing one or more reagent-containing chambers inside the cartridge with a film material by heat sealing or ultrasonic welding. In some embodiments, the manufacturing process includes heat-sealing a film onto the top of the lid device on the cartridge body of the sample cartridge. The automated defect detection system may be integrated into the automated software for controlling the production line. In some embodiments, the defect detection may be integrated with a cartridge automated production line (such as ROBAL). In some embodiments, the defect detection is achieved by inputting automated parameters and / or the output from one or more external sensors into a pre-trained model, which is trained by machine learning. In some embodiments, both supervised learning and unsupervised learning are used in the training of the model. In some embodiments, the model accesses the data set for defect detection through a cloud-based server or a cloud-based data sharing platform, which may include operating parameters and / or the output from the one or more external sensors, allowing the defect detection to be highly scalable across multiple production lines for training and / or use. BRIEF DESCRIPTION OF THE DRAWINGS

[0011] The patent or application is filed with at least one drawing in color. Copies of this patent or patent application publication with color drawings will be provided by the Patent Office upon request and payment of the necessary fees.

[0012] Figure 1A is a flowchart showing a defect detection method according to some embodiments, which uses manufacturing parameter inputs fed into a machine learning model to assist in classifying manufactured products.

[0013] Figure 1B is a flowchart showing a defect detection method according to some embodiments, which uses manufacturing parameter inputs fed as labels into a machine learning model that uses both supervised learning and unsupervised learning to assist in the classification of manufactured products.

[0014] Figure 1C Is a schematic architecture diagram showing a system setup for facilitating automated defect detection according to certain embodiments.

[0015] Figure 1D Is a schematic data flow diagram with cloud-based storage and data sharing according to certain embodiments to enable highly scalable model training.

[0016] Figure 1E Is another flowchart showing a defect detection method according to certain embodiments that uses images and parameters input into a machine learning model to assist in the classification of manufactured products.

[0017] Figure 2A Shows an exemplary sample cartridge provided to a user, which includes a welded lid device and a membrane seal, with a lid opening on the lid top to receive a fluid sample. Figure 2B Shows an exploded view of the sample cartridge, illustrating its main components according to certain embodiments, including a lid device, a multi-chamber body, a reaction vessel, a valve assembly, and a base. Figures 2C to 2D Shows a detailed view of the lid device. Figure 2E Shows the lid device before being placed above the sample cartridge body and waiting for ultrasonic welding by a welding head. Figure 2F Shows a schematic diagram of a part of the production line, which is particularly relevant to the image detection method described herein.

[0018] Figure 3A Shows a manufacturing process flowchart and identifies various sources of seal test failure in an exemplary sample cartridge manufacturing method. Figure 3B Then shows Figure 3A The manufacturing process flow according to certain embodiments as shown, which adds additional external sensors for obtaining parameters for automated defect detection.

[0019] Figure 4 Shows a flowchart of data collection in the manufacturing process according to certain embodiments, including parameters that can be used for automated defect detection.

[0020] Figures 5A to 5D Shows exemplary infrared thermal images obtained in the manufacturing process to assist in automated defect detection, including Figure 5A The images obtained during membrane sealing as shown and Figure 5B The images obtained during the welding process as shown. Figure 5E Shows a flowchart according to certain embodiments, in which images and possibly other parameters are input into a deep learning model.

[0021] Figures 6A to 6DExemplary infrared thermal images acquired during a manufacturing process are shown to facilitate automated defect detection through image matching in accordance with certain embodiments.

[0022] Figure 7 An infrared camera sensor is shown added next to a welding station of a sample clip manufacturing station according to certain embodiments to obtain infrared images for automated defect detection.

[0023] Figures 8A to 8C , Figures 9A to 9C , Figures 10A to 10D as well as Figures 11A to 11C Exemplary infrared thermal images acquired at different points in time during a welding manufacturing process are shown to facilitate automated defect detection in accordance with certain embodiments.

[0024] Figures 12A to 12B An ultrasonic microphone sensor is shown added near a sample holder manufacturing station to capture audio from the manufacturing process for automated defect detection in accordance with certain embodiments.

[0025] Figure 13 , Figures 14A to 14B , Figures 15A to 15B as well as Figure 16 An exemplary ultrasonic audio spectrum acquired during a welding process to facilitate automated defect detection is shown in accordance with certain embodiments.

[0026] Figures 17A to 17E An optical camera positioned for capturing high resolution images of manufactured sample cartridges and high resolution optical images for automated defect detection is shown in accordance with certain embodiments.

[0027] Figures 18A to 18D A top view optical image for evaluating cover alignment for automated defect detection is shown in accordance with certain embodiments.

[0028] Figure 19A A welding station according to certain embodiments is shown, equipped with a perspective infrared camera, an RGB camera, and a top-view RGB camera.

[0029] Figures 19B to 19E Infrared images of baseline and various entrapment defects are shown in accordance with certain embodiments.

[0030] Figure 20 Experimental comparison results of image-based defect detection according to certain embodiments and manual inspection are presented.

[0031] Figures 21A to 21C RGB images, IR images, and computer vision features based on infrared images associated with different clamping defects after welding are shown in accordance with certain embodiments.

[0032] Figures 22A to 22B Shows an infrared image of a membrane seal defect according to certain embodiments.

[0033] Figure 23A Shows a microscopic image of a membrane seal according to certain embodiments, Figure 23B while showing an infrared image of the membrane seal for defect detection.

[0034] Figures 24A to 24B Shows an RGB image and an infrared image indicating a molten shaft defect according to certain embodiments.

[0035] Figures 25A to 25B Shows a flowchart of automated defect detection according to certain embodiments.

[0036] Figure 26 Shows a traditional infrastructure for integrable automated defect detection according to certain embodiments.

[0037] Figure 27 Shows an updated infrastructure for integrating automated defect detection according to certain embodiments. DETAILED DESCRIPTION

[0038] The present invention generally relates to the field of manufacturing defect detection, and more particularly to the detection of sample clamping defects in manufacturing. In certain embodiments, the method and system provide automated defect detection, which can be performed in real time during manufacturing. A flowchart of such an automated detection method is shown in Figures 1A to 1B and is further detailed below.

[0039] I. SYSTEM OVERVIEW

[0040] In one aspect, the present invention relates to an automated defect detection system for detecting defects in the manufacture of a sample cartridge for analyzing a sample of a target analyte. An exemplary sample cartridge is as shown in Figure 2A The sample cartridge includes a lid device 100 sealed to the top of a cartridge body 200, in which reagents and fluid samples are stored. The lid device 100 includes a bottom lid portion sealed to the cartridge body and a top lid portion that can be flipped upward, as shown, so that a user can deposit a fluid sample into the cartridge. When the sample cartridge is provided to the user, the reagent is already placed in a selected chamber and sealed in the cartridge by a film 110 sealed to the top of the bottom lid. The film includes a central opening for a syringe and an opening for inserting a fluid sample.

[0041] Figure 2B Describes an exemplary diagnostic assay cartridge suitable for nucleic acid amplification detection. The shown cartridge is based on Cartridge (Cepheid, Sunnyvale, CA). The cartridge 100 includes a cartridge body 200 having a plurality of chambers 208 defined therein for storing various reagents and / or buffers. These chambers are disposed around a central injection barrel 209 which is in fluid communication with a valve body 210 via a valve body injection tube 211 extending to the injection barrel 209. The valve body 210 is interfaced within the cartridge body and supported on a cartridge base 210. The cartridge typically contains one or more channels or cavities which can accommodate filter materials (such as glass filter columns) which can be used to bind and elute nucleic acids. In different embodiments, the cartridge further includes one or more temperature-controlled channels or chambers which, in some embodiments, can be used as amplification chambers for nucleic acid amplification such as by polymerase chain reaction (PCR) or isothermal amplification. A "plunger" (not shown) can be used to draw fluid into the injection barrel 209 and provide selective fluid communication between the various reagent chambers, channels, and reaction chambers by rotating the valve body / injection tube. Thus, by rotating the valve and operating the plunger, selective fluid communication can be achieved between the various reagent chambers, reaction chambers, matrix materials, and channels, and reagent movement (such as chamber loading or unloading) is achieved by operating the "syringe" action of the plunger. An attached reaction vessel 216 ("PCR tube") provides an optical window to enable real-time detection of, for example, amplification products by operating modules within the system described herein. It will be appreciated that such reaction vessels can contain various different chambers, conduits, or micro-well arrays for detecting target analytes. The sample cartridge can be equipped with means for preparing a biological fluid sample prior to transport to the reaction vessel. Any chemical reagents required for virus or cell lysis, or means for binding or detecting target analytes (such as reagent beads, filter materials, etc.) can be contained within one or more chambers of the sample cartridge and can be used for sample preparation.

[0042] The exemplary use of such a sample cartridge having a reaction vessel for analyzing a biological fluid sample is described in the co-owned U.S. Patent Application No. 6,818,185, entitled "Cartridge for Conducting Chemical Reactions" (filed on May 30, 2000), the content of which is incorporated herein by reference in its entirety for all purposes. Examples of the sample cartridge and related instrument modules are shown and described in U.S. Patent No. 6,374,684, entitled "Fluid Control and Handling System" (filed on August 25, 2000), and U.S. Patent No. 8,048,386, entitled "Fluid Handling and Control" (filed on February 25, 2002), the content of which is incorporated herein by reference in its entirety for all purposes. Various aspects of the sample cartridge can be further understood by referring to U.S. Patent No. 6,374,684, which describes certain aspects of the sample cartridge in more detail. Such a sample cartridge may include a fluid control mechanism, such as a rotary fluid control valve, which is connected to the chambers of the sample cartridge. Rotating the rotary fluid control valve enables fluid communication between the valve and the chambers to control the flow of a biological liquid sample placed in the cartridge into different chambers, where various reagents can be provided according to a specific protocol as needed to prepare a biological fluid sample for analysis. To drive the rotary valve, the cartridge processing module includes a motor, such as a stepper motor typically coupled to a drive train, which mates with a structure on the valve body to control the movement of the valve body and operate in coordination with the movement of a syringe, thereby causing the movement of the fluid sample according to the desired sample preparation process. Regarding the fluid metering and dispensing functions achieved by the rotary valve under a specific sample preparation process, reference can be made to the relevant description in U.S. Patent No. 6,374,684.

[0043] Figures 2C to 2DShows a detailed view of an exemplary cap device 100, which includes a central opening for the syringe / plunger to pass through to enable the movement of fluid between chambers. The central opening is surrounded by a plurality of (channeled) chimneys 102 that project into the opening 104 of the top cover. Thus, the cap device 100 includes a substantially uniform bottom surface 106, such that the illustrated internal weld pattern does not co - extend with any walls extending from the bottom surface 106. The chambers of the fluid container device disclosed by the present invention may contain one or more reagents for different uses. These reagents can be in various forms. Non - limiting exemplary reagent forms can include solutions, dry powders, or lyophilized beads, etc. The reagents can be used for a variety of purposes including but not limited to chemical reactions and / or enzymatic reactions, sample preparation and / or detection, etc. Non - limiting exemplary purposes can include cell or microbial lysis, purification or separation of target analytes (such as specific cell populations, nucleic acids, or proteins), digestion or modification of nucleic acids or proteins, nucleic acid amplification, and / or detection of target analytes, etc. More details of the cap device can be found in U.S. Patent No. 10,273,062, the content of which is incorporated herein by reference in its entirety for all purposes.

[0044] Figure 2C Shows a top view of the bottom of the bottom cap and the bottom surface of the top cover portion. The bottom surface of the bottom cap includes a plurality of chimneys 102 that project upward from the top surface of the bottom cap portion and are received in corresponding openings 104 in the top cover portion. The plurality of chimneys 102 and openings 104 are arranged around a central opening 103 through which the injection instrument of the module extends during operation of the sample cartridge to facilitate fluid flow between chambers through the movement of the valve body. Figure 2D Shows a bottom - side bottom view of the bottom cap of the cap device 100, including the bottom - side main surface and the top - side of the top - cover portion. The bottom - side of the bottom - cover portion is welded to the top edge of the cartridge body. For ease of welding, a continuous raised weld ridge 101 is provided between the edge - alignment structure 107 and the outermost wall on the upper - edge perimeter of the bottom cap. When placed in the correct orientation, the edge - alignment structure 107 and the outermost wall prevent the bottom cap from rotating excessively relative to the fluid container 200, thereby aligning the raised weld ridge 101 on the bottom cap with the welding features (such as the top edge of the wall) of the cartridge body. A plurality of walls 108 extend from the central portion of the bottom - side main surface. These walls are arranged in a petal - like pattern around the central opening 103. Here, the walls consist of six petals. There is a raised weld pattern at the top edge of the walls. The raised weld pattern is connected to the weld ridge 101. In this way, a fluid zone is created outside the petals. When the fluid container and the bottom cap are welded through the raised weld pattern and the weld ridge, the sub - containers within the bottom container are fluid - isolated from each other (at least at the fluid - container and bottom - cap interface).

[0045] Figure 2EShows the relationship between the lid device 100 and the clip body 200. The clip body 200 contains multiple chambers, and fluid coupling or non-coupling between the chambers can be achieved according to the position of the internal valve assembly. These chambers are defined by walls extending to the top of the clip body 200. After the fusion interface between the lid device 100 and the clip body 200 is formed, these chambers are sealed and isolated from each other through the welding interface between the raised welding pattern 160 and the welding ridge 156 and the chambers of the container 200. The lid device 100 is welded to the fluid container by an ultrasonic welding head 1901, and when the lid device is placed on the container 200, the ultrasonic welding head engages with the platform 120. The welding head 1901 generally includes a metal cylinder for fitting around the platform. The welding head is part of a larger welding device (not shown), which provides energy to the welding head. Commercially available ultrasonic welding devices can be obtained from manufacturers such as Hermann Ultrasonics (Bartlett, IL 60103) or Branson Ultrasonics (Emerson Industrial Automation Division, Eden Prairie, MN 55344) and can be used for this process. The above welding operations are usually performed at the welding station of the sample clip manufacturing / production line.

[0046] Figure 2FShows a partial schematic diagram 1500 of a cartridge manufacturing / assembly line, including a welding station 510, a reagent filling station 520, and a membrane sealing station 530. At the welding station 510, automated equipment places the lid device 100 on top of the cartridge body 200, and an ultrasonic welding head applies downward pressure and ultrasonic energy for up to 5 seconds (e.g., at least 3 to 5 seconds) to weld the lid to the cartridge body by ultrasonic welding. As described previously, the shape of the welding ridge on the bottom side of the bottom lid is designed to be hermetically welded to the top edge of the cartridge body chamber. As described previously, an external sensor #1 can be used to monitor parameters or characteristics during the welding process, such that the output of the sensor can be used to evaluate the welding quality. After welding, the cartridge is moved to the reagent filling station 520 in an automated sequence, and one or more reagents (e.g., beads, fluids, or powders) or processing materials are deposited in selected chambers through the shaft openings of the lid. After placing the reagents or other compounds in the chambers, the cartridge is moved to the membrane sealing station 530, where a film is sealed to the top surface of the bottom lid to seal the shaft channel openings, thereby sealing the reagents and processing materials within the cartridge. Automated equipment places the film on top of the top lid (with the lid in the closed configuration) and applies heat to seal the film to the lid. As described herein, an external sensor #2 can be used to monitor parameters or characteristics during the sealing, such that the output of the sensor can be used to evaluate the membrane seal. In some embodiments, the film is ultrasonically welded to the top of the lid. The film includes a central opening (e.g., a cross cut), allowing an injection instrument to pass through the film, and another opening in the lower right allows a user to inject a fluid sample into the sample chamber. However, the remaining openings on the lid (including the shaft openings leading to the reagent chambers) are sealed by the film. As will be described below, an automated defect detection system utilizes one or more external sensors (external sensors #1 and #2) disposed at or adjacent to the welding station 510 and / or one or more external sensors (external sensors #2) disposed at or adjacent to the membrane sealing station 530 to evaluate the integrity of the welded lid structure or the welding and / or membrane seal.

[0047] II. Examples of Defect Detection Methods

[0048] A. Overview

[0049] In one aspect, the systems and methods described herein utilize one or more external sensors disposed at one or more locations on a manufacturing production line to obtain an additional data set of parameters regarding the manufacturing process or the properties of the cartridge assembly during manufacturing to detect defects during manufacturing in real time. The external sensors can include, but are not limited to, any high-resolution RGB or infrared cameras, ultrasonic microphones, or any combination thereof. In certain embodiments, the infrared cameras are configured to obtain the thermal distribution of the lid during ultrasonic welding or the thermal distribution of the film during or after film heat sealing. Due to the complexity of the data and the limited amount of data associated with defects (relatively rare occurrences), it is advantageous to use a machine learning (ML) model to determine the relationship between one or more features or parameters of the data set from the external sensors and the cartridge defects. By using an ML model, a correlation can be established between subtle changes in temperature distribution, image matching, or audio tracking and defects that are difficult to detect by visual inspection or standard inspection methods. Advantageously, this automated defect detection allows for the real-time detection of defects during manufacturing, so that defective cartridges can be removed during manufacturing. In some embodiments, the automated defect detection obviates the need to select cartridges from each batch for post-manufacturing destructive testing, especially destructive testing, thus avoiding waste and reducing costs while significantly improving defect detection.

[0050] B. Example of a method for detecting seal defects

[0051] In one aspect, the automated process uses an ML model to correlate one or more parameters or features of the manufacturing process and / or the product assembly with specific defects. In some embodiments, the defects are associated with weld seals (e.g., overwelding, underwelding, shaft rupture) and / or film seals (e.g., incomplete seal, shaft melting). The parameters or features can include any property associated with the manufacturing process or the product assembly. Advantageously, this method allows for the real-time detection of defects during the manufacturing process so that defective cartridges can be removed during the process.

[0052] As Figure 1AAs shown, it shows a flowchart schematic diagram 1000. The automated method may include obtaining one or more inputs of any number of parameters associated with a manufacturing method and process. In this embodiment, the parameters may be associated with the following stages: the first stage of manufacturing (e.g., process parameters collected by automated control, such as incoming material parameters), the second stage (e.g., welding parameters, clamping parameters, such as batch numbers or serial numbers), or the third stage (e.g., alignment data, sensor data, first article inspection (FAI) data). These one or more inputs are fed into an ML model, which analyzes the various inputs in combination with actual test data (such as seal test data) to determine the association between one or more parameters / features and their respective defects. The ML model is used to determine an algorithm by which a sample clamp can be classified (e.g., qualified, unqualified due to defects) based on one or more parameters / features obtained from a dataset of one or more external sensors and expert labels (qualified / unqualified labels for a given image). It should be understood that the algorithm may utilize one or more inputs, various combinations of parameters, and the weighting of one or more parameters, or the relationships between parameters. Preferably, the algorithm is applied in real time during manufacturing (once trained on certain datasets) so that defective clamps can be removed from the production line. The automatic detection method described herein may supplement or replace standard tests and manual inspections.

[0053] As Figure 1B shown, it shows another flowchart schematic diagram 1100. The automated method may include obtaining one or more inputs of any number of parameters related to a manufacturing method and process. In this embodiment, the parameters may include, but are not limited to, any of the following: factory parameters (such as process parameters), incoming material parameters, specific process data (such as welder data), sensor data (such as optical RGB or infrared imaging). These one or more inputs are fed into an ML model, which analyzes the various inputs in combination with actual test data (such as seal test data) to determine the association between one or more parameters and defects. In this embodiment, the ML model may include supervised learning and / or unsupervised learning, and may use seal test results and functional test results as labels to determine an algorithm by which a sample clamp can be classified (e.g., qualified / unqualified seal test failure, qualified / unqualified functionality). It should be understood that the algorithm may be designed to be associated with the requirements of a given test standard, such as a seal test failure (STF) test or a functional test.

[0054] As Figure 1CAs shown, a schematic overview diagram 1200 of the system settings, the system being configured to capture RGB and infrared images of a cartridge lid during cartridge production. The settings include an RGB camera 1110, a light source 1111, and an optical element 1112, which are configured to acquire an RGB image 1113. In this embodiment, the RGB image is acquired from a top view of the lid device to detect defects (e.g., shaft features) in the lid structure, although it will be understood that various other views from various other angles may also be used to evaluate these or other features. The settings also include an infrared camera 1120, which acquires an infrared image 1121 of the lid. In this embodiment, the infrared camera may be configured to acquire a plurality of sequential images 1122, 1123 before, during, or after the process to evaluate the process, such as when welding the lid to the cartridge and / or applying a film seal.

[0055] As Figure 1DAs shown, the configuration of the system infrastructure diagram 1300 makes it highly scalable for training and prediction data streams. For example, by leveraging cloud infrastructure and online storage, a web-based interface allows the expansion of data streams to include large amounts of information, multiple systems, and / or personnel, including personnel at remote locations. In this embodiment, infrared and RGB images are obtained during the manufacturing process (e.g., during or after welding, or during film sealing), and the images are input into any suitable image monitoring / processing software (e.g., Thermal Process Monitoring System - TPMS), from which the data can be stored on the cloud infrastructure or sent to online storage (e.g., Amazon S3), and the data can be fed from the cloud infrastructure or online storage into an AI training model using any suitable software (e.g., DataRobot). The image data can also be sent to long-term storage (e.g., Amazon Glacier). The image data can also be input back into the programmable logic controller (PLC) of ROBAL, and all other ROBAL data and metadata can be input into the automation module using any suitable automation software (e.g., Ignition). IPT dashboard data (weighted measurements from the Seal Test Failure protocol, visual defect inspection results such as shaft embrittlement or rupture) can also be input into the automation module, and then, using any suitable software, the automation data is output to a data sharing platform, including cloud-based data sharing software such as Snowflake, which can also receive input of functional test data from a QC SQL server, and the AI training module can access the QC SQL server. This setup allows the remote execution of the AI training module from the ROBAL production line using cloud-based infrastructure and data sharing. This setup allows the training to be extended to include larger datasets, as well as optional data from multiple ROBAL production lines, and to be available to multiple personnel and sites. While specific software has been referenced, it is understood that any suitable software can be used and the described data stream variations can be achieved.

[0056] As Figure 1EAs shown, a process flow diagram 1400 is presented, and a trained simple model can be used to test the system. This flow chart may include inputs such as infrared and RGB images, welder time series and discrete value data, chuck metadata, and ROBAL sensor data, which are fed into supervised and unsupervised machine learning (ML) models. One or both of the following application labels can be applied: IPT dashboard data and functional test results. Based on these labels, the ML model determines the classification of the chuck, which may include predictions of pass / fail for STF and chuck functionality.

[0057] Figure 3A Shows various sources of seal test failures. On the left, an exemplary manufacturing process flow 300 is shown. This process flow includes lid welding and film sealing steps, which are typically associated with defects that cause seal test failures. For example, 80% of seal test failures can be traced back to the lid welding step (e.g., insufficient or excessive welding), and approximately 20% of seal test failures can be traced back to the film sealing step. Currently, defects caused by these steps can only be detected after the sample chucks are labeled and unloaded as finished products, either through visual inspection by staff or in STF or functional tests, which may result in partial or complete batch scrapping. Therefore, the automated detection systems and methods described herein can avoid such waste by using one or more external sensors located on the existing manufacturing process line to detect defects in real time before product manufacturing is complete. Thus, the methods described herein can improve defect detection with minimal or no adjustment to the existing production line.

[0058] Figure 3BShows the manufacturing process flow line 310, where various external sensors are added to the existing production line setup. In some embodiments, the lid welding step / station may include an optical RGB and / or infrared camera that provides thermal imaging during welding to evaluate weld integrity. In some embodiments, an ultrasonic microphone 311 may also be added to evaluate the weld seam based on the sound during welding. In some embodiments, an infrared camera 312 is used to image along the imaging plane 313, and alignment rods 314 may be used to ensure proper alignment of the chuck in the imaging plane for imaging with the infrared camera. The film sealing step / station may also include an infrared camera that provides thermal imaging during welding to evaluate the integrity of the film seal. A high-resolution optical camera may also be used to check film alignment. The infrared camera can be oriented at any angle (e.g., top view, oblique view, side view, or any combination thereof). It should be understood that any one of these external sensors can be used alone or in combination with any other sensors at various other locations in the process flow line. It should be understood that in order to implement this method in an existing manufacturing process flow line, the external sensors can be synchronized or matched with the operation of the existing process equipment for a given sample chuck. This can be performed using existing chuck tracking (e.g., by S / N) or various other methods (e.g., RFID tags).

[0059] Figure 4 Shows a flow chart 400 that details a method of matching programmable logic controller (PLC) data with welder data. There is no unique identifier that matches the automation data (e.g., Rockwell database) and the welder data. During the welding process, the welder will create and assign a process ID for each weld seam. By combining the Part ID and batch status data, a unique process ID can be created in the PLC and written to the welder, thereby matching the production line data with the welder data. It should be understood that this is just one way to implement the automatic defect detection method in an existing manufacturing process data flow, and various other ways can be used.

[0060] Infrared image data obtained during welding and / or film heat sealing can be used to evaluate the integrity of the weld and seal. Infrared imaging is superior to standard imaging because it includes additional thermal information during the heating / cooling process. In some embodiments, the infrared data can be used to evaluate geometric fit (e.g., fitting an elliptical feature to a sealed area), provide statistical data on temperature changes (e.g., cooling gradients within a seal ring over multiple images), and reveal temperature inconsistencies (e.g., identifying gaps, missing edges in the seal). After analyzing multiple infrared images of standard manufacturing welds / film seals of sample cartridges that passed post-manufacture seal testing, thresholds for each feature / classification can be determined based on their respective characteristics. For example, thresholds for temperature changes along the seal, mismatch of the seal to the ellipse, etc. can be determined. Figures 5A to 5B An exemplary infrared image of a lid in a welding process is shown, Figure 5A showing a detailed view of the internal seal of two adjacent cartridges taken with an infrared camera at a film station. Figure 5B A perspective view of the lid on the cartridge is shown. These infrared thermal images can inform a thermal imaging model in traditional computer vision methods, which can be used for basic automated defect detection to detect thresholds to distinguish good from bad seal fits (e.g. Figure 6C versus Figure 6D ). In some embodiments, thermal imaging can be used to evaluate defects based on whether the thermal imaging is within the desired range, which can be determined by analyzing experimental results with or without the help of machine learning. Advantageously, the ML model allows for consistent identification of features and parameters associated with defects.

[0061] In another aspect, the image can be a series of consecutive images (e.g., a video of a few seconds or less, including skipping intermediate images), which generates more data from which the welding or sealing operation can be characterized by including the time-temperature gradient during cooling / heating. Given the wider range of the dataset, this method benefits greatly from using an ML model. In some embodiments, the image sequence includes at least 3 to 5 consecutive images (e.g., a short video of a few seconds or less), which are fed into the ML model. In some embodiments, the ML model is a deep learning (DL) model that determines the relationship between the sequential progression of the image and seal test failure by combining feature extraction and detection steps in a single complex model (as opposed to separate models for feature extraction, e.g., Histogram of Oriented Gradients and detection, e.g., Support Vector Classifier). Figures 5C to 5D An example of consecutive images during a welding operation is shown, Figure 5EShows a process schematic diagram depicting an exemplary process flow for automatic defect detection, where 3 to 5 consecutive images are input into a deep learning (DL) model, which outputs whether the cartridge is classified as qualified or unqualified.

[0062] In another aspect, the infrared image can be fitted to a domain knowledge specific model of the lid feature shape profile (such as the profile, individual shafts, central opening, etc.) to evaluate the integrity of the feature or the seal around these features. In some embodiments, these profiles are extracted from a binary image and fitted to the corresponding shapes (e.g., an ellipse corresponding to a circular opening imaged from a perspective view). The temperature distribution within the shape can be analyzed, and pass / fail criteria (such as thresholds) can be established. If the residual of the ellipse fit is higher than the standard / threshold, the sample cartridge is unqualified, and if the ellipse with the residual is below the threshold of multiple consecutive frames with a uniform minimum temperature, the sample cartridge is qualified. An example of such an ellipse fit is as Figure 6A shown in FIGS. 6E. Figure 6A Shows a binary image of the extracted profile (ellipse), Figure 6B Shows the heat distribution fitted to the extracted binary image. Figure 6C Shows an acceptable profile fit (qualified) between the ellipse shape (blue) and the profile (red) (residual ellipse 1: 0.02 and residual ellipse 2: 0.04), indicating qualified, Figure 6D Shows an unacceptable profile fit (unqualified) (residual ellipse 1: 0.32 and residual ellipse 2: 0.29), indicating unqualified. Therefore, Figures 6A to 6D Shows feature extraction for defect detection by applying domain knowledge (such as the ellipse seal shape), which is contrary to automatic feature extraction in deep learning. Figure 6A is a binary image, Figures 6B to 6C indicating a well-fitted shape, Figure 6D indicating a poorly-fitted shape (potential defect).

[0063] Figure 7 Shows a welding station 510 with an external sensor, an infrared camera 601, which is located near the cartridge 100 below the welding head 511 of the welder (welder #1) for welding the lid device to the sample cartridge body. The alignment fixture 315 is used to align the lid device and the cartridge before welding. In this embodiment, infrared images are obtained from the back and below of the cartridge by the infrared camera 312 to determine the integrity of the weld. The space behind the existing welder is sufficient to mount the infrared camera 2000 on the alignment fixture 2050 for this feasibility study. It should be noted that the alignment fixture moves before and after welding. If the space behind the welder (such as welder #2) is limited, infrared detection can be performed through an optical fiber.

[0064] Figures 8A to 8D Shows the infrared thermography during the welding process when the infrared camera is behind / below the lid, as Figure 7 shown. Figure 8A Shows the infrared image before welding, Figure 8B shows the infrared image in the early stage of the welding process, Figure 8C shows the infrared image near the end of the welding process, Figure 8D shows the infrared image after welding when the clamp is ready to move further along the production line. These images clearly show the temperature rise at the rear edge of the clamp lid / body during the welding process. The infrared penetration of the clamp body is insufficient to image all the required welding edges. Therefore, it may be beneficial to use an additional infrared camera or an infrared camera with a top-down view, or multiple infrared cameras from different angles, or an RGB camera to provide an additional dataset.

[0065] Figures 9A to 9C Shows defect detection through consecutive infrared images. Figure 9A Shows the infrared image obtained before welding, Figure 8B shows the infrared image obtained during welding, Figure 9C shows the infrared image after welding. In this case, the defect is poor welding from a protruding sample tube, resulting in a seal failure. This gross negligence was introduced by inserting a protruding tube into the sample chamber, so that a "no-weld" area is seen because that area of the lid is not heated or welded to the clamp body. Figure 10 shows another defect detection through consecutive infrared images. In this case, the defect is a gross negligence caused by a scratch (gouge) inside the clamp body before welding, so that welding occurs but is not continuous around the lid. Figure 10A Shows the infrared image of the non-welded lid, Figure 10B shows the infrared image after an additional lid has fallen off after welding, Figure 10C is the infrared image obtained when the technician removes the additional lid, Figure 10D is the infrared image after the lid has been removed. Figure 11 shows another defect detection through consecutive infrared images. In this case, the defect is a gross negligence caused by a scratch (gouge) inside the clamp body before welding, so that welding occurs but is not continuous around the lid. Figure 11A is the infrared image obtained before welding, Figure 11B is the infrared image obtained during welding, showing the temperature rise at the lid edge (see arrow), Figure 11C is the infrared image obtained after welding, showing no discontinuity at the edge (see arrow).

[0066] In another aspect, the sounds generated during the welding / sealing operation can be recorded and analyzed to determine manufacturing defects. Similar to the infrared images discussed above, the sound recordings during successful and failed clip runs can be analyzed to determine the audio threshold / range of successful clips and / or to determine the sound characteristics of non-failing clips. Notably, the microphone can include an ultrasonic range to detect sound variations associated with ultrasonic welding. Figure 12A Shows a welding station 510 where two welders (welder #1 and #2) are set up on an assembly line of a clip body to be welded to a lid. Figure 12B Illustrates an ultrasonic microphone 603 located near welder 511 at an ultrasonic welding station 530. In this embodiment, the microphone is a microphone with a 96 kHz bandwidth, but any suitable microphone can be used. The microphone is positioned appropriately close (e.g., ~2 inches) to the top of the clip where the welding occurs. Multi-Instrument software can be used to record sound data.

[0067] Figures 13 to 16 Shows the spectrograms and analyses of the ultrasonic microphone, which indicate the sound characteristics of the welding operation and can be used for the above-mentioned defect detection. Figure 13 Shows an amplitude spectrum comparison where the welding processes on welders #1 and #2 are detected, and different characteristics of the left and right welds can be seen. The fundamental frequencies are ~100 kHz apart, but as shown, the harmonic contents are very different. Figures 14A to 14B Shows an autocorrection analysis that improves the characterization of harmonics in the data. The second peak time delay is tracked, and the coefficients characterizing the autocorrelation spectrum are determined. This is a more robust method to characterize the response because the fundamental frequency is not always detected. Figure 14A Shows the amplitude spectrum and autocorrelation spectrum of welder #1. Figure 14B Shows the amplitude spectrum and autocorrelation spectrum of welder #2. Figure 15A and Figure 15B Show the autocorrelation spectra used to determine the weld-to-weld consistency of welders #1 and #2, respectively. The spectra show some differences, but the second peak time delay seems to be a consistent basis for identifying the welds. While some minor variations between welds can be expected, variations beyond a predetermined threshold / range may indicate a welding defect. Figure 16 Shows real-time tracking of welding parameters using the data logger feature in multi-instrument software. This can be used to determine the consistency of the detectability of 100-second welds verified for different welding energy levels and lid / clip combinations.

[0068] In another aspect, a high-resolution optical camera can be used to detect changes in visible features during or after the manufacturing process to identify defects in real time. Figures 17A to 17D andFigures 18A to 18D Shows an image from a high-resolution optical camera for detecting lid alignment issues and related system settings. As Figure 17E shown, the high-resolution camera 1130 (e.g., Cognex In-Sight 8402) is positioned to detect lid alignment from a rear view of the cartridge lid 100. The cartridge shoulder is referenced to determine the alignment of the lid relative to the cartridge body. Figures 17A to 17D Shows a series of images of different cartridges. The top two images ( Figures 17A to 17B ) show acceptable lid alignment (qualified), and the bottom two images ( Figures 17C to 17D ) show unacceptable lid alignment (unqualified) that exceeds a predetermined threshold. Figures 18A to 18D Shows another method by which a high-resolution optical camera is positioned from a top-down view for detecting lid misalignment defects. Figure 18A And Figure 18C demonstrate proper lid alignment (qualified), while Figure 18B and Figure 18D demonstrate incorrect lid alignment (unqualified). It will be appreciated that when viewed by a worker, various mismatch features may not be very obvious, so image analysis techniques can be used to identify the number and nature of lid alignment defects. Additionally, an ML / DL model can be used to determine the relationship of optical features associated with lid alignment defects. Further, it should be understood that this method can be used not only for lid alignment but also for various other functions of defect detection (e.g., weld seams, cartridge body structure, film seal integrity, etc.).

[0069] In another aspect, imaging can also include an optical RGB camera, which is a standard CMOS sensor for detecting light in the visible spectrum and generating an image through the RGB color model, which is an additive color model where the red, green, and blue primary colors of light are added together. The RGB camera can identify features that an infrared image may not be able to show, and vice versa. In some embodiments, the automated defect system and method utilize RGB and infrared images and can utilize any of the above methods to determine thresholds / ranges, including using images and sequential images in an ML model. In some embodiments, the automated method utilizes RGB and infrared images to obtain additional data from which an ML model can more accurately detect defects.

[0070] Figure 19AShows a lid welding station 510 with an RGB camera 2001 and an infrared camera 2000, which is pointed at a sample clip lid 100 on top of a clip body 200, and then the lid is welded to the clip body with an ultrasonic welding head 511. It should be understood that the RGB camera can equally be placed at various other positions / angles, for example, a top-down view at the post-welding position, such as imager 2001, can also be placed at the film heat-sealing station. The RGB and infrared cameras can be used to capture any of the following: the thermal signal of the welding plane of the welding station during the welding process, a high-resolution top-down optical image after welding, and the film thermal signal at the film station. Features can be extracted from one or more images using classical computer vision techniques (such as Histogram of Oriented Gradients). These images were obtained for multiple clips, which were subsequently subjected to standard seal testing and visual inspection. As a result, the computer vision system successfully identified clip defects associated with seal test failures using RGB and / or infrared images with a classifier model (such as Support Vector Machine). Defects that can be detected through these methods via RGB and / or infrared images include any of the following defects: broken shafts (i.e., cylindrical protruding lid features), cracked shafts, poor film seals, melted shafts, and insufficient welds. In some embodiments, broken or cracked shafts and insufficient welds are detected at the welding station, and poor film seals and melted shafts are detected at the film sealing station. Tests have shown that RGB and infrared images can identify defects associated with seal test failures that cannot be identified by manual visual inspection. Examples of RGB and infrared images capable of detecting these defects are as follows Figures 19A to 19D as shown

[0071] Figure 19B Shows a baseline infrared image of the lid, showing the thermal distribution of a successful weld, which was subsequently confirmed by standard seal testing and visual inspection Figures 19C to 19E Shows an infrared image of the lid that characteristically deviates from the baseline image, which was determined to be related to various defects, and was subsequently confirmed by seal testing and inspection Figure 19C Shows an infrared image showing the thermal distribution indicating a cracked shaft Figure 19D Shows an infrared image showing the thermal distribution indicating a melted shaft Figure 19E Shows an infrared image showing the thermal distribution indicating a brittle shaft

[0072] Figure 20Shows the experimental results of defect detection using image analysis as opposed to using standard manual inspection. RGB and infrared images enable the system to identify in-situ non-conformance patterns that otherwise could only be identified through extensive testing. Image analysis identifies broken and cracked shafts more effectively than a technician using a high-resolution optical microscope (such as a VHX). Infrared image analysis alone does not consistently identify seal test failures due to poor sealing; however, the additional data provided by a high-resolution RGB camera after membrane sealing better identifies such defects.

[0073] Figure 21A Shows a top-down RGB image of the lid (Image A1), a perspective infrared image at the welder (Image A2), and a computer vision analysis of the infrared image (Image A3) that are able to identify a broken shaft 8b through image analysis in the production line. The broken shaft was confirmed through subsequent standard seal testing and visual inspection.

[0074] Figure 21B Shows a top-down RGB image of the lid (Image B1), a perspective infrared image at the welder (Image B2), and a computer vision analysis of the infrared image (Image B3) that are able to identify another broken shaft 8b through image analysis in the production line, even though the broken shaft could not be identified during standard testing including visual inspection. Thus, RGB and infrared images are more sensitive than existing defect detection tests / inspections.

[0075] Figure 21C Shows an overhead RGB image of the lid (Image C1), a perspective infrared image at the welder (Image C2), and a computer vision analysis of the RGB image (Image C3) that are able to identify a cracked shaft 8c through image analysis in the production line. The broken shaft was confirmed through subsequent standard seal testing and visual inspection.

[0076] Figures 22A to 22B Shows a film seal anomaly detected through infrared images at the seal film station 530. Figure 22A Shows cold spots in the outer seal (around the shaft opening ring), Figure 22B Shows an incomplete seal at the inner seal (inside the shaft opening ring). Each of these film defects cannot be detected through standard film seal testing and visual inspection. Figure 23A Shows microscope images that can also be used for membrane seal analysis, specifically a passed membrane seal (shaft 102 in Image A1) and a failed membrane seal (shaft 8b in Image A2). These specific defects are in Figure 23BNot shown in the infrared images B1 and B2. Therefore, the film detection can be further improved by adding additional data sets, for example, by adding any of the following: RGB cameras after the sealing station, higher resolution infrared cameras at the sealing station, top-down infrared cameras after the sealing station, and visual shaft ratio estimation. Figure 24A Shows the RGB image (top view of the lid on the left), Figure 24B Shows the infrared image (perspective view of the welding station), which detects the molten shaft, a feature not identified by the standard seal test but found during subsequent visual inspection. The above experimental results show that image analysis using infrared and / or RGB images can generally identify certain defects associated with seal test failures, even defects that are typically undetected by standard seal tests and visual inspections. Therefore, using image analysis for automatic defect detection, especially using infrared images, RGB images, or images using both infrared and RGB simultaneously, can improve and make the detection of manufacturing defects more consistent.

[0077] Figure 25A Shows a schematic diagram of automatic defect detection based on the above imaging method. On the left, the system acquires an image or sequence of images (e.g., infrared / RGB images), extracts features, and compares the extracted features with the corresponding baseline images to detect defects associated with seal test failures. Figure 25B Shows a basic ML model through which defects (e.g., broken shafts) can be obtained with high precision from a series of infrared images. Figures 25A to 25B Refers to a modular defect detection algorithm that can combine computer vision for feature extraction (e.g., histogram of oriented gradients) and classical machine learning (e.g., using support vector machines to classify defects based on the extracted features). In some embodiments, deep learning is used in an end-to-end manner (combining feature extraction and detection through a deep neural network classifier). It is worth mentioning that both methods can be used.

[0078] It should be understood that the infrastructure of the manufacturing process affects the ability to acquire data for automatic defect detection using an ML model. Figure 26 Shows the current infrastructure limitations, where manual data transfer to the database and manual image capture feedback into the ML model. The features shown represent the historical infrastructure in the current system. Figure 27Shows an integrated manufacturing data infrastructure. In some embodiments, ML models are integrated into the data collection system and automated for defect detection in real-time during manufacturing, such that defective units can be removed during manufacturing rather than requiring testing after manufacturing is complete. Additionally, this approach allows for the quick identification of issues that may affect multiple consecutive units, which may have an adverse impact on an entire batch, so that any issues can be quickly resolved and production resumed, thereby avoiding rework and reducing defects and product waste. The historical infrastructure is represented by solid lines. The historical infrastructure can be integrated into an improved system that utilizes machine learning models. The current infrastructure can automatically collect and store data, providing a single source of truth for manufacturing data, and can be easily extended to integrate more tools and / or different data sources. The production infrastructure is represented by dashed lines. The production infrastructure provides a single central hub for data on the manufacturing plant floor, can publish real-time PLC data accessible via APIs and other open protocols, and automates production data streams and low-latency communication. Thus, the systems and methods provided herein ideally leverage existing historical and production infrastructures, utilize automated data to identify defects and / or classify clamps, and feed these classifications back into the production process.

[0079] Figure 26 Shows the current infrastructure of an automated manufacturing process, which has limited data collection available for automated defect detection. In this setup, manual data is still transferred from the plant floor control to the welder database, while additional image capture data is transferred manually back to model training. Data from the automation software is also fed back to the ML model. Thus, this approach can utilize imaging technology to provide improved defect detection with minimal or no changes to the existing manufacturing production line.

[0080] Figure 27 Shows a schematic diagram with an improved manufacturing infrastructure setup that is integrated with the automated defect detection capabilities described herein. In this infrastructure, real-time data and imaging data do not rely on manual transfer, but are transferred in real-time from the plant floor control to automated control software (such as Ignition), such that the real-time data is automatically transferred in real-time to the ML model for defect determination, which in turn is fed back as an output to the automated software so that the manufacturing process can be modified as needed or defective units can be automatically discarded without disrupting the ongoing manufacturing process. This automated data collection allows for scaling to big data, which is typically required for deep learning models. It should be noted that this is just one data infrastructure for integrating automated defect detection in an automated manufacturing process, and various other configurations are possible.

[0081] In the foregoing specification, the invention has been described with reference to its specific embodiments, but those skilled in the art will recognize that the invention is not so limited. The various features, embodiments, and aspects of the foregoing invention may be used singly or in combination. In addition, the invention can be used in any number of environments and applications beyond those described herein without departing from the broader spirit and scope of the specification. Accordingly, the specification and drawings are to be regarded as illustrative rather than restrictive. It should be recognized that the terms "comprising," "including," and "having" are to be read specifically as open-ended art terms. Any reference to a publication, patent, or patent application is hereby incorporated by reference in its entirety for all purposes.

Claims

1. A method for detecting defects in a sample card holder, characterized in that, The method includes: obtaining one or more data sets from one or more external sensors during the manufacturing process of the sample cartridge; comparing the one or more data sets with a baseline data set of the manufacturing process and / or the sample cartridge, the baseline being associated with an acceptable sample cartridge; and identifying defects of the sample cartridge based on variations of the one or more data sets from the baseline.

2. The method according to claim 1, characterized in that, During the manufacturing of the sample cartridge, defects are determined in real time in an automated process.

3. The method according to claim 1, characterized in that, Obtaining one or more data sets includes obtaining a plurality of images from one or more RGB cameras and / or infrared cameras, wherein the one or more data sets include thermal images.

4. The method according to claim 3, characterized in that, The identification based on variations utilizes an algorithm derived by machine / deep learning based on a plurality of data sets associated with acceptable sample cartridges and a plurality of data sets associated with cartridge defects.

5. The method according to claim 3, characterized in that, The method further includes: extracting features corresponding to features of the sample cartridge from the one or more images.

6. The method according to claim 3, characterized in that, The one or more data sets include a plurality of consecutive images obtained during the manufacturing process.

7. The method according to claim 1, characterized in that, The one or more data sets include a plurality of images from different viewpoints during the manufacturing process.

8. The method according to claim 1, characterized in that, The manufacturing process includes ultrasonically welding a lid device to a cartridge body of the sample cartridge.

9. The method according to claim 1, characterized in that, The manufacturing process includes heat-sealing a film on the lid device on the cartridge body of the sample cartridge.

10. The method according to claim 1, characterized in that, The one or more external sensors include RGB cameras.

11. The method according to claim 1 or 10, characterized in that, The one or more external sensors include infrared cameras.

12. The method according to claim 1, characterized in that, The one or more external sensors include ultrasonic microphones.

13. The method according to claim 12, characterized in that, The one or more data sets further include an ultrasonic audio spectrum, which is compared with a baseline of the ultrasonic audio of a successful weld, the characteristics of which include peaks and / or variations in the ultrasonic audio spectrum.

14. The method according to claim 1, characterized in that, The one or more external sensors include high-resolution cameras, and the high-resolution cameras obtain high-resolution optical images.

15. The method according to claim 1, characterized in that, The variation includes a deviation of the extracted feature from the corresponding feature of the baseline high-resolution image.

16. The method according to claim 1, characterized in that, Input the data set into a model, and the model classifies the sample cartridge as qualified or unqualified according to defect prediction.

17. The method according to claim 16, characterized in that, The model is trained based on supervised machine learning and unsupervised machine learning.

18. The method according to claim 16, characterized in that, The model accesses the data set through a cloud-based data sharing, training, and prediction platform.

19. The method according to claim 1, characterized in that, The data set is obtained from an automated module of an automated production line of the cartridge.

20. The method according to claim 1, characterized in that, The one or more external sensors include RGB cameras and infrared cameras.

21. A system for detecting defects in a sample cassette, characterized in that, The system includes: one or more external sensors, which are disposed at or adjacent to a manufacturing process station of a sample cartridge production line, wherein the one or more external sensors include any one of the following: RGB cameras, infrared cameras, or a combination thereof; and a processing unit, which is operably coupled to the one or more external sensors and communicatively connected to a control unit of automated manufacturing equipment of the production line, wherein instructions for performing automatic defect detection on the sample cartridge are recorded thereon, and the instructions include the following steps: Obtain one or more data sets from the one or more external sensors during the manufacturing process of the sample cartridge; Compare the one or more data sets with the manufacturing process and / or a baseline data set of the sample cartridge, the baseline being associated with an acceptable sample cartridge; and Identify defects of the sample cartridge based on variations of the one or more data sets from the baseline.

22. The system according to claim 21, characterized in that, The system is automated, and the processing unit is configured to determine defect detection in real time during the manufacturing of the sample cartridge.

23. The system according to claim 21, characterized in that, The processing unit is further configured to obtain one or more data sets including acquiring images from one or more RGB cameras and / or infrared cameras, wherein the one or more data sets include thermal images.

24. The system according to claim 21, characterized in that, The processing unit is further configured such that the identification of variations is by an algorithm derived by machine / deep learning based on multiple data sets associated with acceptable sample cartridges and multiple data sets associated with cartridge defects.

25. The system according to claim 21, characterized in that, The processing unit is further configured to: Extract one or more features corresponding to standard features of the sample cartridge from the one or more images.

26. The system according to claim 21, characterized in that, The one or more data sets include multiple consecutive images obtained during the manufacturing process.

27. The system according to claim 21, characterized in that, The one or more data sets include multiple images obtained from different viewpoints during the manufacturing process.

28. The system according to claim 21, characterized in that, The manufacturing process includes ultrasonically welding a lid device to the cartridge body of the sample cartridge.

29. The system according to claim 21, characterized in that, The manufacturing process includes heat-sealing a film on top of the lid device on the cartridge body of the sample cartridge.

30. The system according to claim 21, characterized in that, The one or more external sensors include RGB cameras.

31. The system according to claim 21 or 25, characterized in that, The one or more external sensors include infrared cameras.

32. The system according to claim 21, characterized in that, The one or more external sensors include ultrasonic microphones.

33. The system according to claim 27, characterized in that, The one or more data sets include an ultrasonic audio spectrum, which is compared with a baseline of the ultrasonic audio of a successful weld, characterized by peaks and / or variations in the ultrasonic audio spectrum.

34. The system according to claim 21, characterized in that, The one or more external sensors further include a high-resolution camera, which acquires high-resolution optical images.

35. The system according to claim 29, characterized in that, The variation includes a deviation of the extracted features from the corresponding features of the baseline high-resolution image.

36. The system according to claim 21, wherein The system is configured to automatically discard individual cartridges from the production line when a defect is detected, without discarding the entire batch.

37. The system according to claim 21, wherein The system is configured to input the data set into a model that classifies the sample cartridge as qualified or unqualified based on defect prediction.

38. The system according to claim 37, wherein The model is trained based on supervised machine learning and unsupervised machine learning.

39. The system according to claim 37, wherein The model accesses the data set through a cloud-based data sharing, training, and prediction platform.

40. The system according to claim 21, wherein The data set is obtained from an automated module of an automated production line of the cartridges.

41. The system according to claim 21, wherein The one or more external sensors include RGB cameras and infrared cameras.

Images