Systems and methods for visual anomaly detection
Patent Information
- Authority / Receiving Office
- WO · WO
- Patent Type
- Applications
- Current Assignee / Owner
- Filing Date
- 2026-02-05
- Publication Date
- 2026-08-13
Smart Images

Figure US2026014054_13082026_PF_FP_ABST
Abstract
Description
Electronically filed: February 5, 2026 Attorney Docket No: 10928-W001-SECSYSTEMS AND METHODS FOR VISUAL ANOMALY DETECTIONCROSS REFERENCE TO RELATED APPLICATIONS
[0001] This application claims the benefit of U.S. Provisional Patent Application 63,754,915, filed February 6, 2025.FIELD OF THE DISCLOSURE
[0002] The present application relates generally to the use of imaging systems and image analysis algorithms to detect anomalies, such as unexpected items, on or near a manufacturing line. More specifically, the present application relates to systems and methods for performing anomaly detection in biopharmaceutical processes and applications.BACKGROUND
[0003] The pharmaceutical industry is governed by stringent regulations and standards due to the critical nature of pharmaceutical products. Therefore, pharmaceutical manufacturers invest heavily in systems and equipment to monitor quality and compliance. However, conventional monitoring methods in pharmaceutical manufacturing may rely on manual intervention and may not scale efficiently to large manufacturing lines.
[0004] For example, human operators may perform line clearance. During manual line clearance, operators clear packaging lines after each lot, and manually inspect all areas of the manufacturing line to ensure no components or materials have remained at the conclusion of a process. This conventional procedure is time consuming, can require two-person verifications, and generally raises safety and ergonomic concerns for the human operators involved. Moreover, as both the physical inspection operations and the documentation completion of conventional procedures is almost entirely manual, these conventional procedures often introduce a significant amount of human error. As a result, these conventional line clearance procedures inevitably delay subsequent manufacturing operations, place operators in compromising / dangerous positions within the manufacturing line to conduct the manual inspections, yield mislabeled or cross-contaminated products from manual error, and / or result in hazardous conditions when the manual inspections miss or otherwise overlook stray objects in the manufacturing line. Other processes, done manually, suffer from similar deficiencies.
[0005] In some approaches, feature detection algorithms have been implemented in an attempt to automate the physical inspection of these manufacturing lines. However, given the vast number of different conditions in which a line clearance inspection may fail, it is difficult to train a feature detection algorithm that can detect each feature corresponding to a failureElectronically filed: February 5, 2026 Attorney Docket No: 10928-W001-SECcondition. Further still, training such a feature detection algorithm is a lengthy process and is not adaptable to changing conditions at the manufacturing site.
[0006] To address these and other concerns, the present disclosure provides anomaly detection systems and methods that are scalable and automated. Such systems and methods can be useful for manufacturing line clearance, as well as runtime monitoring and pre-runtime setup or configuration.SUMMARY
[0007] One aspect of the present disclosure provides a system for visual inspection of a manufacturing line. The manufacturing line can include multiple segments. The system can include a plurality of imaging devices. The imaging devices can be assigned to different manufacturing line segments to capture image data of the assigned segment during run-time of the manufacturing line. Processing circuitry (executing, e.g., a software application) can receive image data of assigned segments of the manufacturing line from the imaging devices. The processing circuitry can analyze the image data to identify operational states of the manufacturing line segments by applying a machine learning (ML) algorithm to a model (e.g., an ensemble model) trained with training data comprising a set of training images representing an assigned segment of an imaging device during run-time operation of the manufacturing line. The model can be configured to receive image data of the respective imaging device as input and to output an image classification associated with detection of a defect in the assigned segment. The processing circuitry can then receive subsequent image data as input and output image classifications associated with detection of a defect in the assigned segment.
[0008] In some aspects, the techniques described herein relate to a method for detecting anomalies in a manufacturing line comprising a plurality of segments, including: (1) receiving, from the at least one imaging device, image data of assigned segments of the manufacturing line; (2) analyzing the image data to identify operational states of the plurality of segments by applying a machine learning (ML) algorithm to a model (e.g., an ensemble model) trained with a plurality of training data comprising a set of training images representing an assigned segment for an imaging device during run-time operation of the manufacturing line, wherein the model is configured to receive image data of the respective imaging device as input and to output an image classification with detection of a defect in the assigned segment (when a defect is present) or an indication of the assigned segment as being in a state clear of defects (when no defect is present); and (3) responsive to identifying at least one defect in the plurality of segments, cause a display to present a notification regarding the at least one defect and indicating that no defects have been found otherwise.Electronically filed: February 5, 2026 Attorney Docket No: 10928-W001-SEC
[0009] In some aspects, the techniques described herein relate to one or more non-transitory, computer-readable media storing instructions that, when executed by processing hardware of a controller, cause the controller to : (1) receive, from the at least one imaging device, image data of assigned segments of the manufacturing line; (2) analyze the image data to identify operational states of the plurality of segments by applying a machine learning (ML) algorithm to a model (e.g., an ensemble model)trained with a plurality of training data comprising a set of training images representing an assigned segment for an imaging device during run-time operation of the manufacturing line, wherein the model is configured to receive image data of the respective imaging device as input and to output an image classification with detection of a defect in the assigned segment; and (3) responsive to identifying at least one defect in the plurality of segments, cause a display to present a notification regarding the at least one defect.
[0010] One aspect of the present disclosure provides a system for visual inspection of a manufacturing line, the manufacturing line comprising a plurality of segments. The system can include a program memory storing instructions that, when executed by one or more processing units, cause the system to: receive, from a plurality of imaging devices each assigned to a segment of the plurality of segments, image data of assigned segments of the manufacturing line; analyze the image data to identify an operational state of the plurality of segments by applying a machine learning algorithm to a model (e.g., an ensemble model) trained with a plurality of training data comprising a set of training images representing an assigned segment of a respective imaging device during run-time operation of the manufacturing line or during line clearance of the manufacturing line, wherein the model is configured to receive image data of the respective imaging device as input and to output an image classification associated with detection of an anomaly in the assigned segment; and responsive to identifying at least one anomaly in the plurality of segments, cause a display to present a notification regarding the at least one anomaly.
[0011] In some aspects, the techniques described herein relate to a method for detecting anomalies in a manufacturing line comprising a plurality of segments, each segment of the plurality assigned to at least one imaging device, including: (1) receiving, from the at least one imaging device, image data of assigned segments of the manufacturing line; (2) analyzing the image data to identify operational states of the plurality of segments by applying a machine learning algorithm to a model (e.g., an ensemble model )trained with a plurality of training data comprising a set of training images representing an assigned segment of a respective imaging device during run-time operation of the manufacturing line or during line clearance of theElectronically filed: February 5, 2026 Attorney Docket No: 10928-W001-SECmanufacturing line, wherein the model is configured to receive image data of the respective imaging device as input and to output an image classification with detection of an anomaly in the assigned segment; and (3) responsive to identifying at least one anomaly in the plurality of segments, causing a display to present a notification regarding the at least one anomaly.
[0012] In some aspects, the techniques described herein relate to one or more non-transitory, computer-readable media storing instructions that, when executed by one or more processing units, cause a system to: receive, from a plurality of imaging devices each assigned to a segment of a plurality of segments of a manufacturing line, image data of assigned segments of the manufacturing line; analyze the image data to identify an operational state of the plurality of segments by applying a machine learning algorithm to a model (e.g., an ensemble model) trained with a plurality of training data comprising a set of training images representing an assigned segment of a respective imaging device during run-time operation of the manufacturing line or during line clearance of the manufacturing line, wherein the model is configured to receive image data of the respective imaging device as input and to output an image classification associated with detection of an anomaly in the assigned segment; and responsive to identifying at least one anomaly in the plurality of segments, cause a display to present a notification regarding the at least one anomaly.BRIEF DESCRIPTION OF THE DRAWINGS
[0013] The skilled artisan will understand that the figures described herein are included for purposes of illustration and are not limiting on the present disclosure. The drawings are not necessarily to scale, emphasis instead being placed upon illustrating the principles of the present disclosure. It is to be understood that, in some instances, various aspects of the described implementations may be shown exaggerated or enlarged to facilitate an understanding of the described implementations. In the drawings, like primary characters throughout the various drawings generally refer to functionally similar or structurally similar components.
[0014] FIG. 1 is a simplified block diagram of an example system for performing visual anomaly detection in biopharmaceutical processes and applications, in accordance with various aspects disclosed herein.
[0015] FIG. 2A depicts an example implementation of an imaging device disposed within a manufacturing line to perform visual anomaly detection in accordance with various aspects disclosed herein.Electronically filed: February 5, 2026 Attorney Docket No: 10928-W001-SEC
[0016] FIG. 2B depicts an example implementation of a plurality of imaging devices disposed throughout manufacturing line equipment to perform visual anomaly detection, in accordance with various aspects disclosed herein.
[0017] FIG. 3 A and FIG. 3B illustrate mask locations in accordance with various aspects disclosed herein.
[0018] FIG. 3C depicts a heatmap in accordance with various aspects disclosed herein.
[0019] FIG. 4 is a flow diagram depicting an example method for performing visual anomaly (if present) detection in biopharmaceutical processes and applications, in accordance with various aspects disclosed herein.DETAILED DESCRIPTION
[0020] Generally, the systems and methods of the present disclosure may receive image data from an imaging device system including multiple imaging devices that can be placed at different points on a manufacturing line to capture images of assigned portions or segments of the manufacturing line and surrounding areas. The systems and methods of the present disclosure may include an imaging device system including multiple imaging devices that can be placed at different points on a manufacturing line to capture images of assigned portions or segments of the manufacturing line and surrounding areas. The imaging devices can capture images of manufacturing line equipment during operation and provide the images to machine learning models (e.g., ensemble models), which can include or implement anomaly detection algorithms. Each machine learning model may be adapted to one imaging device or segment of the manufacturing line (and one imaging device can have more than one dedicated machine learning model, e.g., an ensemble of models) or different models for each operational states, and be trained to recognize or categorize images pertaining to a normal or operational state of a portion or segment of the manufacturing line in various operating configurations.
[0021] A manufacturing line or a segment of a manufacturing line can have multiple normal or operational states. For example, a manufacturing line can have different machine configurations, different machine parts, different machine speeds, or different products on the manufacturing line. In additional examples, a manufacturing line can have different operational states including idle, setup, steady operation, and close down. Normal appearance in these operational states can vary depending on the operational state and configuration of a manufacturing line. If there is no change in the configuration of the manufacturing line (e.g., within batches), setup and close down operational states may look the same. However, if the configuration of the manufacturing line is changed (e.g., between batches or different products), the normal appearance of setup and closedown may be different. In someElectronically filed: February 5, 2026 Attorney Docket No: 10928-W001-SECembodiments, a different anomaly detection model can be trained for each operational state (e.g., different models for idle, set up, closedown, steady operation). Having different models for different operational states can be beneficial if there is insufficient training data to train a model for all operational states or if different components or portions of the manufacturing line are visible during different operational states.
[0022] Once the model is trained to understand the normal operational state of a segment of the manufacturing line, the algorithm can then be used to identify anomalies (e.g, abnormal conditions) in new images provided by the imaging device. For example, the anomalies may be indicative of non-conforming equipment or equipment having foreign objects or other anomalies present. Non-limiting examples of anomalies included dropped containers (e.g, a dropped autoinjector or syringe), containers or container parts (syringe, autoinjector parts), labels, broken glass, misaligned equipment, broken parts, unexpected human presence in manufacturing space, incorrect parts after a change, or missing parts after a change. Additional non-limiting examples of anomalies include any irrelevant or unexpected object (e.g., a piece of plastic or paper, gloves or goggles, syringes or syringe caps) left on the manufacturing line. When anomalies are detected, the anomalies can be classified as “Anomalous” at the time of inference and the system can notify a user (e.g., detection of an anomaly can be output to a user). When anomalies are detected, the anomalies can be classified as “Anomalous” at the time of inference with a heatmap overlay signaling the anomaly or defect.
[0023] In aspects of the disclosure, a relatively small number of images are needed as training data for an anomaly detection model because of the finite number of possible manufacturing line segments and operational states and configurations thereof. That is, because anomaly detection models can operate to detect deviations from a conforming (or “normal”) condition, an anomaly detection model may need only be trained with sufficient data to understand the normal condition(s). In some embodiments, an anomaly detection model can be trained exclusively with normal training images (e.g., a model may have a training set of exclusively normal images). In some embodiments, an anomaly detection model can include an ensemble of models. An ensemble model can combine the outputs of multiple individual models, each of which can be trained exclusively on normal images or on a combination of normal and anomalous training images. Whether or not an anomaly detection model is trained using anomalous images, anomaly detection models can subsequently be calibrated or validated using labeled data (e.g., validation and testing sets that include normal and anomalous images). In contrast to an anomaly detection model, a feature detection model relies on training data that indicate each type of possible defect, for which there may be very large number ofElectronically filed: February 5, 2026 Attorney Docket No: 10928-W001-SECdefects, and such defects may be unpredictable. Accordingly, training anomaly detection machine learning models with images representing normal operational states of manufacturing line segments may require less training data than similar training using images of defects and can even been trained on-site prior to executing a runtime process. Systems according to embodiments can detect a relatively wider range of anomalies and are sensitive to anomalies that are visually subtle, such as variations in equipment alignment, surface defects, minute contamination, and the presence of transparent materials (such as shards of broken glass). Systems as disclosed herein can be generalized to detect any category of anomaly (e.g., any foreign material or anomalous configuration of the manufacturing line), without training on those specific categories of anomalies or on specific types of defects. For example, the system disclosed herein can detect anomalies in an open-ended way, even if a particular category of anomaly is not predicted ahead of time. In contrast, systems where all detection models are trained with images of defects may only detect those specific defects for which the detection models have been trained. Such an approach tends to be reactive and may be unable to detect new categories of anomalies or defects not previously encountered. Further, training a defect detection model to detect transparent materials may be difficult due to the various ways the transparent surface may appear in the image data. In sum, an anomaly detection approach is able to be trained with less data and detect a greater number of anomalous conditions than a defect detection model. In some instances, an anomaly detection approach may be trained faster than a defect detection model.
[0024] The anomaly detection models disclosed herein can detect anomalies in a scene that includes a manufacturing line during runtime operation or during a line clearance process performed thereafter. During runtime operation and line clearance, there can be various changes in the environment and manufacturing line that are part of normal operation and therefore not anomalous. Identification of such variations that are within the range of normal operation are false positives. The anomaly detection models disclosed herein can be trained to account for variations within the range of normal operation. Non-limiting examples of such variations include changes in the environment (e.g., lighting), locations of parts of the manufacturing line (e.g., parts that move during runtime or line clearance), changes to parts of the manufacturing line (e.g., as part of maintenance or for changeover of product). Using training data that is representative of such variations during normal operation, the anomaly detection models disclosed herein can learn what is normal during runtime operation and / or line clearance. For example, for each segment of the manufacturing line, the training data can be collected over the entire runtime (e.g., with the segment undergoing all movements thatElectronically filed: February 5, 2026 Attorney Docket No: 10928-W001-SECwould occur during operation) and under various conditions that may be encountered during runtime (e.g., different lighting conditions, different amounts of blur, different replacement parts, different products). By using a robust set of training data representative of the variations that can occur during runtime and line clearance, false positives can be reduced.
[0025] In some embodiments, anomaly detection models (including individual models of an ensemble model) can undergo a training phase, a validation phase, and a testing phase. An image library can be split into a training set, a validation set, and a testing set for each of these phases. Training can be either unsupervised or supervised. Training data (e.g., a training set) can include images of multiple operational states. During a training phase, anomaly detection models can be trained using normal images (e.g., a training set) to learn the distribution and appearance of normal or operational states. For semi-supervised or unsupervised training, training data generally excludes anomalous data. For supervised training, training data (e.g., a training set) can include both normal and anomalous data. For supervised training, training data can be labeled. During validation and testing phases of both supervised and unsupervised training, labeled images (e.g., normal vs. anomalous) can be used to validate, calibrate, and / or test anomaly detection models (e.g., validation and test sets). For example, labeled data can be used to evaluate how well anomaly detection models distinguish anomalous images from the learned normal or operational distribution. The model can generate an “anomaly score” for each image and use the anomaly score to classify the image as normal or anomalous with a confidence level. The threshold anomaly score can be calibrated during the validation phase to determine when an image should be classified as normal or anomalous.
[0026] The images generated may be accessed within enterprise networks, manufacturing networks, edge computing networks, and / or private cloud servers, and stored for historical reference / records. Further, the systems and methods of the present disclosure are modular and scalable, such that any number of devices may be used on a single manufacturing line having models adapted for their respective monitored segment of the manufacturing line. These devices may be coordinated using on-premises or remote computer systems. The system design described herein is adaptable to various pharmaceutical manufacturing setups, where equipment and processes can vary significantly from one facility to another. The systems and methods of the present disclosure may include multiple imaging devices installed at select locations in, near, around, and / or otherwise proximate to the manufacturing line to monitor larger portions of the manufacturing line. The systems and methods of the present disclosure may also enable a user / operator to view the processes and line clearance operations associated with the manufacturing line in real-time through live imaging device feeds.Electronically filed: February 5, 2026 Attorney Docket No: 10928-W001-SECExemplary Systems
[0027] FIG. 1 is a simplified block diagram of an example system 100 for performing visual anomaly detection in a manufacturing line 160, which, for example, may package a drug product, although the system 100 or similar systems can implement other phases such as production, etc. In some embodiments, the system 100 includes standalone equipment, though in other embodiments the system 100 is incorporated into other equipment. While the system 100 is illustrated as performing a biomanufacturing process, one of ordinary skill in the art will understand that the present techniques and components of the system 100 may be applied to performing visual anomaly detection in other processes or fields. For example, the present techniques and components of the system 100 may be applied to manufacturing in food / beverage, automotive, electronic, chemical, and / or other industries.
[0028] The system 100 includes components of a computing device 110, and one or more imaging devices 162. Optionally, the imaging devices 162 can be arranged along, or provided as components of, segments of a manufacturing line 160. In example embodiments, the imaging devices 162 can each include, be assigned to, or correspond to specifically-adapted deep learning models described later herein. In FIG. 1, the computing device 110, the manufacturing line 160, and the training image data sources 150 are communicatively coupled via a network 170, which may be or include a proprietary network, a secure public internet, a virtual private network, and / or any other type of suitable wired or wireless network(s) (e.g., dedicated access lines, satellite links, cellular data networks, combinations of these, etc.). In embodiments where the network 170 comprises the Internet, data communications may take place over the network 170 via an Internet communication protocol. In some aspects, more or fewer instances of the various components of the system 100 than are shown in FIG. 1 may be included in the system 100. In some embodiments, the manufacturing line 160 can include several (ten, fifty or more depending on complexity and size of the manufacturing line 160) imaging devices 162 with corresponding dedicated or assigned models.
[0029] In some examples, the system 100 can include or comprise an edge computing system, and the computing device 110 can be considered an edge computer, with multiple computing devices 110 in the system 100. As used herein, the term “edge computing” encompasses many implementations of distributed computing that move processing activities and resources towards the “edge” of the network (e.g., processing that would otherwise have been done in a centralized cloud-based deployment can instead be performed on edge computers or on the imaging devices 162 themselves), in an effort to reduce latency and increase throughput for endpoint users (client devices, user equipment, etc.).Electronically filed: February 5, 2026 Attorney Docket No: 10928-W001-SECSuch edge computing implementations typically involve the offering of such activities and resources in cloud-like services, functions, applications, and subsystems, from one or multiple locations accessible via wireless networks. In some example embodiments, when the system 100 includes multiple edge computing devices 110, each edge computing device 110 can handle or be assigned to processing data for a subset of the multiple imaging devices 162 and corresponding adapted models 163.
[0030] In some example embodiments, one or more of the computing device(s) 110 can include or be implemented on a cloud-based platform such as, for example, Amazon Web Services (AWS). The cloud-based platform may also include a plurality of web-based services to perform services corresponding to the video / image data, security, machine learning, notifications, etc. For example, when web-based services include AWS, the web-based services may include, without limitation, AWS Greengrass security with Virtual Private Cloud (VPC), AWS SageMaker, AWS PyTorch, AWS loT Core, Amazon Cognito, etc. In some aspects, a computing device 110 may receive video / image data from the training image data sources 150 and imaging devices 162 over the network 170 and apply various algorithms / models to the video / image data to identify visual anomalies.
[0031] Further in these aspects, if computing device 110 identifies an anomaly (e.g., identifies that a foreign object is detected on a manufacturing line segment, or that machine configuration is incorrect / invalid, etc.), the computing device 110 may generate and / or cause a notification to be displayed to a user / operator as described with reference to the notification unit 140 later herein.
[0032] The manufacturing line 160 may include a single biomanufacturing process machine, or multiple biomanufacturing process machines that are either co-located or remote from each other and are suitable for producing or packaging biological products, such as drug products (e.g, a medicine or treatment) or drug substances (e.g, an active ingredient in a drug product). The manufacturing line 160 may generally include physical devices configured for use in producing (e.g., manufacturing) biological products (e.g., drug products), such as filling devices, agitating devices, starwheels or other vessel conveyances, syringe loading and verification components, vial inspection components, and so on.
[0033] The manufacturing line 160 may, in some embodiments, be connected with the computing device 110 either via the network 170, or directly, allowing for at least some of the functionality of the manufacturing line 160 to be controlled by the computing device 110. In some embodiments, the manufacturing line 160 may be capable of receiving instruction directly from a user (e.g., the manufacturing line 160 may be manually configurable). ForElectronically filed: February 5, 2026 Attorney Docket No: 10928-W001-SECexample, in some embodiments, the manufacturing line 160 may receive instructions directly from a user to control operation (e.g., start or stop operation).
[0034] The imaging devices 162 may be included in the manufacturing line 160 (e.g., integrated into the manufacturing line 160) or may be external devices connected to and / or otherwise located proximate to the manufacturing line 160. The imaging devices 162 may be used to collect video / image data of a respective segment inside, outside, and / or around the manufacturing line 160. For example, the imaging devices 162 may be oriented to capture image data of equipment that interacts with a product produced via the manufacturing line 160. In some examples, the imaging devices 162 are located in areas having an unobstructed field of view (FoV) of areas of interest, considering that equipment can move at various points during image capture. Imaging devices 162 may be mounted in areas that do not obstruct operation of the manufacturing line 160 and with access to networking and power cables. Imaging devices 162 may be mounted in areas that are not subjected to cleaning or that have frequently scheduled maintenance activities that would requiring moving the imaging devices 162 or that could result in accidental movement of the imaging devices 162. The imaging devices 162 may provide the video / image data to, for example, the computing device 110 (e.g., via the network 170) or to multiple computing devices 110 operating as edge computers in an edge network. The video / image data may be any suitable data type, such as real-time video data of a manufacturing line included as part of the manufacturing line 160, single image frames of the manufacturing line, and / or any other suitable data type or combinations thereof. In some embodiments, the imaging devices 162 associate the captured image data with metadata indicating an identity of the imaging device such that the computing device 110 is able to apply and / or train a machine learning model specific to the imaging device 162. The identity information may also be used to provide guidance to a user as to where an anomaly (e.g., a foreign object) is located in the manufacturing line 160 such that the user is able to perform the line clearance operation more efficiently.
[0035] In one mode of operation, the video / image data is collected when no products are being produced via the manufacturing line 160 for use as training data for machine learning models, as described later herein. This video / image data can be used for training, validating, and testing machine learning models. More particularly, the manufacturing line 160 may be configured to control the corresponding equipment in a manner to produce a particular product for an upcoming production run to generate image data indicative of a normal, specified, or “good” state of the corresponding segment during production of the product. For example, the computing device 110 may control the equipment of the manufacturing line 160 to moveElectronically filed: February 5, 2026 Attorney Docket No: 10928-W001-SECthrough a full range of motion according to a recipe (or other control instructions) for producing the product. Based on the collected data, the computing device may train an anomaly detection model using the training data such that deviations or anomalies relative to the normal state can be detected within video / image data generated during runtime operation of the manufacturing line 160 to produce the product and / or during a line clearance process performed thereafter.
[0036] The manufacturing line 160 may include one or more devices (not shown) used in manufacturing of biological products. The manufacturing line 160 may be configured to be controllable via manual or automated inputs. In some embodiments, the manufacturing line 160 may be configured to receive such control inputs locally, such as via a user input device local to biomanufacturing process machinery of the manufacturing line 160. In some embodiments, the manufacturing line 160 is configured to receive control inputs remotely, such as from the computing device 110 (e.g., via the network 170). The control inputs may include operation instructions, such as instructing the manufacturing line 160 to power on / begin operation. In some aspects, the manufacturing line 160 may end operation in response to one or more of: (i) the manufacturing line 160 completing production of biological product (e.g., a full batch of drug product is finished), or (ii) receiving a manual instruction to end operation. In some aspects, if foreign objects are detected during line clearance, an indication can be provided (e.g., to notify an operator) that manual clearing of the object is required. For example, a syringe may have fallen onto a conveyor belt during and been conveyed to an unexpected location, or a syringe or other component may be left on a manufacturing line after production / packaging has been completed or after a shift completion.
[0037] The training image data is provided by imaging devices 162 and generally includes training video / image data that may correspond to (e.g., may have been collected during performance of) one or more biomanufacturing processes for producing one or more pharmaceutical or biological products using the manufacturing line 160, although embodiments are not limited to biomanufacturing processes and can be implemented on other types of manufacturing lines. The training video / image data may represent segments, which can include (i) manufacturing line components or combinations of two or more manufacturing line components, (ii) a manufacturing line floor area, (iii) a manufacturing line interior (e.g., gaps between components, etc.), (iv) an individual portion of equipment, (v) a manufacturing station, and / or other suitable areas, subareas or portions of areas related to the manufacturing line. A segment in this context is a portion of a manufacturing line. As an example, a single packaging line can include 7 different pieces of individual equipment connected serially. Each of these 7 pieces of equipment can be considered their own segment. A segment could alsoElectronically filed: February 5, 2026 Attorney Docket No: 10928-W001-SECmean a sub-portion / subarea / subregion of a piece of equipment as well. Further, the training video / image data may have been collected (by computing device 110 or another device / system) using designated or assigned image device(s) 162 or other, similar sensors. In some aspects, the training video / image data includes image data corresponding to each component and region of the manufacturing line. As such, the imaging device(s) 162 may have a collective field of view (FOV) that includes each component and / or region of the manufacturing line, such that the models and algorithms described herein may be trained and / or otherwise configured to analyze subsequent run-time image data of any component or region of the manufacturing line based on the training video / image data. In some embodiments, the system 100 may omit the training image data sources 150, and instead receive the training video / image data locally, such as via user input at the computing device 110 (e.g., a user providing a portable memory drive with the training video / image data).
[0038] The computing device 110 is generally configured to input video / image data over a period of interest (e.g., during runtime operation or during a post-runtime line clearance operation) to at least one model / algorithm (e.g, trained using training video / image data) to analyze the video / image data to identify operational states of a plurality of manufacturing line segments. In example embodiments, a plurality of models can be trained, each adapted to one of the imaging devices 162 and / or manufacturing line segments, to identify operational states of the manufacturing line segment corresponding to that imaging device 162. For example, one imaging device 162 may capture images of segment A, and an adapted model can be trained with those images to recognize operational states of segment A. Components of the computing device 110 may be interconnected via an address / data bus or other means. The components included in the computing device 110 may include a processing unit 120, a network interface 122, a display 124, a user input device 126, and a memory 128, discussed in further detail below.
[0039] The processing unit 120 includes one or more processors, each of which may be a programmable microprocessor that executes software instructions stored in the memory 128 to execute some or all of the functions of the computing device 110 as described herein. Alternatively, one or more of the processors in the processing unit 120 may be other types of processors (e.g, application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), etc.).
[0040] The network interface 122 may include any suitable hardware (e.g., front-end transmitter and receiver hardware), firmware, or software configured to use one or more communication protocols to communicate with external devices or systems (e.g., the imagingElectronically filed: February 5, 2026 Attorney Docket No: 10928-W001-SECdevices 162, the manufacturing line 160, the training image data sources 150, other computing devices 110 in an edge network, etc.} via the network 170. For example, the network interface 122 may be or include an Ethernet interface.
[0041] The display 124 may use any suitable display technology (e.g., LED, OLED, LCD, etc. to present information to a user, and the user input device 126 may be a keyboard or other suitable input device. In some aspects, the display 124 and the user input device 126 are integrated within a single device (e.g., a touchscreen display). Generally, the display 124 and the user input device 126 may combine to enable a user to interact with graphical user interfaces (GUIs) or other (e.g., text) user interfaces provided by the computing device 110 (e.g., for purposes such as notifying users of line clearance / monitoring actions, etc.}.
[0042] The memory 128 includes one or more physical memory devices or units containing volatile or non-volatile memory and may or may not include memories located in different computing devices of the computing device 110. Any suitable memory type or types may be used, such as read-only memory (ROM), solid-state drives (SSDs), hard disk drives (HDDs), etc. The memory 128 may store instructions for one or more software applications included in a visual anomaly detection application 130 that can be executed by the processing unit 120. The memory 128 may be configured to store model data representative of dimensions of different manufacturing line segments. In the example system 100, the visual anomaly detection application 130 includes a data collection unit 132, a model training unit 134, a user interface unit 136, and a notification unit 140. The units 132-140 may be distinct software components or modules of the visual anomaly detection application 130 or may simply represent functionality of visual anomaly detection application 130 that is not necessarily divided among different components / modules. For example, in some embodiments, the data collection unit 132 and the user interface unit 136 are included in a single software module. Moreover, in some embodiments, the units 132-140 may be distributed among multiple copies of the visual anomaly detection application 130 (e.g., executing at different components in the computing device 110), or among different types of applications stored and executed at one or more devices of the computing device 110.
[0043] The data collection unit 132 is generally configured to receive data (e.g., video / image data, operator instructions, efc.). In some embodiments, the data collection unit 132 receives the training video / image data (e.g., including historical video / image data of a plurality of instances of the biomanufacturing process and corresponding historical video / image data) of a biomanufacturing process for producing a biological product. Training video / image data can be split into training, validation and testing sets. The data collection unitElectronically filed: February 5, 2026 Attorney Docket No: 10928-W001-SEC132 may receive the training video / image data via, for example, the training image data sources 150, user input received via the user interface unit 136 with the user input device 126, or other suitable means. In some embodiments, the data collection unit 132 may receive video / image data via, for example, the imaging devices 162, user input received via the user interface unit 136 with the user input device 126, or other suitable means. In some embodiments, the computing device 110 may receive at, for example, the data collection unit 132 an indication that a biomanufacturing process and / or a line clearance process has begun and one or more components of the computing device 110 may begin monitoring video / image data provided, e.g., by the imaging devices 162. In some aspects, the data collection unit 132 may apply preprocessing to received video / image data, for example, to ensure compatibility with the anomaly detection models. Some pre-processing can include resizing, re-orienting, color balancing, etc. applied to image data.
[0044] The model training unit 134 is generally configured to generate, train, validate, test, or apply models. In embodiments, a plurality of models are trained, each being adapted to an imaging device 162 or manufacturing line segment. The models may each be trained using training image data provided by the corresponding image device 162. The models may each be any suitable machine learning model, including an ensemble model, for identifying anomalies. For example, some models may use neural network architectures designed for high-accuracy anomaly detection in images. Visual anomaly detection can be done using a supervised or unsupervised learning, or a hybrid of the two (referred to as semi-supervised learning). For example, visual anomaly detection according to some embodiments can use semi-supervised learning wherein models can be trained on hundreds of “good” (normal) images of manufacturing line segments / components (provided or generated by imaging devices 162). In the context of embodiments, a “good” (normal) image corresponds to equipment / components clear of any anomalies or defects. For example, a “good” (normal) image can include an image with no unexpected or abnormal objects present. Models can subsequently be validated and testing using a combination of “good” (normal) and anomalous images. If new equipment or new parts are installed, or a different manufacturing process is to be executed, the models can be retrained by capturing several images of the changes, adding those images to the existing dataset, retraining the model, and redeploying the model.
[0045] Some example models can analyze the distribution of local features in image patches using a pre-trained deep neural network to extract these features. A pre-trained model may have been previously trained on a large datasets, with learned parameters saved for later use in extracting features (e.g., edges, comers, basic textures, and combinations therefor) fromElectronically filed: February 5, 2026 Attorney Docket No: 10928-W001-SECthe pre-training data set. After using the pre-trained neural network, the example model then models the distribution of extracted features to understand the “normal” state (e.g., in the context of embodiments the normal or operational state of a corresponding manufacturing line segment / component). During inference, deviations from this learned distribution are flagged as anomalies.
[0046] In some embodiments, a machine learning model of the present disclosure may be trained in a semi-supervised manner, wherein the training dataset may be assumed to only include normal data instances. In some embodiments, a machine learning model of the present disclosure may be trained in an unsupervised manner, wherein the training dataset may be assumed to only include normal data instances. In some embodiments, a machine learning model of the present disclosure may be trained in a supervised model, where the training data set includes both normal and anomalous data instances. In some embodiments, a machine learning model of the present disclosure is an ensemble model of individual models, each of which may be trained in a semi-supervised, unsupervised, or supervised manner. Whether a machine learning models is supervised or unsupervised, the validation data set and the testing data set include both normal and anomalous data.
[0047] Because anomaly detection models may be very sensitive to anomalous conditions, in some embodiments, the model training unit 134 may synthetically generate additional training data to make the set of training data more robust against false positives and false negatives. Such images can be referred to as augmented images. For example, images can be augmented by adjusting properties (e.g., brightness, contrast, color, noise) of existing images so that anomaly detection models can learn additional variability in imaging conditions. In this way, additional training data can be generated to represent the expected variations during runtime without collecting real image data for all variations, which would be costly and timeconsuming. The additional augmented images generated in this way can be used to generate additional normal images (for unsupervised or supervised training) or additional anomalous images (for supervised training or for validation and testing) from existing images of anomalies. For example, to this end, the model training unit 134 may introduce noise in particular regions of the training data to synthetically simulate slight variations in imaging device position. As one example, the model training unit 134 may introduce noise along the edges of the image data. As another example, the model training unit may implement an edge detection model to identify edges associated with equipment of the manufacturing line 160 and introduce noise proximate to the identified edges. As a result, the training data set has trainingElectronically filed: February 5, 2026 Attorney Docket No: 10928-W001-SECdata of the manufacturing line 160 from subtly distinct FOVs, thereby reducing the likelihood that slight deviations in imaging device position trigger a false positive.
[0048] In some embodiments, the model training unit 134 can generate synthetic images that add anomalies to normal images. Such synthetic anomalous images can improve anomaly detection by exposing models to a wider range of anomalous objections during training, validation, and / or testing. Synthetic anomalous images can be used during training (for supervised models), during validation (for supervised and unsupervised models), and / or during testing (for supervised and unsupervised models). Synthetic anomalous images can be generated by blending anomalous objects onto normal images. Various blending methods (e.g., Poisson blending and machine-leaming-based blending) can be used to generate synthetic images. This allows anomalous objects to be placed at arbitrary locations, including positions that may be difficult to reproduce in real-world data collection. In addition, a broader range of objects can be introduced as anomalies during synthetic image generation than would be practical in real-world data collection.
[0049] The machine-learning programs or algorithms may also include regression analysis, support vector machine (SVM) analysis, decision tree analysis, random forest analysis, K-Nearest neighbor analysis, naive Bayes analysis, clustering, reinforcement learning, coupled-hypersphere-based feature adaptation (CFA), conditional normalizing flows (CFLOW), deep feature kernel density estimation (DFKDE), deep feature model (DFM), discriminatively trained reconstruction embedding (DRAEM), generative adversarial network (GAN), GANomaly, patch distribution modeling (PADIM), PatchCore, reverse distillation, student-teacher feature pyramid matching for unsupervised anomaly detection (STPFM), openvocabulary methods, efficientAD, self-distillation with no labels (DINO) -based anomaly detection, SoftPatch, encoder-decoder reconstruction-error models and / or other machinelearning algorithms or techniques or combinations thereof. In some embodiments, due to the processing power requirements of training machine learning models, the selected model may be trained using additional computing resources (e.g., cloud computing resources) based upon data provided by external sources (e.g., the training image data sources 150). The training data may be unlabeled, or the training data may be labeled, such as by a human.
[0050] The machine-learning programs or algorithms can also include ensemble models. An ensemble model can combine the outputs of multiple individual machine-learning models to determine whether an image is anomalous or normal. In some embodiments, an ensemble model can be trained for an assigned segment of the manufacturing line. In these embodiments, individual models of an ensemble model can each produce a binary output (normal orElectronically filed: February 5, 2026 Attorney Docket No: 10928-W001-SECanomalous) and a corresponding confidence score. An individual model of an ensemble model can be supervised (trained on normal and anomalous images) or unsupervised (trained on normal images only). An ensemble model can include a combination of supervised and unsupervised models. An ensemble model generally includes at least one unsupervised model. Individual models of an ensemble model, whether supervised or unsupervised, can be validated and tested using both normal and anomalous images. Ensemble model approaches can include AND / OR logic, top-k aggregation, score-based ensembling, and majority voting to combine outputs of individual models.
[0051] Ensemble models can include combinations of different types of models. For example, an ensemble can include a combination of two or more of regression analysis, support vector machine (SVM) analysis, decision tree analysis, random forest analysis, K-Nearest neighbor analysis, naive Bayes analysis, clustering, reinforcement learning, coupled-hypersphere-based feature adaptation (CFA), conditional normalizing flows (CFLOW), deep feature kernel density estimation (DFKDE), deep feature model (DFM), discriminatively trained reconstruction embedding (DRAEM), generative adversarial network (GAN), GANomaly, patch distribution modeling (PADIM), PatchCore, reverse distillation, studentteacher feature pyramid matching for unsupervised anomaly detection (STPFM), openvocabulary methods, efficientAD, self-distillation with no labels (DINO) -based anomaly detection, SoftPatch, encoder-decoder reconstruction-error models. An ensemble model can include two or more of any machine learning model or two or more of any anomaly detection model.
[0052] Ensemble models can improve robustness and accuracy of the methods disclosed herein by combining complementary strengths of multiple individual models, thereby reducing sensitivity to any individual model’s errors or biases. As a result, ensemble models can more successfully generalize to new conditions by stabilizing predictions and lowering the risk of false positives / false negatives from edge cases. Although individual models of an ensemble are generally trained using the same training images, different types of models may be better at detecting different types of anomalies. For example, a model that recognizes differences in color may be better at detecting an anomaly related to a change in color (e.g., replacement of a part with a part of the wrong color). In another example, a model that leverages edge or line structure may be better at detecting foreign objects with sharp boundaries (e.g., tools).
[0053] Generally, individual models of an ensemble are trained, validated, and tested using the same image library to avoid data leakage. An image library can be split into nonoverlapping training, validation, and testing sets. For an ensemble of unsupervised models,Electronically filed: February 5, 2026 Attorney Docket No: 10928-W001-SECeach individual model is trained with the same training set, the same validation set, and the same testing set. For an ensemble of unsupervised models, the training set includes only normal images, and the validation and testing sets each include both normal and anomalous images. For an ensemble of both supervised and unsupervised models, the same image library is used for all individual models, but the split into training, validation and testing sets may be different for supervised and unsupervised models of the ensemble. This is because the training set for supervised models includes both normal and anomalous images, but the training set for unsupervised models includes only normal images. The validation sets for both supervised and unsupervised models include normal and anomalous images, but the validation sets differ because some of the anomalous images were included in the training set for the supervised modes of the model. The anomalous images in the training set of the supervised models of the ensemble can be included in the validation set of the unsupervised models of the ensemble, but not in the validation set of the supervised models of the ensemble. However, the same testing set (including both normal and anomalous images) is used for both unsupervised and supervised models of the ensemble.
[0054] The user interface unit 136 is generally configured to receive user input. In one example, the user interface unit 136 may generate a user interface for presentation via the display 124, and receive, via the user interface and user input device 126, user-input training video / image data to be used by the model training unit 134 when training the models, validating the models, and / or testing the models. In another example, the user interface unit 136 may receive, via a user interface and user input device 126, inputs to start operation of the manufacturing line 160 or the imaging devices 162. The user interface unit 136 may also be used to display information. For example, the user interface unit 136 may be used to display an indication that an anomaly or defect has been detected.
[0055] The anomaly detection application may apply or access the models trained by the model training unit 134 (or otherwise obtained by the computing device 110 as a pre-trained model) and / or another model / algorithm (e.g., a motion detection algorithm) when analyzing the video / image data to identify an anomaly. The notification unit 140 is generally configured to notify a user when an anomaly has been identified on the manufacturing line. In some example embodiments, the notification unit 140 may notify a user if line clearance was not performed correctly or sufficiently. For example, detection of an anomaly such as material remaining on the line, broken glass, or components away from a home or default position can indicate that line clearance was not performed correctly or sufficiently. The notification unit 140 may coordinate with the user interface unit 136 to display a notification. The notificationElectronically filed: February 5, 2026 Attorney Docket No: 10928-W001-SECunit 140 may send an electronic message (e.g., e-mail, text, etc.) with a notification to a user of the computing device 110 or an external computing device (e.g., a separate operator station not shown in FIG. 1). The operator workstation may be a computing device / system (e.g., a supervisory control and data acquisition (SCADA) system) that may be communicatively coupled to and / or otherwise configured to control operation of one or more components of the manufacturing line. In some aspects, the notification unit 140 may send control signals to stop operation of the manufacturing line 160 if an anomaly is detected. In some aspects, the notification may be stored (e.g., in the memory 128 or a database (not shown in FIG. 1)), possibly along with other data (such as operation data) related to the manufacturing line 160 that may be useful in diagnosing the cause of the anomaly or defect. In some aspects, notification can include a heatmap overlay, which indicates the location of any anomalies detected in the image to provide guidance when manually performing line clearance.
[0056] In some aspects, some or all of the functionalities of the visual anomaly detection application 130 may be provided by a third-party (i.e., not on the computing device 110). For example, the machine learning models and / or the other algorithms / models may be hosted by a third-party and the visual anomaly detection application 130 may access the machine learning models and / or the other algorithms / models remotely by sending data (e.g., the video / image data) and receiving data (e.g., an identification of an anomaly or defect). In such example, the functionality of the visual anomaly detection application 130 may be hosted by the third-party. Turning to a different example, the machine learning models may be trained by a third-party and the visual anomaly detection application 130 may receive the machine learning models remotely from the third-party (e.g., by the computing device 110 receiving one or more elements of the machine learning models, such as weights or architecture). In such an example, the functionality of the model training unit 134 may be hosted by the third-party. In other examples, one or more instances of functionality of any of the units 132-140 may be hosted by a third-party, on, for example, a remote server accessible via the network 170.Exemplary Imaging device Layout and Field of View (FOV)
[0057] As described earlier herein, methods according to some example embodiments can be used to perform automatic line cleaning and clearance. Line cleaning and clearance is a standardized procedure for ensuring equipment and work areas are free of products, documents, and materials from a previous manufacturing process. Line clearance helps operators avoid mislabeling or cross-contamination of finished products. Line clearance is conventionally done in a time-consuming, error prone manual process and system. Methods according to various embodiments address these and other concerns, including other manufacturing line concerns,Electronically filed: February 5, 2026 Attorney Docket No: 10928-W001-SECusing imaging devices with associated adapted machine learning models to automatically detect foreign objects on a manufacturing line.
[0058] FIG. 2A depicts an example implementation 200 of an imaging device 202 disposed within a manufacturing line to perform visual anomaly detection in accordance with various aspects disclosed herein. While the imaging device 202 is shown in FIG. 2A within the manufacturing line, the imaging device 202 can also be above the manufacturing line. Generally, the imaging device 202 may be assigned to a manufacturing line segment near, below, or proximate the imaging device 202 to capture image data within a FOV 206 of the imaging device 202. More generally, the imaging device 202 may be placed as needed within a system, and / or may be integrated within and / or affixed to the manufacturing line equipment 208. The imaging device 202 may continually capture video / image data, and this video / image data may be streamed or periodically uploaded to a processing device (e.g., computing device 110) for analysis. Continually in this context refers to images collected for the duration of a process (in a specific example: images are collected continuously at a high frame rate for the time duration of a full production run / batch of final drug product on a manufacturing line). This can include multiple hours of image collection. This is in contrast to other applications, in which one image is collected at a single point in time to facilitate a shorter process, like clearing the manufacturing line of goods before the next batch begins and collecting a single image after clearing the manufacturing line.
[0059] The imaging device 202 can be said to be assigned to a segment of the manufacturing line 208 (e.g., to a manufacturing line segment).
[0060] The imaging device 202 can include any suitable type of imaging device configured to capture video / image data. Due to the differences in lighting, available space, and other imaging parameters at various locations near and around the manufacturing line, the imaging device 202 may be of a type to capture video / image data that is useful for the subsequent analysis described herein to perform visual anomaly detection at that location. Thus, the imaging device 202 may include or comprise one of a variety of imaging devices and lens hardware that is specifically configured to monitor and capture video / image data at specific part(s) / area(s) of the manufacturing line.
[0061] The imaging device 202 may support or provide pixel binning, which combines data from multiple adjacent pixels to improve low-light performance by reducing noise. Pixel binning can enhance the ability of systems according to example embodiments to detect small anomalies or defects including surface contaminants on a manufacturing line segment, such as small pieces of broken glass, and the like.Electronically filed: February 5, 2026 Attorney Docket No: 10928-W001-SEC
[0062] The settings of the imaging device 202 can be adjusted to reduce levels of blur in video / image data. For example, the exposure time can be matched to the speed of motion of the manufacturing line.
[0063] The imaging device 202 may include, without limitation, a standard FOV camera with variable or fixed zoom, a specialized wide angle (e.g., 180°+) lens / camera, a gyroscopic style camera, and / or any other suitable imaging device type or combinations thereof. Generally, a standard FOV camera may be configured to capture video / image data corresponding to general observations of various stations / equipment in a manufacturing line. A specialized wide angle / area of observation camera may be specifically configured to observe and capture video / image data corresponding to areas of larger physical volumes, such as floor space under the manufacturing line and associated equipment. A gyroscopic style camera may be configured to capture video / image data that may correspond to observation regions in tighter spaces in and between equipment that other imaging devices are unable to properly capture.
[0064] The imaging device 202 can include or be coupled to an image processor to perform computations on images or videos and perform analysis. For example, the imaging device 202 can be associated with an edge computer or other computing device. The imaging device 202 can be assigned or in communication with the same edge computer as other imaging devices (not shown in FIG. 2A).
[0065] The imaging device 202 can have an associated ML model trained with training data. The associated ML model can include an ensemble model. The training data can include a set of training images representing an assigned manufacturing line segment during run-time operation of the manufacturing line 160 (FIG. 1). After being trained, the ML model receives image data as an input from the imaging device 202 and outputs an image classification associated with detection of an anomaly or defect in the assigned segment. Upon or subsequently to identifying a defect or anomaly, the system (e.g., an edge computer or other device associated with the imaging device 202 can present a notification regarding the at least one anomaly or defect. The display, training, and image classification are described in more detail later herein.
[0066] A plurality of similar imaging devices can be disposed throughout the manufacturing line to capture video / images. For example, FIG. 2B depicts an example implementation 220 of a plurality of imaging devices 224A-224F disposed throughout manufacturing line equipment 222 to perform visual anomaly detection, in accordance with various aspects disclosed herein. As illustrated in FIG. 2B, the manufacturing line equipment 222 may include multiple segments or stations 222A-222F, wherein manufacturingElectronically filed: February 5, 2026 Attorney Docket No: 10928-W001-SECcomponents are positioned and configured to perform operations / processes that result in the manufacture of a particular product. Each station 222A-222F may perform specific operations that contribute a portion of the overall manufacturing process, such that an unfinished product may enter station 222A and become incrementally completed at each station 222A-224F until a finished product exits station 222F.
[0067] At each station 222A-222F, the corresponding imaging device 224A-224F may capture video / image data corresponding to the specific manufacturing line equipment 222 components located at the respective stations 222A-222F. For example, the imaging device 224B may capture video / image data corresponding to the specific components located at station 222B. Moreover, it should be understood that while the example implementation 220 illustrated in FIG. 2B depicts a single imaging device (e.g., imaging devices 224A-224F) at each station 222A-222F, there may be multiple imaging devices 224A-224F at each station 222A-222F. In this manner, the multiple imaging devices 224A-224F may capture video / image data corresponding to multiple different FOVs and may thereby provide a more complete perspective of the manufacturing line equipment 222 and the surrounding areas to more effectively perform visual anomaly detection. For example, a first imaging device (e.g., imaging device 224C) positioned at station 222C may be positioned / oriented to include equipment / manufacturing components that are part of the manufacturing line equipment 222 within the FOV of the first imaging device, such that the first imaging device captures video / image data corresponding to the equipment / manufacturing components. A second imaging device positioned at station 222C may be positioned / oriented to include equipment / manufacturing components that are part of the manufacturing line equipment 222 within the FOV of the second imaging device, such that the second imaging device captures video / image data corresponding to that piece of equipment / manufacturing components.
[0068] Some assigned segments may particularly relate to or include a component subject to modification during a manufacturing process of the manufacturing line or to replacement after a manufacturing process of the manufacturing line. Such segments may be more likely to vary with product changeover (e.g. changeover from one product to another product), although embodiments are not limited thereto. Examples can include base plates, labelers, cartoners, packers, packaging devices, etc., or components having background plates. In another example, when a part of manufacture is replaced during maintenance, the replacement part may have different dimensions or a different color. Imaging devices assigned to these segments may be configured to capture image data of the segment after at least one modification or replacement. This data can be used to train, validate, and test an anomaly detection model. By capturingElectronically filed: February 5, 2026 Attorney Docket No: 10928-W001-SECimage data for each modification or replacement, the anomaly detection model assigned to the segment can be trained to recognize each operational state associated with the modification or replacement as normal. In this way, anomaly detection models as disclosed herein can reduce the frequency of false positives in segments that are modified or replaced frequently.
[0069] Separate models may be trained or adapted to each modification or replacement. For example, an imaging device may capture images of a labeler for product A and images of a labeler for product B, and a separate machine learning model may be trained to recognize operational states of the labeler for each of product A and product B. For example, there can be different operational states for when the manufacturing line is operating with product A vs product B. The labeler for product A and the labeler for product B may include base plates or backgrounds having different colors, and training data can include training images for each color so that machine learning models can determine normal and anomalous figures in each configuration and color. In some examples, masked regions can be defined in areas where frequent modification and replacement is expected or is to be performed. Training separate models for each modification or replacement can also reduce false positives when the manufacturing line is modified (e.g., during product changeover).Exemplary Visual Anomaly Detection
[0070] Referring again to FIG. 1, the visual anomaly detection application 130 receives image data of assigned segments of the manufacturing line from the imaging devices described with reference to FIG. 2A and FIG. 2B. The visual anomaly detection application 130 can analyze the image data to identify operational (e.g., “normal”) states of one or more of the manufacturing line segments.
[0071] In some examples, the visual anomaly detection application 130 can apply an ML algorithm to one or more models. These models can each be trained with training data including training images representing an assigned segment of an imaging device during run-time operation of the manufacturing line. Each model can include an ensemble model. Because each model is adapted to one imaging device and assigned segment, that model is trained to recognize operational states of that associated segment. Furthermore, because each model was trained with image data provided by the imaging devices prior to and / or during run-time of the manufacturing line, each model can recognize a normal state of the manufacturing line segment, so that variation from a normal state can be detected and noted. In some examples, training data can be collected through range of motion of the manufacturing line equipment associated with at least one segment of the manufacturing line. For example, segments thatElectronically filed: February 5, 2026 Attorney Docket No: 10928-W001-SECtypically move (e.g., conveyor belts or robot arms) can have images captured throughout the range of motion.
[0072] Upon being trained to recognize an operational state, the model can receive image data of the respective imaging device as input and output an image classification associated with detection of an anomaly or defect in the assigned segment.
[0073] In some example embodiments, the visual anomaly detection application 130 or other component of the system 100 can receive or generate a mask that defines a masked region corresponding to a location in a segment of the manufacturing line. This allows ML models to ignore specific areas that may be subject to movement that can trigger a false positive. For example, a mask can be applied to location subject to movement that can trigger a false positive (e.g., a location where people walk by or a part that is frequently changed), and the ML model can be applied to the region outside the mask. As another example, if the anomaly detection model is implemented during runtime, a false positive can be detected in the portions of the image data where product is typically positioned in a manufacturing line segment, or where product is expected to be present. It should be appreciated that the same imaging device and / or segment may be associated with different masks for usage during runtime operations and line clearance processes. To this end, the mask for runtime operations may include the same masked regions as the line clearance mask, but with additional masked regions for the portions of the image data at which a product may be present during runtime operations.
[0074] FIG. 3A and FIG. 3B illustrate example mask locations. In FIG. 3A, a masked region can be provided at 300 and 302, where motion is expected to occur under components 304. In FIG. 3B, the masked region 306 is enlarged to prevent false detection that was made at point 308. Masked regions can include any areas that can be periodically moved by operators, such as base plates. Other example masked regions can include areas of a temporary anomaly or defect or noncritical anomaly or defect detected during runtime for the respective segment of the manufacturing line, such as a scratch in metal or other surface of the respective segment. To this end, some anomalies or defects may be minor enough to not warrant shutting down production, but still may need to be addressed during line clearance. Accordingly, the mask may be dynamically updated during runtime to avoid continued triggering of the anomaly detection while the runtime process completes. In some example embodiments, the system can detect a threshold number of false anomaly detections and generate an alert to a user to decide whether to include a masked region at a location corresponding to the false anomaly detection. Training data can include images with the masks in place, so that the models can recognize operational states that are normal despite the presence of the masked regions.Electronically filed: February 5, 2026 Attorney Docket No: 10928-W001-SEC
[0075] The system can also generate a heatmap overlay of image data to indicate which portion(s) of an image contributed the most to an inference that an anomaly is present in that image. In this way, a heatmap can visualize area(s) of an image where the model is confident about the presence of an anomaly. FIG. 3C depicts a heatmap in accordance with embodiments. The heatmap can indicate to a user where on the manufacturing line the anomaly is located, for example, so that the user can remove or fix the anomaly. The heatmaps can also indicate locations that contribute to false positives and therefore be used to improve the model. For example, if the heatmap indicates a region where there is variation in lighting or modification to the manufacturing line, additional training data can be collected or generated to represent the variations in that region. In this way, the model can learn that the variation in the region indicated by the heatmap is normal and not an anomaly. In another example, a mask can be applied to the region indicated by the heatmap so that the model is not applied to that region.
[0076] In embodiments where the model is an ensemble model, individual heatmaps can be generated for each individual model of the ensemble. Individual heatmaps can be used to indicate locations to contribute to false positives for an individual model. Individual heatmaps can be used to indicate which individual models contribute to detecting various anomalies. This information can be used in selection of individual models for an ensemble. For an ensemble model, the individual heatmaps can be merged to generate a composite heatmap representative of the ensemble.
[0077] It should be appreciated that, while FIG. 3C shows the heatmap portions 364A1-364A2, 364B1, and 364C1-364C2 in grayscale shading and / or a patterning, the heatmap portions 364A1-364A2, 364B1, and 364C1-364C2 are in some embodiments portrayed using color-coding. Regardless, in the example visual anomaly detection action 360 of FIG. 3C, the visual anomaly detection application 130 may receive a first image 362 that features a portion of a manufacturing line during normal operation with an identified unexpected object 362 A. In this example visual anomaly detection action 360, the visual anomaly detection application 130 may have executed an anomaly detection algorithm on the first image 362 to identify the unexpected object 362 A.
[0078] Upon identification of the unexpected object 362A, the visual anomaly detection application 130 may access video / image data from some / all of the other imaging devices captured at the same or similar timestamp as the first image 362 and may execute anomaly detection algorithms on the video / image data to identify any additional unexpected objects. In certain aspects, the system may determine that at least one anomaly is a false anomaly based on a user indication (e.g., user feedback). The system can then identify / implement / define atElectronically filed: February 5, 2026 Attorney Docket No: 10928-W001-SECleast one masked region based on the heatmap associated with the false anomaly. The first GUI image 364A may correspond to the first image 362 when the visual anomaly detection algorithm 130 applies an algorithm configured to generate a heatmap graphical overlay on the first image 362 to generate the heatmap portions 364A1 and 364A2. These heatmap portions 364A1 and 364A2 may correspond to portions of the first GUI image 364A that may include an unexpected object, and the unexpected object 362 A may be represented in the first heatmap portion 364A1. Similarly, the second and third GUI images 364B and 364C may include multiple heatmap portions 364B1, 364C1, and 364C2 that also correspond to portions of the GUI images 364B and 364C that may include an unexpected object. The visual anomaly detection application 130 may analyze the fourth GUI image 364D, and may not detect any unexpected objects, such that the fourth GUI image 364D may not include a heatmap graphical overlay.
[0079] In certain aspects, the heatmap graphical overlay may also indicate historical regions of the respective FOVs represented by the images of the example visual anomaly detection GUI 364 that have included identified unexpected objects. Accordingly, the first image 362 may influence the historical heatmap graphical overlay represented by the first GUI image 364A by causing the visual anomaly detection application 130 to update the location and / or the depth of color / patterning / etc. representing the heatmap portions 364A1 and 364A2 based on the identified unexpected object 362A within the first image 362. Moreover, the heatmap graphical overlay included as part of the first GUI image 364 A, the second GUI image 364B, and the third GUI image 364C may indicate areas within the manufacturing line that may have been the cause of an unexpected object (e.g., unexpected object 362A) within an image (e.g., first image 362).Exemplary Flow Diagram
[0080] FIG. 4 is a flow diagram depicting an example method 400 for performing visual anomaly detection in biopharmaceutical processes and applications, in accordance with various aspects disclosed herein. The method 400 may be implemented by one or more components of the system 100, such as the processing unit 120 when executing instructions of the visual anomaly detection application 130, and possibly also the manufacturing line 160 (which may be operating a biomanufacturing process). The method 400 may be or include analysis that is the same as or similar to the example visual anomaly detection actions performed in FIGs. 3 A-3C. The example method 400 may generally include the following elements: (1) receiving image data of assigned segments of a manufacturing line (block 402) from image sensors, (2)Electronically filed: February 5, 2026 Attorney Docket No: 10928-W001-SECanalyzing the image data to identify operational states of the segments (block 404), and (4) causing a display to present a notification regarding an anomaly or defect (block 406).
[0081] The method 400 can begin with operation 402 with a processing unit 120 receive, from a plurality of imaging devices, image data of assigned segments of the manufacturing line. Each imaging device can provide image data to a model adapted to the respective imaging device. A model can include an ensemble of models.
[0082] The method 400 can continue with operation 404 with the processing unit 120 analyzing the image data to identify operational states of the plurality of segments. Analysis can include applying a machine learning (ML) algorithm to a model trained with a plurality of training data comprising a set of training images representing an assigned segment of an imaging device during run-time operation of the manufacturing line. The model can be configured to receive image data of the respective imaging device as input and to output an image classification associated with detection of an anomaly or defect in the assigned segment. The ML algorithm can include an anomaly detection algorithm. The model can include an ensemble model. Training the model can comprise performing unsupervised learning. The set of training images can consist exclusively of normal images (e.g., for unsupervised learning). Training the model can comprise performing supervised learning. The set of training images can include both normal and anomalous images (e.g., for supervised learning). An ensemble models can include a plurality of individual models trained by a combination of unsupervised and supervised learning. The model can be validated with a set of validation images that include both normal and anomalous images.
[0083] The method 400 can continue with operation 406 with the processing unit 120 causing a display to present a notification regarding the at least one anomaly or defect. The display can include a heatmap overlay of image data to indicate location of at least one anomaly, wherein the heatmap indicates which portions of the image data contributes most to an inference made by the model of the presence of the at least one anomaly. For an ensemble model, separate heatmaps can be generated for each individual model of the ensemble. The heatmaps or other portion of the display / notification can include identification information for the segment. Identification information can be provided by metadata included for each imaging device. When an ML model associated with an imaging device triggers an anomaly detection, therefore, notifications can include identification information for the appropriate imaging device and from that identification information the associated manufacturing line segment can be identified. Other information associated with the manufacturing line segment can be provided in notifications, for example, instructions for clearing or otherwise maintaining orElectronically filed: February 5, 2026 Attorney Docket No: 10928-W001-SECperforming quality control on the respective manufacturing line segment. The method 400 can include receiving a user indication that the at least one anomaly is a false anomaly. At least one mask can be identified based on the indicated false anomaly.
[0084] The method 400 can further include collecting training data throughout a range of motion of at least one segment of the manufacturing line. The method 400 can further include collecting training data, including image data provided by the plurality of imaging devices, before manufacturing product is provided to the manufacturing line, to capture training data through a full range of motion of at least one respective segment.
[0085] The method 400 can include receiving or generating a mask that defines a masked region corresponding to a location in a segment of the manufacturing line. The masked region can be at a temporary anomaly or defect or noncritical anomaly or defect in the segment of the manufacturing line. The temporary anomaly or defect or noncritical anomaly or defect can include a scratch in the segment of the manufacturing line. The masked region can correspond to a location of an object for performing maintenance on the segment or other portion with a range of the imaging device. The masked region can correspond to a location of a manufacturing product within the segment. The method 400 can include detecting a threshold number of false anomaly detections and generating at least one additional mask at a location based on at least one false anomaly detection. Defect information, model information, anomaly information, audit ifonrmation, etc. can be stored in database storage or other memory.
[0086] The method 400 can include modifying pixel data within the image data to adjust for periodic or intermittent lighting changes in the image data.Example Embodiments
[0087] Without limitation, some example embodiments / features of the present disclosure include:
[0088] El. A system for visual inspection of a manufacturing line, the manufacturing line comprising a plurality of segments and the system comprising: a program memory storing instructions that, when executed by one or more processing units, cause the system to: receive, from a plurality of imaging devices each assigned to a segment of the plurality of segments, image data of assigned segments of the manufacturing line; analyze the image data to identify an operational state of the plurality of segments by applying a machine learning algorithm to a model trained with a plurality of training data comprising a set of training images representing an assigned segment of a respective imaging device during run-time operation of the manufacturing line or during line clearance of the manufacturing line,Electronically filed: February 5, 2026 Attorney Docket No: 10928-W001-SECwherein the model is configured to receive image data of the respective imaging device as input and to output an image classification associated with detection of an anomaly in the assigned segment; and responsive to identifying at least one anomaly in the plurality of segments, cause a display to present a notification regarding the at least one anomaly.
[0089] E2. The system of El, wherein the system comprises the plurality of imaging devices, wherein imaging devices of the plurality of imaging devices are configured to capture image data of the assigned segments during run-time of the manufacturing line or during line clearance of the manufacturing line.
[0090] E3. The system of any one of E1-E2, wherein each imaging device of the plurality of imaging devices provides image data to a model adapted to the respective imaging device.
[0091] E4. The system of any one of E1-E3, wherein the model is an ensemble model.
[0092] E5. The system of any one of E1-E4, wherein the instructions further cause the one or more processing units to: collect training data, including image data provided by the plurality of imaging devices, during run-time of the manufacturing line or during line clearance of the manufacturing line.
[0093] E6. The system of E5, wherein to collect training data, the instructions further cause the one or more processing units to: collect training data throughout a range of motion of at least one segment of the manufacturing line.
[0094] E7. The system of any one of E1-E6, wherein the instructions further cause the one or more processing units to: collect training data, including image data provided by the plurality of imaging devices, before manufacturing product is provided to the manufacturing line, to capture training data through a full range of motion of at least one respective segment.
[0095] E8. The system of any one of E1-E7, wherein the set of training images consists of normal images.
[0096] E9. The system of any one of E1-E7, wherein the set of training images comprises normal images and anomalous images.
[0097] E10. The system of any one of E1-E9, wherein the instructions further cause the one or more processing units to: receive or generate a mask that defines a masked region corresponding to a location in a segment of the manufacturing line.Electronically filed: February 5, 2026 Attorney Docket No: 10928-W001-SEC
[0098] Ell. The system of E10, wherein the masked region is at a temporary anomaly or noncritical anomaly in the segment of the manufacturing line.
[0099] E12. The system of El 1, wherein the temporary anomaly or noncritical anomaly includes a scratch in the segment of the manufacturing line.
[0100] E13. The system of any one of E10-E12, wherein the masked region corresponds to a location of an object for performing maintenance on the segment or another portion within a range of an imaging device.
[0101] E14. The system of any one of E10-E13, wherein the masked region corresponds to a location of a manufacturing product within the segment.
[0102] E15. The system of any one of E10-E14, wherein the instructions further cause the one or more processing units to: detect a threshold number of false anomaly detections; and generate at least one additional mask at a location based on at least one false anomaly detection.
[0103] E16. The system of any one of E1-E15, wherein the instructions further cause the one or more processing units to: modify pixel data within the image data to adjust for periodic or intermittent lighting changes in the image data.
[0104] E17. The system of any one of E1-E16, wherein the machine learning algorithm comprises an anomaly detection algorithm.
[0105] E18. The system of any one of E1-E17, wherein training the model comprises performing unsupervised learning.
[0106] E19. The system of any one of E1-E18, wherein training the model comprises performing supervised learning.
[0107] E20. The system of any one of E1-E19, wherein training the model comprises training an ensemble model.
[0108] E21. The system of E4 or E20, wherein the ensemble model comprises individual models trained by supervised learning and models trained by unsupervised learning.
[0109] E22. The system of any one of E1-E21, wherein the program memory stores instructions that, when executed by one or more processing units, cause the system to receive a plurality of synthetic images that add anomalies to normal images.
[0110] E23. The system of E22, wherein the set of training images comprises a portion of the plurality of synthetic images.Electronically filed: February 5, 2026 Attorney Docket No: 10928-W001-SEC[oni] E24. The system of any one of E22-E23, wherein the model was validated with a set of validation images comprising a portion of the synthetic images.
[0112] E25. The system of any one of E1-E24, wherein the instructions further cause the one or more processing units to: generate a heatmap overlay of image data to indicate location of at least one anomaly, wherein the heatmap indicates which portions of the image data contributes most to an inference made by the model of presence of the at least one anomaly.
[0113] E26. The system of E25, wherein the at least one anomaly is determined to be a false anomaly based on a user indication.
[0114] E27. The system of E26, wherein the instructions further cause the one or more processing units to identify at least one mask based on the heatmap associated with the false anomaly.
[0115] E28. The system of any one of E1-E27, wherein at least one assigned segment of the assigned segments includes a component subject to modification during a manufacturing process of the manufacturing line or to replacement after a manufacturing process of the manufacturing line.
[0116] E29. The system of E28, wherein an imaging device assigned to the segment including the component is configured to capture image data of the segment after at least one modification or replacement.
[0117] E30. The system of E29, wherein the imaging device provides image data after the at least one modification or replacement for analysis by a separate model adapted to each modification or replacement.
[0118] E31. The system of E28, wherein the component subject to modification is configured for processing different manufacturing line products for at least two different products.
[0119] E32. The system of E31, wherein the component subject to modification or replacement comprises a labeler.
[0120] E33. The system of E32, wherein the labeler includes a base plate having one of at least two colors based on properties of a label applied by the labeler.
[0121] E34. The system of E31, wherein the component subject to modification or replacement comprises a packaging device.Electronically filed: February 5, 2026 Attorney Docket No: 10928-W001-SEC
[0122] E35. The system of E31, wherein the component subject to modification or replacement includes a background plate.
[0123] E36. The system of E35, wherein the background plate has one of at least two colors based on a color of a product being produced in the manufacturing line.
[0124] E37. The system of any one of E1-E36, further comprising at least one edge computing device.
[0125] E38. The system of E37, wherein the at least one edge computing device is coupled to receive outputs of machine learning algorithms associated with at least two imaging devices of the plurality of imaging devices.
[0126] E39. The system of any one of E1-E38, further comprising: database storage configured to store anomaly information and audit information of the system.
[0127] E40. The system of any one of E1-E39, wherein the anomaly includes surface contamination on a surface of the segment.
[0128] E41. The system of E40, wherein the surface contamination includes broken glass.
[0129] E42. The system of any one of E1-E39, wherein the anomaly includes an object that is not an expected component of the manufacturing line.
[0130] E43. The system of E42, wherein the object includes a syringe.
[0131] E44. The system of any one of E1-E43, wherein the model is configured to, during configuration of the manufacturing line: output detection of a misalignment within the assigned segment.
[0132] E45. A method for detecting anomalies in a manufacturing line comprising a plurality of segments, each segment of the plurality assigned to at least one imaging device, the method comprising: receiving, from the at least one imaging device, image data of assigned segments of the manufacturing line; analyzing the image data to identify operational states of the plurality of segments by applying a machine learning algorithm to a model trained with a plurality of training data comprising a set of training images representing an assigned segment of a respective imaging device during run-time operation of the manufacturing line or during line clearance of the manufacturing line, wherein the model is configured to receive image data of the respective imaging device as input and to output an image classification with detection of an anomaly in the assigned segment; and responsive toElectronically filed: February 5, 2026 Attorney Docket No: 10928-W001-SECidentifying at least one anomaly in the plurality of segments, causing a display to present a notification regarding the at least one anomaly.
[0133] E46. The method of E45, wherein each imaging device provides image data to a model adapted to the respective imaging device.
[0134] E47. The method of any one of E45-E46, wherein the model is an ensemble model.
[0135] E48. The method of any one of E45-E47, further comprising: collecting training data, including image data provided by the at least one imaging device, during run-time of the manufacturing line or during line clearance of the manufacturing line.
[0136] E49. The method of E48, wherein collecting training data comprises: collecting training data throughout a range of motion of at least one segment of the manufacturing line.
[0137] E50. The method of any one of E46-E49, further comprising: collecting training data, including image data provided by the at least one imaging device, before manufacturing product is provided to the manufacturing line, to capture training data through a full range of motion of at least one respective segment.
[0138] E51. The method of any one of E45-E50, wherein the set of training images consists of normal images.
[0139] E52. The method of any one of E45-E50, wherein the set of training images comprises normal images and anomalous images.
[0140] E53. The method of any one of E45-E52, further comprising: receiving or generating a mask that defines a masked region corresponding to a location in a segment of the manufacturing line.
[0141] E54. The method of E53, wherein the masked region is at a temporary anomaly or noncritical anomaly in the segment of the manufacturing line.
[0142] E55. The method of E54, wherein the temporary anomaly or noncritical anomaly includes a scratch in the segment of the manufacturing line.
[0143] E56. The method of any one of E53-E55, wherein the masked region corresponds to a location of an object for performing maintenance on the segment or another portion within a range of an imaging device.
[0144] E57. The method of any one of E53-E56, wherein the masked region corresponds to a location of a manufacturing product within the segment.Electronically filed: February 5, 2026 Attorney Docket No: 10928-W001-SEC
[0145] E58. The method of any one of E53-E57, further comprising: detecting a threshold number of false anomaly detections; and generating at least one additional mask at a location based on at least one false anomaly detection.
[0146] E59. The method of any one of E45-E58, further comprising: modifying pixel data within the image data to adjust for periodic or intermittent lighting changes in the image data.
[0147] E60. The method of any one of E45-E59, wherein the machine learning algorithm comprises an anomaly detection algorithm.
[0148] E61. The method of any one of E45-E60, wherein training the model comprises performing unsupervised learning.
[0149] E62. The method of any one of E45-E61, wherein training the model comprises performing supervised learning.
[0150] E63. The method of any one of E45-E62, wherein training the model comprises training an ensemble model.
[0151] E64. The method of E47 or E63, wherein the ensemble model comprises individual models trained by supervised learning and models trained by unsupervised learning.
[0152] E65. The method of any one of E45-E64, further comprising receiving a plurality of synthetic images that add anomalies to normal images.
[0153] E66. The method of E65, wherein the set of training images comprises a portion of the plurality of synthetic images.
[0154] E67. The method of any one of E65-E66, wherein the model was validated with a set of validation images comprising a portion of the synthetic images.
[0155] E68. The method of any one of E45-E67, further comprising: generating a heatmap overlay of image data to indicate location of at least one anomaly, wherein the heatmap indicates which portions of the image data contributes most to an inference made by the model of presence of the at least one anomaly.
[0156] E69. The method of E68, further comprising: receiving a user indication that the at least one anomaly is a false anomaly.
[0157] E70. The method of E69, further comprising: identifying at least one mask based on the heatmap associated with the false anomaly.Electronically filed: February 5, 2026 Attorney Docket No: 10928-W001-SEC
[0158] E71. The method of any one of E45-E70, further comprising: storing anomaly information and audit information in database storage.
[0159] E72. The method of any one of E45-E71, further comprising, during configuration of the manufacturing line: outputting detection of a misalignment within the assigned segment.
[0160] E73. One or more non-transitory, computer-readable media storing instructions that, when executed by processing hardware of a controller, cause the controller to perform the method of any one of E45-E72.Additional Considerations
[0161] Some of the figures described herein illustrate example block diagrams having one or more functional components. It will be understood that such block diagrams are for illustrative purposes and the devices described and shown may have additional, fewer, or alternative components than those illustrated. Additionally, in various aspects, the components (as well as the functionality provided by the respective components) may be associated with or otherwise integrated as part of any suitable components.
[0162] Some aspects of the disclosure relate to a non-transitory computer-readable storage medium having instructions stored thereon for performing various computer-implemented operations. The term “instructions / computer-readable storage medium” is used herein to include any medium that is capable of storing or encoding a sequence of instructions or computer codes for performing the operations, methodologies, and techniques described herein. The media and computer code may be those specially designed and constructed for the purposes of the aspects of the disclosure, or they may be of the kind well known and available to those having skill in the computer software arts. Examples of computer-readable storage media include, but are not limited to: magnetic media such as hard disks, floppy disks, solid state drives (SSDs), and magnetic tape; optical media such as CD-ROMs and holographic devices; magneto-optical media such as optical disks; and hardware devices that are specially configured to store and execute program code, such as ASICs, programmable logic devices (“PLDs”), and ROM and RAM devices.
[0163] Examples of computer code include machine code, such as produced by a compiler, and files containing higher-level code that are executed by a computer using an interpreter or a compiler. For example, an aspect of the disclosure may be implemented using Java, C++, or other object-oriented programming languages and development tools. Additional examples of computer code include encrypted code and compressed code. Moreover, an aspect of theElectronically filed: February 5, 2026 Attorney Docket No: 10928-W001-SECdisclosure may be downloaded as a computer program product, which may be transferred from a remote computer (e.g., a server computer) to a requesting computer (e.g., a computer or a different server computer) via a transmission channel. Another aspect of the disclosure may be implemented in hardwired circuitry in place of, or in combination with, machine-executable software instructions.
[0164] As used herein, the singular terms “a,” “an,” and “the” may include plural referents, unless the context clearly dictates otherwise. This description, and the claims that follow, should be read to include one or at least one and the singular also includes the plural unless expressly stated or it is obvious that it is meant otherwise. As used herein, the terms “comprises,” “comprising,” “includes,” “including,” “has,” “having” or any other variation thereof, are intended to cover a non-exclusive inclusion. For example, a process, method, article, or apparatus that comprises a list of elements is not necessarily limited to only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Further, unless expressly stated to the contrary, “or” refers to an inclusive or and not to an exclusive or. For example, a condition A or B is satisfied by any one of the following: A is true (or present) and B is false (or not present), A is false (or not present) and B is true (or present), and both A and B are true (or present).
[0165] As used herein, the terms “approximately,” “substantially,” “substantial,” “roughly” and “about” are used to describe and account for small variations. When used in conjunction with an event or circumstance, the terms can refer to instances in which the event or circumstance occurs precisely as well as instances in which the event or circumstance occurs to a close approximation. For example, when used in conjunction with a numerical value, the terms can refer to a range of variation less than or equal to ±10% of that numerical value, such as less than or equal to ±5%, less than or equal to ±4%, less than or equal to ±3%, less than or equal to ±2%, less than or equal to ±1%, less than or equal to ±0.5%, less than or equal to ±0.1%, or less than or equal to ±0.05%. For example, two numerical values can be deemed to be “substantially” the same if a difference between the values is less than or equal to ±10% of an average of the values, such as less than or equal to ±5%, less than or equal to ±4%, less than or equal to ±3%, less than or equal to ±2%, less than or equal to ±1%, less than or equal to ±0.5%, less than or equal to ±0.1%, or less than or equal to ±0.05%.
[0166] Additionally, amounts, ratios, and other numerical values are sometimes presented herein in a range format. It is to be understood that such range format is used for convenience and brevity and should be understood flexibly to include numerical values explicitly specifiedElectronically filed: February 5, 2026 Attorney Docket No: 10928-W001-SECas limits of a range, but also to include all individual numerical values or sub-ranges encompassed within that range as if each numerical value and sub-range is explicitly specified.
[0167] While the techniques disclosed herein have been described with primary to particular operations performed in a particular order, it will be understood that these operations may be combined, sub-divided, or re-ordered to form an equivalent technique without departing from the teachings of the present disclosure. Accordingly, unless specifically indicated herein, the order and grouping of the operations are not limitations of the present disclosure.
Claims
Electronically filed: February 5, 2026 Attorney Docket No: 10928-W001-SECCLAIMS1. A system for visual inspection of a manufacturing line, the manufacturing line comprising a plurality of segments and the system comprising:a program memory storing instructions that, when executed by one or more processing units, cause the system to:receive, from a plurality of imaging devices each assigned to a segment of the plurality of segments, image data of assigned segments of the manufacturing line; analyze the image data to identify an operational state of the plurality of segments by applying a machine learning algorithm to a model trained with a plurality of training data comprising a set of training images representing an assigned segment of a respective imaging device during run-time operation of the manufacturing line or during line clearance of the manufacturing line,wherein the model is configured to receive image data of the respective imaging device as input and to output an image classification associated with detection of an anomaly in the assigned segment; and responsive to identifying at least one anomaly in the plurality of segments, cause a display to present a notification regarding the at least one anomaly.
2. The system of claim 1, wherein the system comprises the plurality of imaging devices, wherein imaging devices of the plurality of imaging devices are configured to capture image data of the assigned segments during run-time of the manufacturing line or during line clearance of the manufacturing line.
3. The system of any one of claims 1-2, wherein each imaging device of the plurality of imaging devices provides image data to a model adapted to the respective imaging device.
4. The system of any one of claims 1-3, wherein the model is an ensemble model.
5. The system of any one of claims 1-4, wherein the instructions further cause the one or more processing units to:collect training data, including image data provided by the plurality of imaging devices, during run-time of the manufacturing line or during line clearance of the manufacturing line.Electronically filed: February 5, 2026 Attorney Docket No: 10928-W001-SEC6. The system of claim 5, wherein to collect training data, the instructions further cause the one or more processing units to:collect training data throughout a range of motion of at least one segment of the manufacturing line.
7. The system of any one of claims 1-6, wherein the set of training images consists of normal images.
8. The system of any one of claims 1-6, wherein the set of training images comprises normal images and anomalous images.
9. The system of any one of claims 1-8, wherein the instructions further cause the one or more processing units to:receive or generate a mask that defines a masked region corresponding to a location in a segment of the manufacturing line.
10. The system of claim 9, wherein the masked region is at a temporary anomaly or noncritical anomaly in the segment of the manufacturing line.
11. The system of any one of claims 1-10, wherein the instructions further cause the one or more processing units to:modify pixel data within the image data to adjust for periodic or intermittent lighting changes in the image data.
12. The system of any one of claims 1-11, wherein the machine learning algorithm comprises an anomaly detection algorithm.
13. The system of any one of claims 1-12, wherein training the model comprises performing unsupervised learning.
14. The system of any one of claims 1-13, wherein training the model comprises performing supervised learning.Electronically filed: February 5, 2026 Attorney Docket No: 10928-W001-SEC15. The system of any one of claims 1-14, wherein training the model comprises training an ensemble model.
16. The system of claim 4 or claim 15, wherein the ensemble model comprises individual models trained by supervised learning and models trained by unsupervised learning.
17. The system of any one of claims 1-16, wherein the program memory stores instructions that, when executed by one or more processing units, cause the system to receive a plurality of synthetic images that add anomalies to normal images.
18. The system of claim 17, wherein the set of training images comprises a portion of the plurality of synthetic images.
19. The system of any one of claims 17-18, where the model was validated with a set of validation images comprising a portion of the synthetic images.
20. The system of any one of claims 1-19, wherein the instructions further cause the one or more processing units to:generate a heatmap overlay of image data to indicate location of at least one anomaly, wherein the heatmap indicates which portions of the image data contributes most to an inference made by the model of presence of the at least one anomaly.
21. The system of any of claims 1-20, wherein at least one assigned segment of the assigned segments includes a component subject to modification during a manufacturing process of the manufacturing line or to replacement after a manufacturing process of the manufacturing line.
22. The system of claim 21, wherein an imaging device assigned to the segment including the component is configured to capture image data of the segment after at least one modification or replacement.
23. The system of claim 22, wherein the imaging device provides image data after the at least one modification or replacement for analysis by a separate model adapted to each modification or replacement.Electronically filed: February 5, 2026 Attorney Docket No: 10928-W001-SEC24. The system of any one of claims 1-23, further comprising:database storage configured to store anomaly information and audit information of the system.
25. The system of any one of claims 1-24, wherein the anomaly includes surface contamination on a surface of the segment.
26. The system of any one of claims 1-24, wherein the anomaly includes an object that is not an expected component of the manufacturing line.
27. The system of any one of claims 1-26, wherein the model is configured to, during configuration of the manufacturing line:output detection of a misalignment within the assigned segment.
28. A method for detecting anomalies in a manufacturing line comprising a plurality of segments, each segment of the plurality assigned to at least one imaging device, the method comprising:receiving, from the at least one imaging device, image data of assigned segments of the manufacturing line;analyzing the image data to identify operational states of the plurality of segments by applying a machine learning algorithm to a model trained with a plurality of training data comprising a set of training images representing an assigned segment of a respective imaging device during run-time operation of the manufacturing line or during line clearance of the manufacturing line,wherein the model is configured to receive image data of the respective imaging device as input and to output an image classification with detection of an anomaly in the assigned segment; andresponsive to identifying at least one anomaly in the plurality of segments, causing a display to present a notification regarding the at least one anomaly.
29. The method of claim 28, wherein each imaging device provides image data to a model adapted to the respective imaging device.Electronically filed: February 5, 2026 Attorney Docket No: 10928-W001-SEC30. The method of any one of claims 28 or 29, wherein the model is an ensemble model.
31. The method of any one of claims 28-30 further comprising:collecting training data, including image data provided by the at least one imaging device, during run-time of the manufacturing line or during line clearance of the manufacturing line.
32. The method of claim 31, wherein collecting training data comprises:collecting training data throughout a range of motion of at least one segment of the manufacturing line.
33. The method of any one of claims 28-32, wherein the set of training images consists of normal images.
34. The method of any one of claims 28-32, wherein the set of training images comprises normal images and anomalous images.
35. The method of any one of claims 28-34, further comprising:receiving or generating a mask that defines a masked region corresponding to a location in a segment of the manufacturing line.
36. The method of any one of claims 28-35, further comprising:modifying pixel data within the image data to adjust for periodic or intermittent lighting changes in the image data.
37. The method of any one of claims 28-36, wherein the machine learning algorithm comprises an anomaly detection algorithm.
38. The method of any one of claims 28-37, wherein training the model comprises performing unsupervised learning.
39. The method of any one of claims 28-38, wherein training the model comprises performing supervised learning.Electronically filed: February 5, 2026 Attorney Docket No: 10928-W001-SEC40. The method of any one of claims 28-39, wherein training the model comprises training an ensemble model.
41. The method of claim 30 or claim 40, wherein the ensemble model comprises individual models trained by supervised learning and models trained by unsupervised learning.
42. The method of any one of claims 28-41, further comprising receiving a plurality of synthetic images that add anomalies to normal images.
43. The method of claim 42, wherein the set of training images comprises a portion of the plurality of synthetic images.
44. The method of any one of claims 42-43, where the model was validated with a set of validation images comprising a portion of the synthetic images.
45. The method of any one of claims 28-44, further comprising:generating a heatmap overlay of image data to indicate location of at least one anomaly, wherein the heatmap indicates which portions of the image data contributes most to an inference made by the model of presence of the at least one anomaly.
46. One or more non-transitory, computer-readable media storing instructions that, when executed by processing hardware of a controller, cause the controller to perform the method of any one of claims 28-45.