Sensor-based liquid pharmaceutical product manufacturing processes

Abstract

Systems and methods for inspecting and quality control of manufacturing and packaging of pharmaceutical product are disclosed. An exemplary? system comprises sensor circuitry / and a controller circuit. The sensor circuitry comprises an embedded sensor platform, including an embedded accelerometer in an interior of the container empty of pharmaceutical product. The embedded accelerometer can measure container motion or position, orientation and movement information during experimental agitation of the container. The controller circuit generates an agitation protocol using the container motion or position information, which can be used to assist in subsequent agitation and inspection of containers filled with pharmaceutical products. The controller circuit can detect anomalies or defects associated with the container or the pharmaceutical product using data, collected by the embedded sensors.

Description

Docket No.6063.002WO1 SENSOR-BASED LIQUID PHARMACEUTICAL PRODUCT MANUFACTURING PROCESSES CLAIM OF PRIORITY

[0001] This application claims the benefit of priority to U.S. Provisional Patent Application Serial No. 63 / 651,861, filed on May 24, 2024. TECHNICAL FIELD

[0002] This document relates generally to pharmaceutical product inspection and quality control systems, and more particularly, but not by way of limitation, to embedded sensors in containers for characterizing pharmaceutical product manufacturing and packaging processes. BACKGROUND

[0003] Pharmaceutical containers (e.g., vials, pre-filled syringes, ampules, etc.) generally need to be inspected for anomalies during manufacturing and packaging processes to ensure product quality and patient safety. The inspection process can be performed automatically using a computerized inspection machine (also referred to as an automated visual inspection (AVI) process), or manually by human inspectors (thus referred to as a manual visual inspection (MVI) process). In the AVI process, liquid-filled containers are typically maneuvered and inspected using one of the following methods. The first method involves spinning the pharmaceutical liquid-filled container at a high speed (e.g., 1000-7000 revolutions per minute, or RPM) in order to create a vortex inside the container. Spin speed can be set such that the vortex climbs up the sidewall of the container but not so fast as to reach the shoulder of the container. Dense foreign particles or fibers resting on the bottom of the container, invisible from a side view of the container, may be elevated into suspension by the centrifugal force of the spinning container. A camera facing the sidewall is used to image the spinning container. To detect large particles, the images are processed to isolate the blob created by the particle. The second method involves abruptly stopping the spinning container, and inspecting the contained liquid that continues to swirl due to its angular momentum. Images of the swirling liquid can be taken and processed such as by subtracting one image from another toDocket No.6063.002WO1 highlight anything that moves from one image to another. Particles swirling with the fluid can be detected from the subtracted image. Compared to the first method where the container is imaged while the container is spinning, the second method may provide in many cases better detection of smaller particles because small cosmetic blemishes on the surface of the container do not appear in the resulting processed image. These two methods can be used in conjunction to improve the detection rates for particles and fibers through the AVI process.

[0004] The MVI process depends on human inspectors agitating the liquid in the container and inspecting it for particles and fibers while the liquid swirls around the container. Mobilizing particles in the liquid product makes it easier to detect smaller particles and fibers. Manual agitation includes quickly flicking, flipping, and / or rotating the container in a circular, semi-circular, or other forms of cyclic motion.

[0005] Manual agitation and inspection is considered to be the “gold standard” for inspecting pharmaceutical liquid-filed containers and the technique that automated machines and AVI methods are tested against. The MVI process is more commonly used for small manufacturing batches and clinical batch production at least because it is more easily transferred from one container type, product, fill level, etc. to another. However, compared to the AVI process, the MVI process is more labor intensive, and may introduce inconsistent inspection results from inspector to inspector or over time. On the other hand, the AVI process generally requires considerable experimentation for every drug product, fill level, container size and type, to determine optimal, container-specific agitation (e.g., spin speed) and timing of the image acquisition to achieve an acceptable anomaly or defect detection performance. SUMMARY

[0006] This document describes systems and methods for characterizing inspection during manufacturing, packaging and distribution processes for pharmaceutical product, particularly liquid product. The inspection processes include manual or automated agitation of containers of liquid pharmaceutical product. Characterization of liquid agitation that a human expert inspector typically performs may help elucidate why manual agitation can effectively and efficiently get particles swirling within the liquid that, in certain cases,Docket No.6063.002WO1 outperforms automated agitation in an AVI process using automated inspection machines that are able to spin at high speeds with higher acceleration rates than humans can manually perform. The agitation process characterization may also be used to improve MVI procedures, such as for training inexperienced inspectors, or to inform potential improvements to AVI system design. As described in accordance with various embodiments in this document, embedded sensors may be integrated in a sensor platform in an interior of a container empty of liquid pharmaceutical product, and collect container motion or position information during agitation of the container. Data collected from the embedded sensors may be fed into a computational fluid dynamics (CFD) model to make predictions of fluid dynamics characteristics of the liquid pharmaceutical product (that would have been filled in the container), and further to determine one or more agitation parameters that may be used to assist in subsequent agitation and inspection of containers filled with liquid pharmaceutical products.

[0007] Sensor solutions exist for characterizing manufacturing and distribution processes of various products. Conventional sensor-based inspection technologies, however, usually involve sensors placed on crates or other packaging external to the product units. The present document describes, among other things, an embedded sensor platform configured to support a suite of sensors to collect information about the motion of the container from an interior of the container (e.g., a vial or a syringe), monitor environmental factors affecting the containers, and detect anomalies (e.g., excessive light exposure, or glass breakage), among other inspection and quality control tasks. The embedded sensor platform as described herein allows for more precise and complete characterization of agitation movement and various aspects of manufacturing, packing, distribution, and handling processes as experienced by the pharmaceutical product. For example, embedded sensors allow for direct in situ measurement of agitation motion parameters, which may be used to assist in subsequent agitation and inspection of liquid pharmaceutical products in containers of different sizes or shapes. In various examples, optical and thermal aspects of the manufacturing and handling processes as experienced by the liquid pharmaceutical products may also be better characterized. While the sensor platform is ideal for collecting information about the processes as experienced by liquid pharmaceutical product, the sensor platform can also be used to collectDocket No.6063.002WO1 information about the processes as experienced by a lyophilized pharmaceutical product.

[0008] In accordance with an embodiment, an embedded accelerometer (e.g., a three-axis accelerometer, x, y, z) can be used to characterize agitation of a container for holding pharmaceutical product. In accordance with another embodiment, an embedded accelerometer and gyroscope combination (sometime referred to as a six-axis accelerometer) includes an orientation measurement (pitch, yaw, roll, etc.) and can be used to characterize agitation of a container for holding pharmaceutical product. The container, which can be an actual container (e.g., a vial or a syringe) or a fabricated alternative with matching optical and mechanical properties, is empty of pharmaceutical liquid or lyophilized product. The embedded accelerometer can be operably positioned in the interior of the container and securely attached thereto, such as at an interior surface of a cylindrical portion of the container. The accelerometer can collect motion data when a human inspector holds and spins the container to cause agitated motion (e.g., swirling) of the container. Motion and position parameters (e.g., velocities or displacement of the container) that characterize the manual agitation may be determined from the accelerometer data.

[0009] In accordance with another embodiment, other embedded sensors (e.g., embedded acoustic sensors, embedded optical sensors, or embedded pressure sensors) may be used in conjunction with, or as an alternative to, the embedded accelerometer as described above to provide more complete characterization of the environmental factors that the product-filled containers experience as they traverse one or more phases of manufacturing, packaging, shipment, and / or storage. In an example, an embedded optical sensor may sense light, including for example visible light or ultraviolet (UV) light, and detect light properties (e.g., spectral properties). In another example, an embedded pressure sensor may sense and characterize gripping force applied to a container either by a human inspector or a machine. Coupling and synchronization of different sensor types within the context of a known manufacturing process may help resolve issues, such as drift, which are commonly associated with contemporary embedded accelerometers.

[0010] In accordance with another embodiment, one or more sensors of the same or different types may be used to characterize anomalies or defectsDocket No.6063.002WO1 experienced during the manufacturing, packaging, and / or handling of contained pharmaceutical product. In an example, an optical sensor may be used to detect light intensity irradiated on the pharmaceutical container, which may be used to assist in anomaly or defect detection. In yet another example, one or more embedded acoustic sensors may be used to detect glass breakage, which may occur on a wide range of different parts of the process and has been observed on fill lines, inspection lines and during device assembly and packaging.

[0011] Example 1 is a system for characterizing inspection and quality control processes of pharmaceutical product manufacturing and packaging in containers. The system comprises: sensor circuitry, comprising an embedded sensor platform that includes an embedded accelerometer in an interior of a container empty of pharmaceutical product, the embedded accelerometer configured to collect container motion or position information during agitation of the container; and a controller circuit configured to, based at least in part on the collected container motion or position information, generate an agitation protocol for agitating containers.

[0012] In Example 2, the subject matter of Example 1 optionally includes the controller circuit that can include executable instructions to calculate one or more agitation parameters using the collected container motion or position information, and to generate the agitation protocol using the calculated one or more agitation parameters.

[0013] In Example 3, the subject matter of Example 2 optionally includes the one or more agitation parameters that can include at least one of a position, an orientation, a velocity, an acceleration, a displacement, or a range of movement of the container.

[0014] In Example 4, the subject matter of any one or more of Examples 1–3 optionally includes the controller circuit that can be configured to provide the agitation protocol to a robotic system for subsequent agitation and inspection of containers filled with pharmaceutical products.

[0015] In Example 5, the subject matter of any one or more of Examples 1–4 optionally include the controller circuit that can be configured to program a robotic system with the agitation protocol for subsequent agitation and inspection of containers filled with pharmaceutical products.Docket No.6063.002WO1

[0016] In Example 6, the subject matter of any one or more of Examples 1–5 optionally includes the controller circuit that can be configured to provide the agitation protocol to a user to assist in subsequent agitation and inspection of containers filled with pharmaceutical products.

[0017] In Example 7, the subject matter of any one or more of Examples 1–6 optionally includes the controller circuit that can apply the collected container motion or position information to a computational fluid dynamics (CFD) model to make predictions of fluid dynamics characteristics of the pharmaceutical product, and to determine one or more agitation parameters based at least in part on the predictions of fluid dynamics characteristics.

[0018] In Example 8, the subject matter of Example 7 optionally includes the predictions of fluid dynamics characteristics that can include predictions of at least one of a shear stress distribution, a flow pattern, or a velocity profile of the pharmaceutical product.

[0019] In Example 9, the subject matter of any one or more of Examples 1–8 optionally includes the embedded sensor platform that can be sized and shaped to be removably inserted into and substantially conform to a cylindrical portion of the interior of the container.

[0020] In Example 10, the subject matter of Example 9 optionally includes the embedded accelerometer configured to be aligned to a longitudinal axis of the cylindrical portion of the interior of the container.

[0021] In Example 11, the subject matter of any one or more of Examples 1– 10 optionally includes the embedded accelerometer that can be configured to be mounted on a base unit of the embedded sensor platform.

[0022] In Example 12, the subject matter of Example 11 optionally includes the embedded accelerometer that can be configured to be positioned at substantially a center of mass of the embedded system.

[0023] In Example 13, the subject matter of any one or more of Examples 1– 12 optionally includes an external control device external to the container and including at least a portion of the controller circuit, the external control device configured to communicate with the embedded sensor platform via a wireless communication link.

[0024] In Example 14, the subject matter of any one or more of Examples 1– 13 optionally includes the controller circuit that can be configured to remove orDocket No.6063.002WO1 attenuate a temporal drift from the collected container motion or position information.

[0025] In Example 15, the subject matter of any one or more of Examples 1– 14 optionally includes the sensor circuitry that can include an optical sensor configured to collect optical information from the container.

[0026] In Example 16, the subject matter of Example 15 optionally includes the controller circuit that can be configured to use the collected optical information to synchronize collection of the container motion or position information, or to calibrate the container motion or position information collected by the embedded accelerometer.

[0027] In Example 17, the subject matter of Example 16 optionally includes, wherein to calibrate the container motion or position information, the controller circuit is configured to remove or attenuate a temporal drift in the container motion or position information.

[0028] In Example 18, the subject matter of any one or more of Examples 1– 17 optionally include the sensor circuitry electrically coupled to a motion capture system configured to collect ergonomic information of an operator agitating the contained pharmaceutical product, wherein the controller circuit is configured to estimate a pose or to track motion of the operator using the collected ergonomic information, and to generate the agitation protocol further based on the estimated pose or the tracked motion of the operator.

[0029] In Example 19, the subject matter of Example 18 optionally includes the motion capture system that comprises a plurality of spatially distributed imaging sensors configured to collect the ergonomic information from respective different positions or angles with respect to the operator.

[0030] In Example 20, the subject matter of any one or more of Examples 1– 19 optionally includes a user interface to present the agitation protocol to a user to assist in subsequent manual agitation and inspection of the containers filled with pharmaceutical products.

[0031] In Example 21, the subject matter of any one or more of Examples 1– 20 optionally includes the controller circuit that can be configured to generate a control signal to a robotic system to initiate automatic agitation and inspection of the containers filled with pharmaceutical products.Docket No.6063.002WO1

[0032] In Example 22, the subject matter of any one or more of Examples 1– 21 optionally includes the sensor circuitry that can include one or more embedded optical sensors at the interior of the container, the one or more embedded optical sensors configured to sense a light intensity, in a specific wavelength or wavelength range, irradiated on the container.

[0033] In Example 23, the subject matter of Example 22 optionally includes the controller circuit that can be configured to detect light exposure of the pharmaceutical products contained in the containers based on the sensed light intensity.

[0034] In Example 24, the subject matter of any one or more of Examples 22–23 optionally includes the one or more embedded optical sensors that can include a photosensor array along a length of a cylindrical portion of the container.

[0035] In Example 25, the subject matter of any one or more of Examples 22–24 optionally includes the one or more embedded optical sensors that can include an insertable photovoltaic membrane configured to be inserted into and substantially conform to an inner surface of a cylindrical portion of the container.

[0036] In Example 26, the subject matter of any one or more of Examples 1– 25 optionally includes the sensor circuitry that can include an embedded acoustic sensor at the interior of the container, the embedded acoustic sensor configured to collect acoustic data in response to manipulation of the container, wherein the controller circuit is configured to detect a glass breakage event in a glass portion of the container using the collected acoustic data.

[0037] In Example 27, the subject matter of Example 26 optionally includes the controller circuit that can be configured to use the collected acoustic data to synchronize collection of the container motion or position information.

[0038] In Example 28, the subject matter of any one or more of Examples 1– 27 optionally includes the embedded accelerometer that can be further configured to collect vibration data in response to manipulation of the container, wherein the controller circuit is configured to detect a glass breakage event in a glass portion of the container using the collected vibration data.

[0039] In Example 29, the subject matter of any one or more of Examples 1– 28 optionally includes the agitation protocol that can include a helical motionDocket No.6063.002WO1 protocol, a circular motion protocol, an elliptical motion protocol, a conical spiral motion protocol, a flicking motion protocol, or a flipping motion protocol.

[0040] Example 30 is a method of characterizing inspection and quality control of processes of pharmaceutical product manufacturing and packaging in containers, the method comprising: introducing agitation to a container empty of pharmaceutical product, the container associated with an embedded sensor platform including an embedded accelerometer in an interior of the container; collecting sensor data including container motion or position information sensed by the embedded accelerometer during the agitation of the container; and based at least in part on the collected container motion or position information, generating an agitation protocol for agitating containers.

[0041] In Example 31, the subject matter of Example 30 optionally includes calculating one or more agitation parameters using the collected container motion or position information, wherein generating the agitation protocol includes using the calculated one or more agitation parameters.

[0042] In Example 32, the subject matter of Example 31 optionally includes the one or more agitation parameters that can include at least one of a position, an orientation, a velocity, an acceleration, a displacement, or a range of movement of the container.

[0043] In Example 33, the subject matter of any one or more of Examples 31–32 optionally includes calculating the one or more agitation parameters including making predictions of fluid dynamics characteristics of the pharmaceutical product by applying the collected container motion or position information to a computational fluid dynamics (CFD) model, and calculating the one or more agitation parameters based at least in part on the predictions of fluid dynamics characteristics.

[0044] In Example 34, the subject matter of any one or more of Examples 30–33 optionally includes providing the agitation protocol to a robotic system for subsequent agitation and inspection of containers filled with pharmaceutical products.

[0045] In Example 35, the subject matter of any one or more of Examples 30–34 optionally includes programming a robotic system with the agitation protocol for subsequent agitation and inspection of containers filled with pharmaceutical products.Docket No.6063.002WO1

[0046] In Example 36, the subject matter of any one or more of Examples 30–35 optionally includes providing the agitation protocol to a user to assist in subsequent agitation and inspection of containers filled with pharmaceutical products.

[0047] In Example 37, the subject matter of any one or more of Examples 30–36 optionally includes the embedded sensor platform that can be inserted into and substantially conforms to a cylindrical portion of the interior of the container.

[0048] In Example 38, the subject matter of Example 37 optionally includes the embedded sensor platform such positioned that the embedded accelerometer is aligned to a longitudinal axis of the cylindrical portion of the interior of the container.

[0049] In Example 39, the subject matter of any one or more of Examples 30–38 optionally includes the embedded sensor platform secured on a base unit of the embedded sensor platform.

[0050] In Example 40, the subject matter of Example 39 optionally includes the embedded sensor platform positioned at substantially a center of mass of the embedded system.

[0051] In Example 41, the subject matter of any one or more of Examples 30–40 optionally includes removing or attenuating a temporal drift from the collected container motion or position information.

[0052] In Example 42, the subject matter of any one or more of Examples 30–41 optionally includes collecting optical information from the container via one or more embedded optical sensors at the interior of the container.

[0053] In Example 43, the subject matter of Example 42 optionally includes synchronizing collection of the container motion or position information or calibrating the container motion or position information collected by the embedded accelerometer using the collected optical information.

[0054] In Example 44, the subject matter of any one or more of Examples 30–43 optionally includes collecting, via a plurality of spatially distributed imaging sensors, ergonomic information from an operator agitating the contained pharmaceutical product; estimating pose or tracking motion of the operator using the collected ergonomic information; and generating the agitationDocket No.6063.002WO1 protocol further based on the estimated pose or the tracked motion of the operator.

[0055] In Example 45, the subject matter of any one or more of Examples 30–44 optionally includes detecting an anomaly associated with the container or the contained pharmaceutical product using the collected sensor data including during the agitation of the contained pharmaceutical product; and generating quality control diagnostics based at least on the detected anomaly.

[0056] In Example 46, the subject matter of any one or more of Examples 30–45 optionally includes sensing a light intensity, in a specific wavelength or wavelength range, irradiated on the container via one or more embedded optical sensors at the interior of the container; and determining light exposure of the pharmaceutical product based on the sensed light intensity.

[0057] In Example 47, the subject matter of any one or more of Examples 30–46 optionally includes collecting acoustic data via an embedded acoustic sensor at the interior of the container in response to manipulation of the container; and detecting a glass breakage event in a glass portion of the container using the collected acoustic data.

[0058] In Example 48, the subject matter of Example 47 optionally includes synchronizing the collection of the container motion or position information using the collected acoustic data.

[0059] In Example 49, the subject matter of any one or more of Examples 30–48 optionally includes collecting vibration data via the embedded accelerometer in response to diagnostic striking of the container; and detecting a glass breakage event in a glass portion of the container using the collected vibration data.

[0060] Example 50 is a system for characterizing inspection and quality control processes of pharmaceutical product manufacturing and packaging in containers. The system comprises: sensor circuitry, comprising an embedded sensor platform that includes an embedded accelerometer in an interior of a container empty of pharmaceutical product, the embedded accelerometer configured to collect container motion or position information during agitation of the container; and a controller circuit configured to detect an anomaly of a robotic system for agitation and inspection of containers based at least in part on the collected container motion or position information.Docket No.6063.002WO1

[0061] In Example 51, the subject matter of Example 50 optionally includes the controller circuit that can be configured to generate quality control diagnostics based at least in part on the detected anomaly.

[0062] In Example 52, the subject matter of any one or more of Examples 50–51 optionally includes the controller circuit that can be configured to: calculate one or more agitation parameters using the collected container motion or position information; and detect the anomaly of the robotic system using information using the calculated one or more agitation parameters.

[0063] In Example 53, the subject matter of Example 52 optionally includes the one or more agitation parameters that can include at least one of a position, an orientation, a velocity, an acceleration, a displacement, or a range of movement of the container.

[0064] In Example 54, the subject matter of any one or more of Examples 52–53 optionally includes the controller circuit that can be configured to: make predictions of fluid dynamics characteristics of the pharmaceutical product by applying the collected container motion or position information to a computational fluid dynamics (CFD) model; and calculate the one or more agitation parameters using the predictions of fluid dynamics characteristics.

[0065] In Example 55, the subject matter of any one or more of Examples 52–54 optionally includes the controller circuit that can be configured to pre- process the collected container motion or position information including removing or attenuating a temporal drift, and to calculate the one or more agitation parameters using the pre-processed container motion or position information.

[0066] In Example 56, the subject matter of any one or more of Examples 50–55 optionally includes the embedded sensor platform sized and shaped to be removably inserted into and substantially conform to a cylindrical portion of the interior of the container.

[0067] In Example 57, the subject matter of Example 56 optionally includes the embedded accelerometer that can be configured to be aligned to a longitudinal axis of the cylindrical portion of the interior of the container.

[0068] In Example 58, the subject matter of any one or more of Examples 50–57 optionally includes the embedded accelerometer that can be configured to be mounted on a base unit of the embedded sensor platform.Docket No.6063.002WO1

[0069] In Example 59, the subject matter of Example 58 optionally includes the embedded accelerometer that can be configured to be positioned at substantially a center of mass of the embedded system.

[0070] In Example 60, the subject matter of any one or more of Examples 50–59 optionally includes the sensor circuitry that can include an embedded sensor in the interior of the empty container, the embedded sensor selected from an optical sensor, an acoustic sensor, a vibration sensor, or a combination thereof and configured to collect information during agitation of the empty container, wherein the controller circuit is configured to detect an anomaly of a second container filled with pharmaceutical product in agitation and inspection of the second container based at least in part on the collected information from the embedded sensor.

[0071] In Example 61, the subject matter of any one or more of Examples 50–60 optionally includes the sensor circuitry that can include one or more embedded optical sensors at the interior of the empty container, the one or more embedded optical sensors configured to sense a light intensity, in a specific wavelength or wavelength range, irradiated on the empty container, wherein the controller circuit is configured to synchronize collection of the container motion or position information or calibrate the container motion or position information using the collected optical information.

[0072] In Example 62, the subject matter of any one or more of Examples 50–61 optionally includes the sensor circuitry that can include one or more optical sensors at the interior of the empty container and configured to sense a light intensity, in a specific wavelength or wavelength range, irradiated on a second container filled with pharmaceutical product, wherein the controller circuit is configured to detect an anomaly of the second container or the contained pharmaceutical product based at least in part on the sensed light intensity.

[0073] In Example 63, the subject matter of any one or more of Examples 50–62 optionally includes the sensor circuitry that can include an acoustic sensor at the interior of the empty container and configured to collect acoustic data in response to manipulation of a second container filled with pharmaceutical product, wherein the controller circuit is configured to detect a glass breakage event in a glass portion of the second container using the collected acoustic data.Docket No.6063.002WO1

[0074] In Example 64, the subject matter of any one or more of Examples 50–63 optionally includes the embedded accelerometer further configured to collect vibration data in response to manipulation of a second container filled with pharmaceutical product, wherein the controller circuit is configured to detect a glass breakage event in a glass portion of the second container using the collected vibration data.

[0075] Example 65 is a method of characterizing inspection and quality control of processes of pharmaceutical product manufacturing and packaging in containers, the method comprising: introducing agitation to a container empty of pharmaceutical product, the container associated with an embedded sensor platform including an embedded accelerometer in an interior of the container; collecting sensor data including container motion or position information sensed by the embedded accelerometer during the agitation of the container; and based at least in part on the collected sensor data, detecting an anomaly of a robotic system for agitation and inspection of containers based at least in part on the collected container motion or position information.

[0076] In Example 66, the subject matter of Example 65 optionally includes generating quality control diagnostics based at least in part on the detected anomaly of the robotic system.

[0077] In Example 67, the subject matter of any one or more of Examples 65–66 optionally includes calculating one or more agitation parameters using the collected container motion or position information, wherein detecting the anomaly includes using calculated one or more agitation parameters.

[0078] In Example 68, the subject matter of Example 67 optionally includes the one or more agitation parameters that can include at least one of a position, an orientation, a velocity, an acceleration, a displacement, or a range of movement of the container.

[0079] In Example 69, the subject matter of any one or more of Examples 67–68 optionally includes making predictions of fluid dynamics characteristics of the pharmaceutical product by applying the collected container motion or position information to a computational fluid dynamics (CFD) model; and calculating the one or more agitation parameters using the predictions of fluid dynamics characteristics.Docket No.6063.002WO1

[0080] In Example 70, the subject matter of any one or more of Examples 67–69 optionally includes pre-processing the collected container motion or position information including removing or attenuating a temporal drift; and calculating the one or more agitation parameters using the pre-processed container motion or position information.

[0081] In Example 71, the subject matter of any one or more of Examples 65–70 optionally includes the embedded sensor platform that can be inserted into and substantially conforms to a cylindrical portion of the interior of the container.

[0082] In Example 72, the subject matter of Example 71 optionally includes the embedded sensor platform such positioned that the embedded accelerometer is aligned to a longitudinal axis of the cylindrical portion of the interior of the container.

[0083] In Example 73, the subject matter of any one or more of Examples 65–72 optionally includes the embedded sensor platform secured on a base unit of the embedded sensor platform.

[0084] In Example 74, the subject matter of Example 73 optionally includes the embedded sensor platform positioned at substantially a center of mass of the embedded system.

[0085] In Example 75, the subject matter of any one or more of Examples 65–74 optionally include collecting information during the agitation of the empty container using an embedded sensor in the interior of the empty container, the embedded sensor selected from an optical sensor, an acoustic sensor, a vibration sensor, or a combination thereof; and detecting an anomaly of a second container filled with pharmaceutical product in agitation and inspection of the second container based at least in part on the collected information from the embedded sensor during the agitation of the empty container.

[0086] In Example 76, the subject matter of any one or more of Examples 65–75 optionally include sensing a light intensity, in a specific wavelength or wavelength range, irradiated on the empty container using one or more embedded optical sensors at the interior of the empty container; and synchronizing collection of the container motion or position information or calibrate the container motion or position information using the sensed light intensity.Docket No.6063.002WO1

[0087] In Example 77, the subject matter of any one or more of Examples 65–76 optionally include sensing a light intensity, in a specific wavelength or wavelength range, irradiated on a second container filled with pharmaceutical product using one or more optical sensors at the interior of the empty container; and detecting an anomaly of the second container or the contained pharmaceutical product based at least in part on the sensed light intensity.

[0088] In Example 78, the subject matter of any one or more of Examples 65–77 optionally include collecting acoustic data in response to manipulation of a second container filled with pharmaceutical product using an acoustic sensor at the interior of the empty container; and detecting a glass breakage event in a glass portion of the second container using the collected acoustic data.

[0089] In Example 79, the subject matter of any one or more of Examples 65–78 optionally include collecting vibration data in response to manipulation of the empty container using the embedded accelerometer; and detecting a glass breakage event in a glass portion of a second container filled with pharmaceutical product using the collected vibration data.

[0090] Example 80 is a system for characterizing inspection and quality control processes of pharmaceutical product manufacturing and packaging in containers. The system comprises: sensor circuitry, comprising an embedded sensor platform that includes an embedded sensor in an interior of a first container empty of pharmaceutical product, the embedded sensor selected from an optical sensor, an acoustic sensor, a vibration sensor, or a combination thereof and configured to collect information during agitation of the first container; and a controller circuit configured to detect an anomaly of a second container filled with pharmaceutical product in agitation and inspection of the second container based at least in part on the collected information.

[0091] In Example 81, the subject matter of Example 80 optionally includes the controller circuit configured to generate quality control diagnostics based at least in part on the detected anomaly.

[0092] In Example 82, the subject matter of any one or more of Examples 80–81 optionally includes the sensor circuitry that can include one or more optical sensors configured to sense a light intensity, in a specific wavelength or wavelength range, irradiated on the second container, wherein the controller circuit is configured to detect the anomaly of the second container or theDocket No.6063.002WO1 contained pharmaceutical product based at least in part on the sensed light intensity.

[0093] In Example 83, the subject matter of Example 82 optionally includes the controller circuit that can be configured to detect light exposure of the pharmaceutical products contained in the containers based on the sensed light intensity.

[0094] In Example 84, the subject matter of any one or more of Examples 80–83 optionally includes the sensor circuitry that can include an acoustic sensor configured to collect acoustic data in response to manipulation of the second container, wherein the controller circuit is configured to detect a glass breakage event in a glass portion of the second container using the collected acoustic data.

[0095] In Example 85, the subject matter of any one or more of Examples 80–84 optionally includes the sensor circuitry that can include an embedded accelerometer configured to collect vibration data in response to manipulation of the second container, wherein the controller circuit is configured to detect a glass breakage event in a glass portion of the second container using the collected vibration data.

[0096] In Example 86, the subject matter of any one or more of Examples 80–85 optionally include the sensor circuitry that can include an embedded accelerometer in an interior of the first container, the embedded accelerometer configured to collect container motion or position information during agitation of the first container, wherein the controller circuit is configured to detect an anomaly of a robotic system for agitation and inspection of containers based at least in part on the collected container motion or position information.

[0097] In Example 87, the subject matter of Example 86 optionally includes the controller circuit that can be configured to: calculate one or more agitation parameters using the collected container motion or position information; and detect the anomaly of the second container or the contained pharmaceutical product using information acquired during the agitation of the second container in accordance with the calculated one or more agitation parameters.

[0098] In Example 88, the subject matter of Example 87 optionally includes the one or more agitation parameters that can include at least one of a position, an orientation, a velocity, a displacement, or a range of movement of the first container.Docket No.6063.002WO1

[0099] In Example 89, the subject matter of any one or more of Examples 87–88 optionally includes the controller circuit that can be configured to: make predictions of fluid dynamics characteristics of the pharmaceutical product by applying the collected container motion or position information to a computational fluid dynamics (CFD) model; and calculate the one or more agitation parameters using the predictions of fluid dynamics characteristics.

[0100] In Example 90, the subject matter of any one or more of Examples 87–89 optionally includes the controller circuit that can be configured to pre- process the collected container motion or position information including removing or attenuating a temporal drift, and to calculate the one or more agitation parameters using the pre-processed container motion or position information.

[0101] In Example 91, the subject matter of any one or more of Examples 87–90 optionally includes the controller circuit that can be configured to program a robotic system with the calculated one or more agitation parameters, and to detect the anomaly of the second container or the contained pharmaceutical product using information acquired during a robotic agitation of the second container by the robotic system.

[0102] In Example 92, the subject matter of any one or more of Examples 86–91 optionally includes the sensor circuitry that can include one or more embedded optical sensors at the interior of the first container, the one or more embedded optical sensors configured to sense a light intensity, in a specific wavelength or wavelength range, irradiated on the first container, wherein the controller circuit is configured to synchronize collection of the container motion or position information or calibrate the container motion or position information using the collected optical information.

[0103] In Example 93, the subject matter of any one or more of Examples 86–92 optionally includes the controller circuit that can be configured to: collect ergonomic information sensed by a plurality of spatially distributed imaging sensors from an operator agitating the first container; estimate pose or tracking motion of the operator using the collected ergonomic information; and detect the anomaly of the second container or the contained pharmaceutical product using information acquired during the agitation of the second container based on the estimated pose or the tracked motion of the operator.Docket No.6063.002WO1

[0104] Example 94 is a method of characterizing inspection and quality control of processes of pharmaceutical product manufacturing and packaging in containers, the method comprising: introducing agitation to a first container empty of pharmaceutical product, the first container associated with an embedded sensor platform including an embedded accelerometer in an interior of the container; collecting sensor data including container motion or position information sensed by the embedded accelerometer during the agitation of the first container; and based at least in part on the collected sensor data, detecting an anomaly of a second container filled with pharmaceutical product in subsequent agitation and inspection of the second container based at least in part on the collected container motion or position information.

[0105] In Example 95, the subject matter of Example 94 optionally includes generating quality control diagnostics based at least in part on the detected anomaly of the second container or the contained pharmaceutical product.

[0106] In Example 96, the subject matter of any one or more of Examples 94–95 optionally includes collecting container motion or position information during agitation of the first container using an embedded accelerometer in an interior of the first container, and based at least in part on the collected sensor data, detecting an anomaly of a robotic system for agitation and inspection of containers based at least in part on the collected container motion or position information.

[0107] In Example 97, the subject matter of any one or more of Examples 94–96 optionally includes calculating one or more agitation parameters using the collected container motion or position information, wherein detecting the anomaly includes using information acquired during the agitation of the second container in accordance with the calculated one or more agitation parameters.

[0108] In Example 98, the subject matter of Example 97 optionally includes the one or more agitation parameters that can include at least one of a position, an orientation, a velocity, a displacement, or a range of movement of the first container.

[0109] In Example 99, the subject matter of any one or more of Examples 97–98 optionally include making predictions of fluid dynamics characteristics of the pharmaceutical product by applying the collected container motion or position information to a computational fluid dynamics (CFD) model; andDocket No.6063.002WO1 calculating the one or more agitation parameters using the predictions of fluid dynamics characteristics.

[0110] In Example 100, the subject matter of any one or more of Examples 97–99 optionally includes pre-processing the collected container motion or position information including removing or attenuating a temporal drift; and calculating the one or more agitation parameters using the pre-processed container motion or position information.

[0111] In Example 101, the subject matter of any one or more of Examples 94–100 optionally includes collecting optical information from the first container via one or more embedded optical sensors at the interior of the first container; and synchronizing collection of the container motion or position information or calibrating the container motion or position information using the collected optical information.

[0112] In Example 102, the subject matter of any one or more of Examples 94–101 optionally includes collecting ergonomic information sensed by a plurality of spatially distributed imaging sensors from an operator agitating the first container; and estimating pose or tracking motion of the operator using the collected ergonomic information, wherein detecting the anomaly includes using information acquired during the agitation of the second container based on the estimated pose or the tracked motion of the operator.

[0113] In Example 103, the subject matter of any one or more of Examples 94–102 optionally includes the collected sensor data that can include light intensity data, in a specific wavelength or wavelength range, irradiated on the second container sensed by one or more optical sensors, wherein detecting the anomaly includes determining light exposure of the pharmaceutical product based on the sensed light intensity.

[0114] In Example 104, the subject matter of any one or more of Examples 94–103 optionally includes the collected sensor data that can include acoustic data sensed by an acoustic sensor in response to manipulation of the second container, wherein detecting the anomaly includes detecting a glass breakage event in a glass portion of the second container using the collected acoustic data.

[0115] In Example 105, the subject matter of any one or more of Examples 94–104 optionally includes the collected sensor data that can include vibration data collected by the embedded accelerometer in response to diagnostic strikingDocket No.6063.002WO1 of the second container, wherein detecting the anomaly includes detecting a glass breakage event in a glass portion of the second container using the collected vibration data.

[0116] Although the sensor-based characterization of liquid agitation and detection of anomalies or defects as described herein are discussed in the context of manufacturing, packaging, and quality control of contained liquid pharmaceutical products, such techniques may be similarly used in other liquid product (e.g., food or consumer products) and lyophilized pharmaceutical product manufacturing and quality assurance processes with little or no modification (e.g., adjusting monitoring or control parameters), without departing from the scope of the present disclosure. The above summary is an overview of some of the teachings of the present application and not intended to be an exclusive or exhaustive treatment of the present subject matter. Further details about the present subject matter are found in the detailed description and appended claims. Other aspects of the disclosure will be apparent to persons skilled in the art upon reading and understanding the following detailed description and viewing the drawings that form a part thereof, each of which are not to be taken in a limiting sense. The scope of the present disclosure is defined by the appended claims and their legal equivalents. BRIEF DESCRIPTION OF THE DRAWINGS

[0117] Various embodiments are illustrated by way of example in the figures of the accompanying drawings. Such embodiments are demonstrative and not intended to be exhaustive or exclusive embodiments of the present subject matter.

[0118] FIG.1 illustrates an example of a sensor-based liquid pharmaceutical product inspection and quality control system, according to one embodiment of the present subject matter.

[0119] FIGS. 2A-2E illustrate examples of sensor platforms embedded within or attached to a pharmaceutical liquid container to characterize liquid agitation and to detect anomalies.

[0120] FIG.3 illustrates an example of agitation characterization in an MVI process using a motion capture system comprising multiple imaging devices to capture body positions and motions of an operator.Docket No.6063.002WO1

[0121] FIGS. 4A-4B illustrate examples of raw and processed acceleration data before and after drift removal or de-trending.

[0122] FIG.5 illustrates examples of synchronized sensor signals which can be used to determine or calibrate positions of a container during a production process.

[0123] FIG.6 is a flow chart illustrating an example method of monitoring and quality control of manufacturing and packaging operations of liquid pharmaceutical product in a container using embedded sensors.

[0124] FIG.7 illustrates generally a block diagram of an example machine upon which any one or more of the techniques (e.g., methodologies) discussed herein may be performed. DETAILED DESCRIPTION

[0125] Disclosed herein are systems, devices, and methods for characterizing inspection and quality control processes of manufacturing and packaging of pharmaceutical products in containers, particularly liquid pharmaceutical products. An embedded sensor platform is customized and configured to support a suite of sensing and / or measurement devices to perform various quality control tasks including, for example, characterizing container agitation, monitoring environmental factors affecting the containers, or detecting anomalies or defects with respect to the content of a container or with respect to a container. In accordance with one embodiment, an exemplary system comprises sensor circuitry configured to collect information about a container for holding liquid pharmaceutical product, and a controller circuit. The sensor circuitry comprises an embedded sensor platform, including an embedded accelerometer (three-axis or six-axis), in an interior of the container empty of liquid pharmaceutical product. The embedded accelerometer can measure container motion or position information during agitation of the container. The controller circuit can determine one or more agitation parameters using the collected container motion or position information, generate an agitation protocol using the determined one or more agitation parameters, and provide the agitation protocol to a user or a robotic system to assist in subsequent agitation and inspection of containers filled with liquid pharmaceutical products. For example, a robotic system can be programmed with the agitation for subsequent agitation and inspection ofDocket No.6063.002WO1 containers filled with pharmaceutical products. In another example, the agitation protocol can assist the user as instructions or training material for subsequent agitation and inspection of containers filled with pharmaceutical products. Data collected by the sensor circuitry may also be used by the controller circuit to detect anomalies or defects associated with the container or another container filled with pharmaceutical product.

[0126] The following detailed description of the present subject matter refers to the accompanying drawings which show, by way of illustration, specific aspects and embodiments in which the present subject matter may be practiced. These embodiments are described in sufficient detail to enable those skilled in the art to practice the present subject matter. Other embodiments may be utilized and structural, logical, and electrical changes may be made without departing from the scope of the present subject matter. References to “an”, “one”, or “various” embodiments in this disclosure are not necessarily to the same embodiment, and such references contemplate more than one embodiment. The following detailed description is, therefore, not to be taken in a limiting sense, and the scope is defined only by the appended claims, along with the full scope of legal equivalents to which such claims are entitled.

[0127] FIG.1 illustrates, by way of example and not limitation, a sensor- based pharmaceutical product inspection and quality control system 100 for characterizing inspection and quality control of manufacturing and packaging of liquid pharmaceutical product in a container. Examples of containers may include vials, syringes or other injection devices (e.g., self-injection pens), ampoules, cartridges, and bottles, among others. The system 100 may characterize one or several aspects of a manual visual inspection (MVI) process 101 where a human inspector simulates agitation of a pharmaceutical liquid in the container such as by flicking, flipping, and / or rotating the container with an embedded sensor, or an automated visual inspection (AVI) process 102 where an automated agitation system spins the container with an embedded sensor at a controllable high speed to generate a vortex.

[0128] The system 100 may include an embedded sensor platform 110, a controller circuit 120, a user interface 140, a robotic system 150, and, optionally, a machine learning (ML) / artificial intelligence (AI) system 160. The embedded sensor platform 110 may be configured to be removably embedded within orDocket No.6063.002WO1 attached to the interior of a pharmaceutical container. In an example, the embedded sensor platform 110 may be sized and shaped to be removably inserted into, and substantially (e.g., within 5-10% tolerance) conform to, a cylindrical portion of the container. The embedded sensor platform 110 may include a base unit to support one or more sensors configured to collect information about the pharmaceutical container. The base unit can be a substrate (e.g., a circuit board) to which the one or more sensors, along with other components, are attached. The base unit may be fabricated, machined, or 3D- printed to a specific size, shape, or profile to facilitate implementation of the sensors thereon and insertion into and retention within the container. In some examples (such as those illustrated in FIGS. 2C-2E and to be described further below), the base unit may be made of flexible materials, such as polymers, composites, or shape memory alloys, to provide deformation, compliant movement, and customary fit within containers of various shapes and sizes, such as a cylindrical interior of a vial or a syringe. In some examples, the embedded sensor platform 110 may have a 3D geometry to accommodate more electronic components yet with more compact size.

[0129] By way of nonlimiting example and as illustrated in FIG. 1, the embedded sensor platform 110 may support one or more of an accelerometer 112, an optical sensor 114, or an acoustic sensor 116. Other sensor types or modalities (e.g., thermal sensors as an example) may also be included in the embedded sensor platform 110. The accelerometer 112 can be configured to collect container motion or position information during the manual agitation of the pharmaceutical container in the MVI process 101, or during the automated agitation in the AVI process 102. In some examples, the accelerometer 112 (optionally along with other sensors included in the embedded sensor platform 110) may be used in post-fill manufacturing or distribution processes, including, for example, labelling, packaging, downstream transport, storage and logistics. The accelerometer 112 can be a three-axis accelerometer configured to collect container motion or position information in different directions (i.e., x-, y-, and z-directions) or a six-axis accelerometer configured to additionally collect container motion or position information in different orientations (yaw, pitch, roll, etc.). The container motion or position information may include, for example, container position, orientation or tilt angle, rotation speed, agitatedDocket No.6063.002WO1 movement pattern (e.g., circular, elliptical, helical or conical spiral path) and parameters characterizing any such pattern (e.g., curvature, radius, major and minor radii, pitch, twist direction), among other information. Additionally, or alternatively, the accelerometer 112 may collect motion information of the pharmaceutical container during agitation. In an example, the accelerometer 112 may be adjustably positioned at substantially (e.g., within 5-10% tolerance) a center of mass of the embedded system of the embedded sensor platform 110 to prevent or reduce confounding accelerations due to weight imbalance during the agitated spinning motion. In an example, the accelerometer 112 can be adjustably aligned to a longitudinal axis of a cylindrical portion of the container. In some examples, a gyroscope may be used in addition or alternative to the accelerometer 112 to collect motion or position information of the container. In some examples, multiple accelerometers (or other types of motion sensors) may be distributed at different locations inside the container and collaboratively detect container motion or position with improved accuracy.

[0130] The optical sensor 114 may collect optical information from the container in response to light irradiation thereon. The collected optical information, such as light intensity, may be used to detect one or more anomalies or defects associated with the container or with another container filled with pharmaceutical product, as will be discussed further below. The optical information may additionally or alternatively be used for detecting and characterizing motion of the container based on a cyclic optical signal received in response to light irradiation on the container being agitated. The optical sensor 114 may include one optical sensor, or an array of multiple optical sensors. In an example, multiple optical sensors may be oriented around the internal surface of the container to detect light incident from different angles. The optical sensor 114 may take different forms with different profiles, including, for example, a flexible photovoltaic membrane as illustrated in FIGS. 2C-2E and to be discussed further below.

[0131] The acoustic sensor 116 may collect acoustic data in response to manipulation of the container during manufacturing, packaging, shipping, or handling of the pharmaceutical container. As will be discussed further below, the acoustic data may be used to detect anomalies or defects associated with theDocket No.6063.002WO1 container or with another container filled with pharmaceutical product, such as glass breakage.

[0132] To assist with capturing manual agitation by a human operator, a fixed camera(s) may be configured to collect ergonomic information from a human operator agitating the pharmaceutical container. During the MVI process 101, a human operator can flick, flip or rotate the container (or a combination of these motions) to simulate agitation of a liquid product contained within. The agitation can help dislodge foreign particulates from the container’s interior surfaces, and mobilize the particles so they are both clearly in the field of view of the operator and easily differentiable from static features (such as dust on the container exterior) due to their relative mobility. Ergonomic information captured by the fixed camera(s) may be analyzed to characterize manual agitation, such as to estimate a pose or to track motion of the operator. The characterization of agitation may provide guidance to optimize the manual or automated agitation process. An agitation protocol can be generated to improve MVI procedures (e.g., for training inexperienced human operators), or to inform potential improvements to AVI system design, as will be described further with respect to FIG. 3.

[0133] The controller circuit 120 can include circuit sets comprising one or more other circuits or sub-circuits, such as an agitation analyzer 121, an anomaly / defect detector 124, and an operator motion analyzer 129. The anomaly / defect detector 124 may further include one or more of a light exposure detector 125 or a glass breakage detector 126. These circuits or sub-circuits may, alone or in combination, perform the functions, methods, or techniques described herein. In an example, hardware of the circuit set may be immutably designed to carry out a specific operation (e.g., hardwired). In an example, the hardware of the circuit set may include variably connected physical components (e.g., execution units, transistors, simple circuits, etc.) including a computer readable medium physically modified (e.g., magnetically, electrically, moveable placement of invariant massed particles, etc.) to encode instructions of the specific operation. In connecting the physical components, the underlying electrical properties of a hardware constituent are changed, for example, from an insulator to a conductor or vice versa. The instructions enable embedded hardware (e.g., the execution units or a loading mechanism) to create membersDocket No.6063.002WO1 of the circuit set in hardware via the variable connections to carry out portions of the specific operation when in operation. Accordingly, the computer readable medium is communicatively coupled to the other components of the circuit set member when the device is operating. In an example, any of the physical components may be used in more than one member of more than one circuit set. For example, under operation, execution units may be used in a first circuit of a first circuit set at one point in time and reused by a second circuit in the first circuit set, or by a third circuit in a second circuit set at a different time.

[0134] In various examples, portions of the functions of the controller circuit 120 may be implemented as a part of a microprocessor circuit. The microprocessor circuit can be a dedicated processor such as a digital signal processor (DSP), application specific integrated circuit (ASIC), microprocessor, or other type of processor for processing information including physical activity information. Alternatively, the microprocessor circuit can be a general purpose processor or combinational logic that can receive and execute a set of instructions for performing the methods or techniques described herein.

[0135] The controller circuit 120 may be communicatively coupled to the embedded sensor platform 110, and receive therefrom various sensor data, and perform one or more agitation characterization and quality control tasks with respect to manufacturing, packaging, and handling of pharmaceutical products. In some examples, the controller circuit 120 (or the microprocessor circuit that implements the controller circuit 120) may be included in a controller or computing device separated from and external to the container being inspected, such as a personal computer or a mobile device. The embedded sensor platform 110 may include a communication circuit configured to be communicatively coupled to the controller device. Information collected by the sensor circuitry may be transmitted to the controller device via a wired communication link or a wireless communication link (e.g., Bluetooth connection).

[0136] The agitation analyzer 121 may analyze, among other things, information collected by the accelerometer 112 during the agitation of the pharmaceutical container and generate container motion and simulated fluid characteristics 122 based on the analysis results. In an example, the agitation, either manually or robotically activated, may include a spin or rotation of the container, followed by an immediate halt of action. This may produce an effectDocket No.6063.002WO1 that the pharmaceutical liquid continues to swirl even after the immediate halt. The container motion and fluid characteristics 122 may include motion and position information of the container (also referred to as container-perspective motion characterization), such as a position, an orientation or tilt angle, a velocity, a displacement, or a range of movement of the container at multiple time instances during the agitation process. Raw acceleration data collected by the accelerometer 112 can include an acceleration component and a quaternion component. In an example, the acceleration data may be processed to convert acceleration data from moving frames to inertial frames using quaternion components. Temporal drift or baseline trends may be removed or attenuated from the acceleration component using a filter circuit, such as a least-mean- squares (LMS) adaptive filter. Alternatively or additionally, digital signal processing techniques such as a linear discontinuous fit may be used to de-trend the acceleration component. As an example, a linear fit over short durations of the acceleration signal can be determined, and then subtracted from the acceleration signal to remove the drift. The filtered or de-trended acceleration data may be integrated to obtain a velocity signal. The velocity signal may be integrated to obtain a displacement signal of the container. The integration operation may be performed using trapezoidal rule. Referring to FIGS.4A-4B, the diagrams therein illustrate examples of raw and processed (e.g., integrated) acceleration data before (FIG. 4A) and after (FIG. 4B) drift removal or signal de-trending. A three-axis or six-axis accelerometer collects raw acceleration time-series 410A in x-, y-, and z-directions when a robotic agitating system agitates a pharmaceutical container (e.g., a vial) in a repetitive sinusoidal motion. An LMS (least mean square) filter is used to detrend the acceleration time-series 410A to produce the detrended acceleration time-series 410B. Velocity time-series 420A and 420B can be obtained by integrating respective acceleration data. Displacement time-series 430A and 430B can be obtained by integrating respective velocity data.

[0137] For the motion data to be useful for characterizing the agitation process, or manufacturing processes in general, the position of the container needs to be accurately mapped throughout the agitation process. Device position can be inferred using, for example, Indoor Positioning System (IPS) technologies, such as Ultra-Wide Band (UWB) radio, Bluetooth or WiFiDocket No.6063.002WO1 localization, which can produce time-stamped positions with an adequate accuracy. These technologies, however, generally require additional devices and infrastructure to be attached on or close to the production or testing equipment bench of a manufacturing system (e.g., the bench where a container rests and is agitated). Computer vision systems have also been used for position tracking, which also requires additional equipment. Furthermore, these localization solutions are also vulnerable to blind spots or restricted access in densely constructed machines.

[0138] The agitation analyzer 121 may determine the position and motion of the container by leveraging multiple synchronized sensor signals. In an example, the agitation analyzer 121 may infer position of the container by analyzing the sensor data directly. Pattern recognition, thresholding and other signal processing techniques can be used to identify key phases of container manufacturing and testing. These checkpoints may be used to synchronize data acquisition from different sensors and infer the container’s location in the production or testing equipment. For example, sensor data can be time-stamped and associated with events during the container inspection and packaging process (e.g., beginning or end of spinning, inverting the container, or exposing the container to light). By temporal and spatial tracking and identifying matching events, positions of the container during the inspection and packaging process may be inferred from the sensor data. Additionally or alternatively, data obtained from one sensor may be used to calibrate data collected by another different sensor. For example, motion information and optical information may be simultaneously collected from, respectively, the accelerometer 112 and the optical sensor 114. Information collected by the optical sensor 114 may be used to calibrate or adjust the container position information determined based on the accelerometer 112, or to synchronize data acquisition by the accelerometer 112. In another example, acoustic data collected by the acoustic sensor 116 at a checkpoint may be used to synchronize collection of the container motion or position information by the embedded accelerometer. With the multiple synchronized sensor signals as described herein, data drift present in a sensor signal (e.g., accelerometer data) can be periodically aligned and corrected for, and more precise container location can be determined. FIG. 5 illustrates an example of using synchronized sensor signals 500 to determine or calibrateDocket No.6063.002WO1 container position during an inspection process, where a container can be moved mechanically in front of a light, and photodiode signals are synchronized with the movement signals at a known point in the machine to allow an “anchor point” to be used for correcting for any drift in the signals. Such anchor point can be a specific mechanical movement that may be used to identify the container position when the container is transferred between inspection and packaging operations. During the operation, the container can be agitated in a circular motion around a star wheel. A three-axis accelerometer embedded within the container collects motion / acceleration data during the agitated rotation, which can be integrated to produce a sinusoidal displacement signal 530 in the x-axis and a sinusoidal displacement signal 540 in the y-axis. A photodiode embedded in the interior of the container can detect a strobing light intensity signal 520 from the inspection station. As illustrated, a step change 522 in the light intensity signal 520 may be used as a reference to locate and timestamp the position of a container in the inspection station, which can then be used to locally correct for any drift in the accelerometer output.

[0139] In addition to the container-perspective motion characterization, the container motion and simulated fluid characteristics 122 may further include, in some examples, fluid dynamics characteristics modeled for a contained pharmaceutical liquid during the agitated motion. Examples of the fluid dynamics characteristics include a shear stress distribution, a flow pattern, or a velocity profile of the contained pharmaceutical liquid. The container motion and simulated fluid characteristics 122 may further include particle motion and separation, cavitation and bubble formation and distribution, etc. In an example, the agitation analyzer 121 may apply the collected container motion and position information to a computational fluid dynamics (CFD) model to determine one or more of fluid dynamics characteristics, and to determine agitation parameters corresponding to optimal or desired fluid dynamics characteristics. The CFD model can provide quantitative predictions of fluid-flow phenomena based on the conservation laws (conservation of mass, momentum, and energy) governing fluid motion. Examples of the CFD model include finite-difference and finite- element methods, spectral method and the spectral element method, among other numerical methods. As described above, the embedded sensor platform 110 can collect container motion and position information during experimental agitationDocket No.6063.002WO1 of the container empty of liquid product in the container. The motion and position information thus collected can be fed into the CFD model, which makes predictions for behaviors of normal pharmaceutical liquid-filled containers, and infers liquid motion as they go through regular agitation and inspection processes, yet without direct measurement of liquid behavior.

[0140] In an example, as illustrated in FIG. 3, images or video sequences of a human operator performing manual agitation during a MVI process may be captured using a motion capture system comprising one or more fixed cameras, which may then be used for characterizing agitation. Agitation characterization may include pose estimation and motion tracking of the operator derived from the images of video sequences. FIG. 3 illustrates a plurality of cameras positioned at different locations relative to the container held by a human operator. The cameras can image the operator’s manual agitation motion from different angles or perspectives. In some examples, the cameras may capture the operator’s full-body motion during the agitation process. Ergonomic information, including pose estimation and motion tracking, may be derived from the images or videos.

[0141] The embedded sensors (e.g., accelerometer 112) generally provide “local” information about the container’s position and motion. Information collected by the motion capture system, such as images of the agitation of the container and operator ergonomic information produced by the fixed camera(s), may be used to complement the container motion and position information produced by the embedded sensors such as the embedded accelerometer in the sensor platform 110. For example, the ergonomic information can be combined or synchronized with data from the embedded sensors (e.g., the accelerometer 112) to provide comprehensive, complete characterization of the agitation process, which can in turn be used to optimize ergonomics and efficiency, or to inform the design and development of automated alternatives. Ergonomic information acquisition and pose estimation and tracking are advantageous due to the ease of implementation and low operation cost. It can also be used to track operator’s agitation process in their native work environment and in their normal clothes without the need for additional sensors or accessories. Images or video sequences from multiple cameras placed at different locations (as illustrated inDocket No.6063.002WO1 FIG. 3) may be processed (e.g., correlated) to provide additional information such as depth in the image.

[0142] The agitation analyzer 121 can generate an agitation protocol 123 based on the container motion and simulated fluid characteristics 122. The agitation protocol 123 may include an assembly of optimal or desired agitation parameters (including, for example, a position, an orientation, a velocity, a displacement, or a range of movement, of the container, which can be obtained from an MVI process operated by a human expert), or instructions to a human operator or a robotic agitation system to be followed or referenced during an MVI or AVI process. The agitation protocol may be formulated as a motion parameter set or vector, a mathematical motion model (e.g., equations describing circular, elliptical, or helical motion path), or a navigation path such as a “cookie crumb” trail that tracks locations of the container over time during agitation. The agitation protocol can be interpretable and executable by a machine, such as the robotic system 150. In some examples, depending on the movement pattern, different agitation protocols may be generated, such as a helical motion protocol, a circular motion protocol, an elliptical motion protocol, a conical spiral motion protocol, a flicking motion protocol, or a flipping motion protocol, or combinations thereof among others. Beyond the alignment of robotic inspection systems with a human expert-initiated MVI process, the characterization of MVI has applications in assessing existing MVI process efficacy, understanding how different operators approach their role within established guidelines and use tools with embedded sensor platforms, and providing data for future training materials and regulatory engagement. The system and methods as described herein may also be used for root-cause investigations (e.g. non-conformances associated with observed glass breakage events) and also general manufacturing system characterization, either during design, construction, install and qualification, or during routine production as a maintenance / performance check.

[0143] The anomaly / defect detector 124 can detect anomalies or defects associated with manufacturing and quality control of the container and aid in predicting of the effect of anomalies on a pharmaceutical liquid contained therewithin. By way of example and not limitation, the anomaly / defect detector 124 may include one or more of a light exposure detector 125 or a glass breakage detector 126. An anomaly / defect detector 124 in an interior of aDocket No.6063.002WO1 container empty of pharmaceutical product (e.g., a first container) can be used to detect an anomaly of another container filled with pharmaceutical product (e.g., a second container). For example, if light exposure is detected by a detector in an interior of an empty container, it can be predicted that containers filled with pharmaceutical product would also be affected by light exposure during inspection. In another example, if a glass breakage event is detected by a detector in an interior of an empty container, it can be predicted that containers filled with pharmaceutical product experienced a glass breakage event or potentially would experience a glass breakage event.

[0144] The light exposure detector 125 may be operatively coupled to the optical sensor 114 to detect light intensity irradiated on the container and aid in determining excessive light exposure to a pharmaceutical liquid in the container. In some examples, the light exposure detector 125 may detect intensity of light at a particular wavelength or wavelength range or covering a specific electromagnetic spectrum hazardous to a pharmaceutical liquid, such as ultraviolet (UV) range. High-frequency UV light may cause damage to many liquid pharmaceutical products at the molecular level through photochemistry. Prolonged exposure to lights of high intensity may also cause heating of the product through absorption.

[0145] Conventional light intensity detection involves a hand-held Lux meter to measure light intensities. The Lux measurement may aggregate across wavelengths, and may not provide detailed spectral information, which is important since higher-frequency light is in general more likely to be harmful to many liquid pharmaceutical liquids. Moreover, Lux measurements are generally not conducted from the perspective of actual drug product inside a container, and do not capture the interaction of incident light with the container itself.

[0146] Automatic light intensity detection and excessive exposure determination enabled by the light exposure detector 125 as described herein improves the effectiveness and efficiency of light intensity detection and excessive exposure determination processes. In an example, the optical sensor 114 may include an array of photosensors coated with filter materials, or specially made photosensors, to concentrate the optical measurement on specific wavelengths or wavelength ranges, such as UV light which is of particular concern for the pharmaceutical industry due to the risk of high-energy lightDocket No.6063.002WO1 inducing photochemical degradation at the molecular level. In one example, the optical sensor 114 may include a linear sensor array running a longitudinal portion of the container. In another example, the optical sensor 114 may include a two-dimensional (2D) sensor array spreading over at least a portion of the inner surface of the container. Such optical sensor array may advantageously characterize light accurately across the full volume of the container. In some examples, individual photosensors in the optical sensor 114 may be configured to cover specific parts of the electromagnetic spectrum. Examples of the optical sensor 114 and sensor packaging within a container are described below with reference to FIGS.2C-2E.

[0147] The glass breakage detector 126 may be coupled to the one or more sensors in the embedded sensor platform 110 to detect glass breakage of at least a glass portion of the container. Glass breakage can occur on a wide range of different parts of a manufacturing and packaging process and has been observed on fill lines, inspection lines and during device assembly and packaging. Causes of glass breakage may include native weaknesses present in incoming glass materials, deviations in the shape of the glassware, or slight misalignments in production equipment. Generally, glass breakage events are observed after the fact, during unit count reconciliation and line clearance activities. Once an event has been observed, substantial line-time is usually required to observe the equipment in detail to try and pinpoint the root cause. To fully capture glass breakage events accurately, machinery typically is required to run at full speed, which often limits the value of human observation. Glass breakage events can be intermittent and may require relatively long test runs just to capture a single incident. Glass breakage events may be detected using vibration data collected by the accelerometer 112, or acoustic data collected by the acoustic sensor 116 (e.g., a microphone or an ultrasound sensor), in response to manipulation stresses on the container. Spectral analysis of the vibration data or the acoustic data may be performed to detect a glass breakage event. To improve detection accuracy, sensor(s) for detecting glass breakage may be embedded in or affixed at certain strategic locations of the interior of the container being inspected, such as the base of the container in one example. In an example, the accelerometer 112 may be aligned with the center-of-mass of the container. Location of the optical sensor 114 may depend at least in part onDocket No.6063.002WO1 specific use case. As will be discussed further below, artificial intelligence (AI) or machine learning (ML) technologies may be used to detect glass breakage, among other anomalies or defects.

[0148] Data from a glass breakage detector 124 can be synchronized with data from the accelerometer 112 so that the system can identify motions or positions that are associated with a glass breakage event. By associating glass breakage events with motion or position information from the accelerometer 112, the system can identify agitation parameters (e.g., position, orientation, velocity, acceleration, displacement, range of motion) that are associated with a glass breakage event. Using this synchronized data, the system can generate an agitation protocol that avoids motions or positions that may cause glass breakage events.

[0149] Image or video-based glass breakage detection may face challenges in practice. For example, capture of this video sequence may take several weeks of time on the factory floor (in addition to several additional weeks to development of the test rig) and generally require careful estimation by experts on where the event might be occurring, and required the temporary integration of cameras inside the machine. Furthermore, some production systems are not amenable to the introduction of cameras or other diagnostic imaging devices, whether due to moving parts, aseptic enclosures or dense or otherwise unfavorable layouts. In contrast, embedded sensors, such as accelerometer 112 and acoustic sensor 116, packaged within a sealed product unit (e.g., the embedded sensor platform 110), may address various issues associated with image or video-based glass breakage detection system. In one example, one or more embedded accelerometers can be run through a suspectedly problematic system at normal speed and detect and locate glass breakage events as they happen. Accelerometer data can be collected in real time via the communication circuit (e.g., Bluetooth or other wireless communication circuit) on the embedded sensor platform 110. In addition to alternative to in situ breakage detection and diagnostics, in some examples the glass breakage events may be detected and resolved at the end of the run by examining the sensor data recorded by the sensor.

[0150] In various examples, an anomaly / defect detector 124, or a component therein, may be coupled to an ML / AI system 160 to detect one orDocket No.6063.002WO1 more anomalies or defects using one or more trained ML model(s) 162 trained using a training module. The trained ML model(s) 162 may be stored in a storage device. In an example, the ML / AI system 160 may be a remote computing and storage system, such as a cloud comprising one or more computing devices (e.g., servers) configured to provide cloud-based services including, for example, data storage, computing services, and provisioning of customer services, among others. The controller circuit 120, or portions thereof such as the agitation analyzer 121 or the anomaly / defect detector 124, may remotely and securely access the data and services, such as the trained ML model(s) 162, from the ML / AI system 160.

[0151] In an example, the trained ML model(s) 162 can have a neural network structure. The neural network structure can include an input layer, one or more hidden layers, and an output layer. Sensor data about the pharmaceutical container collected from one or more sensors implemented in the embedded sensor platform 110 can be fed into the input layer of the trained ML model(s) 162, get manipulated through one or more hidden layers, and output from the output layer. The trained ML model(s) 162 can provide the system 100 with the ability to perform various tasks including objection recognition and feature extraction, without explicitly being programmed, by making inferences based on patterns found in the analysis. The trained ML model(s) 162 may learn from existing data and make predictions about new data. Such algorithms operate by building the trained ML model(s) 162 from training dataset in order to make data-driven predictions or decisions expressed as outputs or assessments.

[0152] The ML model(s) 162 may be trained using a training dataset. The training dataset may include sensor data such as obtained from one or more sensors included in the embedded sensor platform 110. The training module can train the ML model(s) using supervised learning or unsupervised learning. Supervised learning uses prior knowledge (e.g., examples that correlate inputs to outputs or outcomes) to learn the relationships between the inputs and the outputs. The goal of supervised learning is to learn a function that, given some training data, best approximates the relationship between the training inputs and outputs so that the ML model can implement the same relationships when given inputs to generate the corresponding outputs. When the ML model is trained using supervised learning, the training dataset may further include “desired”Docket No.6063.002WO1 output, such as known light exposure information (for training an ML model to be used by the light exposure detector 125), or glass breakage information (for training an ML model to be used by the glass breakage detector 126). The “desired” output may be stored in a database. Unsupervised learning is the training of an ML algorithm using information that is neither classified nor labeled, and allowing the algorithm to act on that information without guidance. Unsupervised learning is useful in exploratory analysis because it can automatically identify structure in data.

[0153] Common tasks for supervised learning are classification problems and regression problems. Classification problems, also referred to as categorization problems, aim at classifying items into one of several category values. Regression algorithms aim at quantifying some items (for example, by providing a score to the value of some input). Some examples of commonly used supervised-ML algorithms are Logistic Regression (LR), Naive-Bayes, Random Forest (RF), neural networks (NN), deep learning, deep neural networks (DNN), matrix factorization, and Support Vector Machines (SVM). The deep learning or DNN generally refers to a neural network that consists of multiple (e.g., more than three) hidden layers. Examples of DNN include a convolutional neural network (CNN), a recurrent neural network (RNN), a deep belief network (DBN), or a hybrid neural network comprising two or more neural network models of different types or different model configurations. Some common tasks for unsupervised learning include clustering, representation learning, and density estimation. Some examples of commonly used unsupervised learning algorithms are K-means clustering, principal component analysis, and autoencoders.

[0154] Another type of ML is federated learning (also known as collaborative learning) that trains an algorithm across multiple decentralized devices holding local data, without exchanging the data. This approach stands in contrast to traditional centralized machine-learning techniques where all the local datasets are uploaded to one server, as well as to more classical decentralized approaches which often assume that local data samples are identically distributed. Federated learning enables multiple actors to build a common, robust machine learning model without sharing data, thus allowing to address critical issues such as data privacy, data security, data access rights and access to heterogeneous data.Docket No.6063.002WO1

[0155] The training of the ML model may be carried out continuously or periodically, or in near real time as additional sensor data become available. The training involves algorithmically adjusting one or more ML model parameters, until the ML model being trained satisfies a specified training convergence criterion, such as a difference between the model output (for the given input) and the “desired” output falling below a specified threshold value.

[0156] In some examples, a plurality of ML models can be separately trained, validated, and used (in an inference phase) to achieve different goals. In an example, a first ML model may be trained, and used by the light exposure detector 125, to detect light intensity and to determine excessive light exposure to liquid pharmaceutical product. A second ML model may be trained, and used by the glass breakage detector 126, to detect glass breakage and to determine the root cause thereof. For example, data from a glass breakage detector 124 can be synchronized with data from the accelerometer 112 so that system and the ML model can identify agitation parameters that are associated with a glass breakage event. The separately trained ML models may differ by a model architecture (e.g., number of hidden layers, number of neurons in any layer), or at least one model parameter (e.g., weights applied to any neurons or bias introduced in any layer).

[0157] Anomalies or defects detected by one or more of the light exposure detector 125 or the glass breakage detector 126 may be combined to produce quality control diagnostics 128. The quality control diagnostics 128 and the agitation protocol 123 may be provided to a user, such as via the user interface 140. The user interface 140 may include an output unit and an input unit. The output unit may include a display to display the quality control diagnostics 128 and the agitation protocol 123, among others. Sensor data from the embedded sensor platform 110 may also be displayed to the user. The quality control diagnostics 128 (including the detected anomalies or defects) may trigger an alert, or a recommendation to the user to remedy the detected anomaly or defect, such as further inspection, repairs, parts ordering, initiating an automated maintenance process, or equipment shutdown, among others. The input unit of the user interface 140 may receive a user input to program a parameter or adjust a component of the system 100. In some examples, the input unit may receive from the user, or from an external source or storage, auxiliary input that may beDocket No.6063.002WO1 used to train the ML model(s). The quality control diagnostics 128 and the agitation protocol 123 may additionally or alternatively be provided to a robotic system 150 to facilitate robotic agitation and inspection of containers of different types or formats. In some examples, machine health and mechanics of machine parts of the robotic system 150 may be evaluated using sensor data collected from the embedded sensor platform 110. For example, if the speed or acceleration of a machine part for activating robotic agitation of a container is determined to deviate from a normal range, a machine anomaly may be flagged to indicate further check or maintenance is warranted.

[0158] FIGS. 2A-2E illustrate examples of embedded sensor platforms customized to be removably embedded within or attached to a container empty of liquid product. The container can be an actual container product (e.g., a vial or a syringe) for holding liquid pharmaceutical product, or a fabricated alternative with matching optical and mechanical properties. The embedded sensor platform contains sensors that may collect data used for, among other things, characterizing agitation motion and developing an agitation protocol for use in subsequent agitation and inspection of containers filled with liquid pharmaceutical products. FIG. 2A illustrates an example of embedded sensor platform 210 with a profile customized to be removably embedded within a syringe 200A. FIG. 2B illustrates another example of an embedded sensor platform 220 with a profile customized to be removably embedded within a vial 200B. The embedded sensor platforms 210 and 220, which are embodiments of the embedded sensor platform 110 of FIG. 1, can be configured to support one or more sensors to collect information about the container, including an embedded accelerometer in an interior of the container empty of liquid product in the container. The embedded accelerometer can collect container motion or position information during experimental agitation of the container. The container motion or position information may be used to characterize agitation motion. Data collected by the embedded sensors may also be used to detect anomalies or defects, among other inspection and quality control tasks.

[0159] FIG.2C illustrates a vial 200C including a cap and crimp 231, a vial body 233, a seal 232 to tightly secured between an opening of the vial body 233 and the cap and crimp 231. The seal 232 can be made of rubber, plastic, or other materials. The cap and crimp 231 and the seal 232 can have matchingDocket No.6063.002WO1 dimensions. In a non-limiting example, the cap and crimp 231 is a conventional 13 millimeter (mm) cap and crimp, and the seal 232 is a conventional 13 mm rubber seal. The vial 200C can be an actual vial for holding liquid pharmaceutical product, or a fabricated alternative with matching optical and mechanical properties. The vial body 233 can be made of glass or other transparent or semitransparent materials. In an example, the vial body 233 is a sawn-off 2R ISO glass vial. An embedded sensor platform 230 (an embodiment of the embedded sensor platform 110 of FIG. 1) may include a flexible photovoltaic membrane 234 and a sense and control circuit board 235, both sized and shaped to be removably embedded within the liquid-containing space within the vial body 233. The flexible photovoltaic membrane 234 and the sense and control circuit board 235 can be mounted on and secured by a base unit 237 attached to the base of the vial body 233. In an example, the base unit 237 is a 3D-printed substitute vial base to provide mechanical support for the flexible photovoltaic element or flexible photovoltaic membrane 234. A variable ballast module 236 may be coupled between a distal portion (approaching the vial base) of the embedded sensor platform (e.g., a distal portion of the flexible photovoltaic membrane 234 and / or the distal portion of the sense and control circuit board 235) and the base unit 237 to stabilize the embedded sensor platform 230 (including the flexible photovoltaic membrane 234 and the sense and control circuit board 235) within the vial body 233.

[0160] The flexible photovoltaic membrane 234 comprises photovoltaic sensors configured to measure light intensities in a specific spectral operating range. The measurements may be used to characterize interaction of the light on the container and can be used to determine a risk of prolonged or excessive light exposure, which may cause heating or other quality issues associated with the container or a pharmaceutical liquid contained within.

[0161] The flexible photovoltaic membrane 234 has a membrane body made of flexible materials to facilitate deformation and customary fit within the vial body 233, and to conform to an interior surface of a cylindrical portion of the vial body 233. In the illustrated example, a two-dimensional (2D) sensor array comprising horizontal rings of photovoltaic sensors may be distributed around and completely or substantially (e.g., with at least 80-90% of circumferential surface area) cover the container’s interior surface. The ring-shape photovoltaicDocket No.6063.002WO1 sensor array ensures the measurement is not uni-directional. In an example, the flexible photovoltaic membrane 234 may include multiple small, isolated segments with photovoltaic sensor arrays (e.g., photodiode arrays) spatially distributed over a range of surface areas of the vial body 233. This is advantageous over the photovoltaic sensors which generally lack spatial feedback (i.e., light intensities at different spatial locations of the vial body 233).

[0162] The sense and control circuit board 235 may include one or more sensors, such as the accelerometer 112, the optical sensor 114, and / or the acoustic sensor 116. The sense and control circuit board 235 may further include a microcontroller, and a communication circuit for data communication between the embedded sensors and an external controller or computing device (not shown), as described above with reference to FIG. 1.

[0163] The embedded sensor platform, including the flexible photovoltaic membrane and / or the sense and control circuit board 235, may be powered by a battery embedded within or attached to the vial 200C. FIG. 2D illustrates a perspective view of a vial 200D that, similar to the vial 200C, contains the embedded sensor platform 230 that includes the flexible photovoltaic membrane 234 and / or the sense and control circuit board 235, as shown in FIG. 2C. FIG. 2E illustrates a side view of the vial 200D. A battery 241 may power the embedded sensor platform 230 (or a portion thereof such as sensors, data collection and communication circuitry, etc.). In the illustrated example, the battery 241 and affiliated electronics may be embedded inside a chamber of the cap and crimp 231. Alternatively, in some examples, the battery 241 and affiliated electronics may be secured at other locations, such as the vial base. The battery 241 and affiliated electronics can be insulated from liquid or moisture using insulating materials. In an example, the battery can be a rechargeable battery.

[0164] FIG. 6 is a flow chart illustrating an example method 600 of characterizing inspection and quality control of processes of liquid pharmaceutical product manufacturing and packaging in containers. The method 600 may be carried out using the sensor-based pharmaceutical product inspection and quality control system 100. Although the processes of the method 600 are drawn in one flow chart, they are not required to be performed in aDocket No.6063.002WO1 particular order. In various examples, some of the processes can be performed in a different order than that illustrated herein.

[0165] At step 610, an embedded sensor platform, including an embedded accelerometer, may be mounted in an interior of a container empty of liquid pharmaceutical product, and agitation applied to the container. In an example, the container may be manually agitated in a manual visual inspection (MVI) process, where a human operator can flick, flip, or rotate the container, followed by an immediate halt of action. Alternatively, the container may be agitated automatically in an automated visual inspection (AVI) process, whereby a mechanical agitation system spins the container in a controller maneuver (e.g., an adjustable spinning speed) followed by an immediate halt of action to produce a swirl in the contained liquid pharmaceutical product during and after the agitation. The data obtained can then be used for other purposes.

[0166] At step 620, information of the container may be collected by embedded sensors, including container motion or position information sensed by the embedded accelerometer from an interior of the container during the agitation process. As described above with respect to FIG. 1, the sensor circuitry, including the embedded accelerometer, can be mounted on a base unit of an embedded sensor platform sized, shaped, and / or configured to be removably embedded within or attached to the pharmaceutical container. In an example, the embedded sensor platform may be sized and shaped to be removably inserted into and substantially (e.g., within 5-10% tolerance) conform to a cylindrical portion of the container, as illustrated in FIGS. 2C through 2E. The accelerometer may collect motion or position information of the container during agitation of the pharmaceutical container. The collected container motion or position information may include, for example, container position, orientation or tilt angle, rotation speed, among other information. The accelerometer may additionally or alternatively collect motion data of the agitated liquid during agitation of the of the pharmaceutical container. In some examples, a gyroscope may be used in addition or alternative to the accelerometer to collect motion or position information of the container and / or the motion data of the agitated liquid.

[0167] In addition to the container motion or position information collected by the accelerometer, other sensor data may be collected at step 620 viaDocket No.6063.002WO1 respective sensors included in the sensor circuitry and implemented in the embedded sensor platform. As described above with respect to FIG. 1, in an example, an optical sensor may be included to collect optical information from the container in response to light irradiation thereon. In another example, an acoustic sensor may be included to collect acoustic data in response to manipulation of the container during manufacturing, shipping, or handling of the liquid pharmaceutical product. The optical information and / or the acoustic information collected may be used to detect one or more anomalies or defects associated with the container.

[0168] In some examples, the information of the pharmaceutical container collected at step 620 may also be collected by one or more imaging sensors or fixed cameras other than those included in the embedded sensor platform.

[0169] At step 630, one or more agitation parameters may be determined using the collected container motion or position information, and an agitation protocol may be generated using the one or more agitation parameters. Examples of the agitation parameters include a position, an orientation, a velocity, a displacement, or a range of movement, of the container. In some examples, the collected container motion or position information may be fed into a computational fluid dynamics (CFD) model to make inference of fluid dynamics characteristics (e.g., a shear stress distribution, a flow pattern, or a velocity profile) of a contained liquid pharmaceutical product, and to predict behaviors of normal pharmaceutical liquid-filled containers, without direct measurement of liquid behavior. Examples of the CFD model include finite-difference and finite- element methods, spectral method and the spectral element method, among other numerical methods. One or more agitation parameters can then be determined based on the determined fluid dynamics characteristics.

[0170] In some examples, information collected by one or more imaging sensors or one or more fixed cameras in the environment may be used to assist in characterizing container motion and fluid dynamics during agitation. For example, imaging data of the container and / or the contained pharmaceutical liquid, and images or video sequences of a human operator agitating the pharmaceutical liquid, may be used to characterize container-perspective motion and / or fluid dynamics characteristics during the agitation process. Such information may be used to help determine the one or more agitation parameters.Docket No.6063.002WO1

[0171] An agitation protocol may be generated using the one or more agitation parameters. The agitation protocol may include an assembly of optimal or desired agitation parameters (such as obtained from an MVI process operated by a human expert). At step 640, the agitation protocol may be provided to a user or a mechanical system (e.g., a robotic system) to assist in subsequent agitation and inspection of liquid pharmaceutical products. Beyond the alignment of mechanical inspection systems with human expert-initiated MVI process, the characterization of MVI has applications in assessing existing MVI process efficacy, understanding how different operators approach their role within established guidelines and use tools with embedded sensor platforms, and providing data for future training materials and regulatory engagement.

[0172] At step 650, anomalies or defects associated with the container may be detected using the collected sensor data, which may be collected during, for example, the agitation of the pharmaceutical container. The anomaly or defect may be detected by the anomaly / defect detector 124 as described above with respect to FIG. 1. In some examples, AI or ML technology may be used to assist in the detection of one or more anomalies or defects, such as by using the ML / AI system 160. Various anomalies or defects may be detected. One example of anomalies or defects that can be detected is excessive light exposure in the container. Intensity of light irradiation on the container may be sensed using an optical sensor, such as the optical sensor 114. In some examples, intensity of light at a particular wavelength or wavelength range (e.g., high-frequency UV light), which is hazardous to a contained pharmaceutical liquid, may be sensed, and the level of exposure of such light may be determined. Another example of anomalies or defects that can be detected is glass breakage of at least a glass portion of the container, such as by using the glass breakage detector 126. Glass breakage can occur on a wide range of different parts of the process, and caused by native weaknesses present in incoming glass materials, deviations in the shape of the glassware, or slight misalignments in production equipment, among other factors. Glass breakage events may be detected based on sensor data including, for example, accelerometer data collected in response to manipulation of the container or acoustic data collected by an acoustic sensor (e.g., a microphone or an ultrasound sensor), as described above with respect to FIG. 1.Docket No.6063.002WO1

[0173] At step 660, quality control diagnostics may be generated based at least on the detected anomaly or defect, which can be provided to a user or a robotic system to assist in liquid pharmaceutical product inspection and quality control. For example, the quality control diagnostics may perform one or more of, trigger an alert or a recommendation to the user to remedy the detected anomaly or defect, such as further inspection, repairs, parts ordering, initiate a pre-determined maintenance process, or equipment shutdown, among other actions.

[0174] FIG. 7 illustrates generally a block diagram of an example machine 700 upon which any one or more of the techniques (e.g., methodologies) discussed herein may perform. Portions of this description may apply to the computing framework of various portions of the sensor-based liquid pharmaceutical product inspection and quality control system 100.

[0175] In alternative examples, the machine 700 may operate as a standalone device or may be connected (e.g., networked) to other machines. In a networked deployment, the machine 700 may operate in the capacity of a server machine, a client machine, or both in server-client network environments. In an example, the machine 700 may act as a peer machine in peer-to-peer (P2P) (or other distributed) network environment. The machine 700 may be a personal computer (PC), a tablet PC, a set-top box (STB), a personal digital assistant (PDA), a mobile telephone, a web appliance, a network router, switch or bridge, or any machine capable of executing instructions (sequential or otherwise) that specify actions to be taken by that machine. Further, while only a single machine is illustrated, the term “machine” shall also be taken to include any collection of machines that individually or jointly execute a set (or multiple sets) of instructions to perform any one or more of the methodologies discussed herein, such as cloud computing, software as a service (SaaS), among other computer cluster configurations.

[0176] Examples, as described herein, may include, or may operate by, logic or a number of components, or mechanisms. Circuit sets are a collection of circuits implemented in tangible entities that include hardware (e.g., simple circuits, gates, logic, etc.). Circuit set membership may be flexible over time and underlying hardware variability. Circuit sets include members that may, alone or in combination, perform specified operations when operating. In an example,Docket No.6063.002WO1 hardware of the circuit set may be immutably designed to carry out a specific operation (e.g., hardwired). In an example, the hardware of the circuit set may include variably connected physical components (e.g., execution units, transistors, simple circuits, etc.) including a computer readable medium physically modified (e.g., magnetically, electrically, moveable placement of invariant massed particles, etc.) to encode instructions of the specific operation. In connecting the physical components, the underlying electrical properties of a hardware constituent are changed, for example, from an insulator to a conductor or vice versa. The instructions enable embedded hardware (e.g., the execution units or a loading mechanism) to create members of the circuit set in hardware via the variable connections to carry out portions of the specific operation when in operation. Accordingly, the computer readable medium is communicatively coupled to the other components of the circuit set member when the device is operating. In an example, any of the physical components may be used in more than one member of more than one circuit set. For example, under operation, execution units may be used in a first circuit of a first circuit set at one point in time and reused by a second circuit in the first circuit set, or by a third circuit in a second circuit set at a different time.

[0177] Machine (e.g., computer system) 700 may include a hardware processor 702 (e.g., a central processing unit (CPU), a graphics processing unit (GPU), a hardware processor core, algorithm specific ASIC, or any combination thereof), a main memory 704 and a static memory 706, some or all of which may communicate with each other via an interlink (e.g., bus) 708. The machine 700 may further include a display unit 710 (e.g., a raster display, vector display, holographic display, etc.), an alphanumeric input device 712 (e.g., a keyboard), and a user interface (UI) navigation device 714 (e.g., a mouse). In an example, the display unit 710, input device 712 and UI navigation device 714 may be a touch screen display. The machine 700 may additionally include a storage device (e.g., drive unit) 716, a signal generation device 718 (e.g., a speaker), a network interface device 720, and one or more sensors 721, such as a global positioning system (GPS) sensor, compass, accelerometer, or other sensors. The machine 700 may include an output controller 728, such as a serial (e.g., universal serial bus (USB), parallel, or other wired or wireless (e.g., infrared (IR), near fieldDocket No.6063.002WO1 communication (NFC), etc.) connection to communicate or control one or more peripheral devices (e.g., a printer, card reader, etc.).

[0178] The storage device 716 may include a machine-readable medium 722 on which is stored one or more sets of data structures or instructions 724 (e.g., software) embodying or utilized by any one or more of the techniques or functions described herein. The instructions 724 may also reside, completely or at least partially, within the main memory 704, within static memory 706, or within the hardware processor 802 during execution thereof by the machine 700. In an example, one or any combination of the hardware processor 702, the main memory 704, the static memory 706, or the storage device 716 may constitute machine readable media.

[0179] While the machine-readable medium 722 is illustrated as a single medium, the term "machine readable medium" may include a single medium or multiple media (e.g., a centralized or distributed database, and / or associated caches and servers) configured to store the one or more instructions 724.

[0180] The term “machine readable medium” may include any medium that is capable of storing, encoding, or carrying instructions for execution by the machine 700 and that cause the machine 700 to perform any one or more of the techniques of the present disclosure, or that is capable of storing, encoding or carrying data structures used by or associated with such instructions. Non- limiting machine-readable medium examples may include solid-state memories, and optical and magnetic media. In an example, a massed machine-readable medium comprises a machine readable medium with a plurality of particles having invariant (e.g., rest) mass. Accordingly, massed machine-readable media are not transitory propagating signals. Specific examples of massed machine- readable media may include: non-volatile memory, such as semiconductor memory devices (e.g., Electrically Programmable Read-Only Memory (EPROM), Electrically Erasable Programmable Read-Only Memory (EPSOM)) and flash memory devices; magnetic disks, such as internal hard disks and removable disks; magneto-optical disks; and CD-ROM and DVD-ROM disks.

[0181] The instructions 724 may further be transmitted or received over a communication network 726 using a transmission medium via the network interface device 720 utilizing any one of a number of transfer protocols (e.g., frame relay, internet protocol (IP), transmission control protocol (TCP), userDocket No.6063.002WO1 datagram protocol (UDP), hypertext transfer protocol (HTTP), etc.). Example communication networks may include a local area network (LAN), a wide area network (WAN), a packet data network (e.g., the Internet), mobile telephone networks (e.g., cellular networks), Plain Old Telephone (POTS) networks, and wireless data networks (e.g., Institute of Electrical and Electronics Engineers (IEEE) 802.11 family of standards known as WiFi®, IEEE 802.16 family of standards known as WiMax®), IEEE 802.15.4 family of standards, peer-to-peer (P2P) networks, among others. In an example, the network interface device 820 may include one or more physical jacks (e.g., Ethernet, coaxial, or phone jacks) or one or more antennas to connect to the communication network 726. In an example, the network interface device 720 may include a plurality of antennas to wirelessly communicate using at least one of single-input multiple-output (SIMO), multiple-input multiple-output (MIMO), or multiple-input single-output (MISO) techniques. The term “transmission medium” shall be taken to include any intangible medium that is capable of storing, encoding or carrying instructions for execution by the machine 700, and includes digital or analog communications signals or other intangible medium to facilitate communication of such software.

[0182] Various examples are illustrated in the figures above. One or more features from one or more of these examples may be combined to form other examples.

[0183] The method examples described herein may be machine or computer-implemented at least in part. Some examples may include a computer- readable medium or machine-readable medium encoded with instructions operable to configure an electronic device or system to perform methods as described in the above examples. An implementation of such methods may include code, such as microcode, assembly language code, a higher-level language code, or the like. Such code may include computer readable instructions for performing various methods. The code may form portions of computer program products. Further, the code may be tangibly stored on one or more volatile or non-volatile computer-readable media during execution or at other times.

[0184] The above detailed description is intended to be illustrative, and not restrictive. The scope of the disclosure should, therefore, be determined withDocket No.6063.002WO1 references to the appended claims, along with the full scope of equivalents to which such claims are entitled.

Claims

Docket No.6063.002WO1 What is claimed is:

1. A system for characterizing inspection and quality control processes of pharmaceutical product manufacturing and packaging in containers, the system comprising: sensor circuitry, comprising an embedded sensor platform that includes an embedded accelerometer in an interior of a container empty of pharmaceutical product, the embedded accelerometer configured to collect container motion or position information during agitation of the container; and a controller circuit configured to, based at least in part on the collected container motion or position information, generate an agitation protocol for agitating containers.

2. The system of claim 1, wherein the controller circuit includes executable instructions to calculate one or more agitation parameters using the collected container motion or position information, and to generate the agitation protocol using the calculated one or more agitation parameters.

3. The system of claim 2, wherein the one or more agitation parameters include at least one of a position, an orientation, a velocity, an acceleration, a displacement, or a range of movement of the container.

4. The system of any of claims 1-3, wherein the controller circuit is configured to provide the agitation protocol to a robotic system for subsequent agitation and inspection of containers filled with pharmaceutical products.

5. The system of any of claims 1-4, wherein the controller circuit is configured to program a robotic system with the agitation protocol for subsequent agitation and inspection of containers filled with pharmaceutical products.

6. The system of any of claims 1-5, wherein the controller circuit is configured to provide the agitation protocol to a user to assist in subsequent agitation and inspection of containers filled with pharmaceutical products.Docket No.6063.002WO1 7. The system of any of claims 1-6, wherein the controller circuit is configured to apply the collected container motion or position information to a computational fluid dynamics (CFD) model to make predictions of fluid dynamics characteristics of the pharmaceutical product, and to determine one or more agitation parameters based at least in part on the predictions of fluid dynamics characteristics.

8. The system of claim 7, wherein the predictions of fluid dynamics characteristics include predictions of at least one of a shear stress distribution, a flow pattern, or a velocity profile of the pharmaceutical product.

9. The system of any of claims 1-8, wherein the embedded sensor platform is sized and shaped to be removably inserted into and substantially conform to a cylindrical portion of the interior of the container.

10. The system of claim 9, wherein the embedded accelerometer is configured to be aligned to a longitudinal axis of the cylindrical portion of the interior of the container.

11. The system of any of claims 1-10, wherein the embedded accelerometer is configured to be mounted on a base unit of the embedded sensor platform.

12. The system of claim 11, wherein the embedded accelerometer is configured to be positioned at substantially a center of mass of the embedded system.

13. The system of any of claims 1-12, comprising an external control device external to the container and including at least a portion of the controller circuit, the external control device configured to communicate with the embedded sensor platform via a wireless communication link.

14. The system of any of claims 1-13, wherein the controller circuit is configured to remove or attenuate a temporal drift from the collected container motion or position information.Docket No.6063.002WO1 15. The system of any of claims 1-14, wherein the sensor circuitry includes an optical sensor configured to collect optical information from the container.

16. The system of claim 15, wherein the controller circuit is configured to use the collected optical information to synchronize collection of the container motion or position information, or to calibrate the container motion or position information collected by the embedded accelerometer.

17. The system of claim 16, wherein to calibrate the container motion or position information, the controller circuit is configured to remove or attenuate a temporal drift in the container motion or position information.

18. The system of any of claims 1-17, wherein the sensor circuitry is electrically coupled to a motion capture system configured to collect ergonomic information of an operator agitating the contained pharmaceutical product, wherein the controller circuit is configured to estimate a pose or to track motion of the operator using the collected ergonomic information, and to generate the agitation protocol further based on the estimated pose or the tracked motion of the operator.

19. The system of claim 18, wherein the motion capture system comprises a plurality of spatially distributed imaging sensors configured to collect the ergonomic information from respective different positions or angles with respect to the operator.

20. The system of any of claims 1-19, comprising a user interface configured to present the agitation protocol to a user to assist in subsequent manual agitation and inspection of the containers filled with pharmaceutical products.

21. The system of any of claims 1-17, wherein the controller circuit is configured to generate a control signal to a robotic system to initiate automatic agitation and inspection of the containers filled with pharmaceutical products.Docket No.6063.002WO1 22. The system of any of claims 1-21, wherein the sensor circuitry includes one or more embedded optical sensors at the interior of the container, the one or more embedded optical sensors configured to sense a light intensity, in a specific wavelength or wavelength range, irradiated on the container.

23. The system of claim 22, wherein the controller circuit is configured to detect light exposure of the pharmaceutical products contained in the containers based on the sensed light intensity.

24. The system of any of claims 22-23, wherein the one or more embedded optical sensors include a photosensor array along a length of a cylindrical portion of the container.

25. The system of any of claims 22-24, wherein the one or more embedded optical sensors include an insertable photovoltaic membrane configured to be inserted into and substantially conform to an inner surface of a cylindrical portion of the container.

26. The system of any of claims 1-25, wherein the sensor circuitry includes an embedded acoustic sensor at the interior of the container, the embedded acoustic sensor configured to collect acoustic data in response to manipulation of the container, wherein the controller circuit is configured to detect a glass breakage event in a glass portion of the container using the collected acoustic data.

27. The system of claim 26, wherein the controller circuit is configured to use the collected acoustic data to synchronize collection of the container motion or position information.

28. The system of any of claims 1-27, wherein the embedded accelerometer is further configured to collect vibration data in response to manipulation of the container, wherein the controller circuit is configured to detect a glass breakage event in a glass portion of the container using the collected vibration data.Docket No.6063.002WO1 29. The system of any of claims 1-28, wherein the agitation protocol includes a helical motion protocol, a circular motion protocol, an elliptical motion protocol, a conical spiral motion protocol, a flicking motion protocol, or a flipping motion protocol.

30. A method of characterizing inspection and quality control of processes of pharmaceutical product manufacturing and packaging in containers, the method comprising: introducing agitation to a container empty of pharmaceutical product, the container associated with an embedded sensor platform including an embedded accelerometer in an interior of the container; collecting sensor data including container motion or position information sensed by the embedded accelerometer during the agitation of the container; and based at least in part on the collected container motion or position information, generating an agitation protocol for agitating containers.

31. The method of claim 30, comprising calculating one or more agitation parameters using the collected container motion or position information, wherein generating the agitation protocol includes using the calculated one or more agitation parameters.

32. The method of claim 31, wherein the one or more agitation parameters include at least one of a position, an orientation, a velocity, an acceleration, a displacement, or a range of movement of the container.

33. The method of any of claims 31-32, wherein calculating the one or more agitation parameters incudes making predictions of fluid dynamics characteristics of the pharmaceutical product by applying the collected container motion or position information to a computational fluid dynamics (CFD) model, and calculating the one or more agitation parameters based at least in part on the predictions of fluid dynamics characteristics.Docket No.6063.002WO1 34. The method of any of claims 30-33, further comprising providing the agitation protocol to a robotic system for subsequent agitation and inspection of containers filled with pharmaceutical products.

35. The method of any of claims 30-34, further comprising programming a robotic system with the agitation protocol for subsequent agitation and inspection of containers filled with pharmaceutical products.

36. The method of any of claims 30-35, further comprising providing the agitation protocol to a user to assist in subsequent agitation and inspection of containers filled with pharmaceutical products.

37. The method of any of claims 30-36, wherein the embedded sensor platform is inserted into and substantially conforms to a cylindrical portion of the interior of the container.

38. The method of claim 37, wherein the embedded sensor platform is positioned such that the embedded accelerometer is aligned to a longitudinal axis of the cylindrical portion of the interior of the container.

39. The method of any of claims 30-38, wherein the embedded sensor platform is secured on a base unit of the embedded sensor platform.

40. The method of claim 39, wherein the embedded sensor platform is positioned at substantially a center of mass of the embedded system.

41. The method of any of claims 30-40, further comprising removing or attenuating a temporal drift from the collected container motion or position information.

42. The method of any of claims 30-41, further comprising collecting optical information from the container via one or more embedded optical sensors at the interior of the container.Docket No.6063.002WO1 43. The method of claim 42, further comprising synchronizing collection of the container motion or position information or calibrating the container motion or position information collected by the embedded accelerometer using the collected optical information.

44. The method of any of claims 30-43, further comprising: collecting, via a plurality of spatially distributed imaging sensors, ergonomic information from an operator agitating the contained pharmaceutical product; estimating pose or tracking motion of the operator using the collected ergonomic information; and generating the agitation protocol further based on the estimated pose or the tracked motion of the operator.

45. The method of any of claims 30-44, further comprising: detecting an anomaly associated with the container or the contained pharmaceutical product using the collected sensor data including during the agitation of the contained pharmaceutical product; and generating quality control diagnostics based at least on the detected anomaly.

46. The method of any of claims 30-45, further comprising: sensing a light intensity, in a specific wavelength or wavelength range, irradiated on the container via one or more embedded optical sensors at the interior of the container; and determining light exposure of the pharmaceutical product based on the sensed light intensity.

47. The method of any of claims 30-46, further comprising: collecting acoustic data via an embedded acoustic sensor at the interior of the container in response to manipulation of the container; and detecting a glass breakage event in a glass portion of the container using the collected acoustic data.Docket No.6063.002WO1 48. The method of claim 47, further comprising synchronizing the collection of the container motion or position information using the collected acoustic data.

49. The method of any of claims 30-48, further comprising: collecting vibration data via the embedded accelerometer in response to diagnostic striking of the container; and detecting a glass breakage event in a glass portion of the container using the collected vibration data.

50. A system for characterizing inspection and quality control processes of pharmaceutical product manufacturing and packaging in containers, the system comprising: sensor circuitry, comprising an embedded sensor platform that includes an embedded accelerometer in an interior of a container empty of pharmaceutical product, the embedded accelerometer configured to collect container motion or position information during agitation of the container; and a controller circuit configured to detect an anomaly of a robotic system for agitation and inspection of containers based at least in part on the collected container motion or position information.

51. The system of claim 50, wherein the controller circuit is configured to generate quality control diagnostics based at least in part on the detected anomaly.

52. The system of any of claims 50-51, wherein the controller circuit is configured to: calculate one or more agitation parameters using the collected container motion or position information; and detect the anomaly of the robotic system using information using the calculated one or more agitation parameters.

53. The system of claim 52, wherein the one or more agitation parameters include at least one of a position, an orientation, a velocity, an acceleration, a displacement, or a range of movement of the container.Docket No.6063.002WO1 54. The system of claim any of claim 52-53, wherein the controller circuit is configured to: make predictions of fluid dynamics characteristics of the pharmaceutical product by applying the collected container motion or position information to a computational fluid dynamics (CFD) model; and calculate the one or more agitation parameters using the predictions of fluid dynamics characteristics.

55. The system of claim any of claims 52-54, wherein the controller circuit is configured to pre-process the collected container motion or position information including removing or attenuating a temporal drift, and to calculate the one or more agitation parameters using the pre-processed container motion or position information.

56. The system of any of claims 50-55, wherein the embedded sensor platform is sized and shaped to be removably inserted into and substantially conform to a cylindrical portion of the interior of the container.

57. The system of claim 56, wherein the embedded accelerometer is configured to be aligned to a longitudinal axis of the cylindrical portion of the interior of the container.

58. The system of any of claims 50-57, wherein the embedded accelerometer is configured to be mounted on a base unit of the embedded sensor platform.

59. The system of claim 58, wherein the embedded accelerometer is configured to be positioned at substantially a center of mass of the embedded system.Docket No.6063.002WO1 60. The system of any of claims 50-59, wherein the sensor circuitry includes an embedded sensor in the interior of the empty container, the embedded sensor selected from an optical sensor, an acoustic sensor, a vibration sensor, or a combination thereof and configured to collect information during agitation of the empty container, wherein the controller circuit is configured to detect an anomaly of a second container filled with pharmaceutical product in agitation and inspection of the second container based at least in part on the collected information from the embedded sensor.

61. The system of any of claims 50-60, wherein the sensor circuitry includes one or more embedded optical sensors at the interior of the empty container, the one or more embedded optical sensors configured to sense a light intensity, in a specific wavelength or wavelength range, irradiated on the empty container, wherein the controller circuit is configured to synchronize collection of the container motion or position information or calibrate the container motion or position information using the collected optical information.

62. The system of any of claims 50-61, wherein the sensor circuitry includes one or more optical sensors at the interior of the empty container and configured to sense a light intensity, in a specific wavelength or wavelength range, irradiated on a second container filled with pharmaceutical product, wherein the controller circuit is configured to detect an anomaly of the second container or the contained pharmaceutical product based at least in part on the sensed light intensity.

63. The system of any of claims 50-62, wherein the sensor circuitry includes an acoustic sensor at the interior of the empty container and configured to collect acoustic data in response to manipulation of a second container filled with pharmaceutical product, wherein the controller circuit is configured to detect a glass breakage event in a glass portion of the second container using the collected acoustic data.Docket No.6063.002WO1 64. The system of claim any of claims 50-63, wherein the embedded accelerometer is further configured to collect vibration data in response to manipulation of a second container filled with pharmaceutical product, wherein the controller circuit is configured to detect a glass breakage event in a glass portion of the second container using the collected vibration data.

65. A method of characterizing inspection and quality control of processes of pharmaceutical product manufacturing and packaging in containers, the method comprising: introducing agitation to a container empty of pharmaceutical product, the container associated with an embedded sensor platform including an embedded accelerometer in an interior of the container; collecting sensor data including container motion or position information sensed by the embedded accelerometer during the agitation of the container; and based at least in part on the collected sensor data, detecting an anomaly of a robotic system for agitation and inspection of containers based at least in part on the collected container motion or position information.

66. The method of claim 65, further comprising generating quality control diagnostics based at least in part on the detected anomaly of the robotic system.

67. The method of any of claims 65-66, further comprising calculating one or more agitation parameters using the collected container motion or position information, wherein detecting the anomaly includes using calculated one or more agitation parameters.

68. The method of claim 67, wherein the one or more agitation parameters include at least one of a position, an orientation, a velocity, an acceleration, a displacement, or a range of movement of the container.Docket No.6063.002WO1 69. The method of any of claims 67-68, further comprising: making predictions of fluid dynamics characteristics of the pharmaceutical product by applying the collected container motion or position information to a computational fluid dynamics (CFD) model; and calculating the one or more agitation parameters using the predictions of fluid dynamics characteristics.

70. The method of any of claims 67-69, further comprising: pre-processing the collected container motion or position information including removing or attenuating a temporal drift; and calculating the one or more agitation parameters using the pre-processed container motion or position information.

71. The method of any of claims 65-70, wherein the embedded sensor platform is inserted into and substantially conforms to a cylindrical portion of the interior of the container.

72. The method of claim 71, wherein the embedded sensor platform is such positioned that the embedded accelerometer is aligned to a longitudinal axis of the cylindrical portion of the interior of the container.

73. The method of any of claims 65-72, wherein the embedded sensor platform is secured on a base unit of the embedded sensor platform.

74. The method of claim 73, wherein the embedded sensor platform positioned at substantially a center of mass of the embedded system.

75. The method of any of claims 65-74, further comprising: collecting information during the agitation of the empty container using an embedded sensor in the interior of the empty container, the embedded sensor selected from an optical sensor, an acoustic sensor, a vibration sensor, or a combination thereof; and detecting an anomaly of a second container filled with pharmaceutical product in agitation and inspection of the second container based at least in partDocket No.6063.002WO1 on the collected information from the embedded sensor during the agitation of the empty container.

76. The method of any of claims 65-75, further comprising: sensing a light intensity, in a specific wavelength or wavelength range, irradiated on the empty container using one or more embedded optical sensors at the interior of the empty container; and synchronizing collection of the container motion or position information or calibrate the container motion or position information using the sensed light intensity.

77. The method of any of claims 65-76, further comprising: sensing a light intensity, in a specific wavelength or wavelength range, irradiated on a second container filled with pharmaceutical product using one or more optical sensors at the interior of the empty container; and detecting an anomaly of the second container or the contained pharmaceutical product based at least in part on the sensed light intensity.

78. The method of any of claims 65-77, further comprising: collecting acoustic data in response to manipulation of a second container filled with pharmaceutical product using an acoustic sensor at the interior of the empty container; and detecting a glass breakage event in a glass portion of the second container using the collected acoustic data.

79. The method of claim any of claims 65-78, further comprising: collecting vibration data in response to manipulation of the empty container using the embedded accelerometer; and detecting a glass breakage event in a glass portion of a second container filled with pharmaceutical product using the collected vibration data.

80. A system for characterizing inspection and quality control processes of pharmaceutical product manufacturing and packaging in containers, the system comprising:Docket No.6063.002WO1 sensor circuitry, comprising an embedded sensor platform that includes an embedded sensor in an interior of a first container empty of pharmaceutical product, the embedded sensor selected from an optical sensor, an acoustic sensor, a vibration sensor, or a combination thereof and configured to collect information during agitation of the first container; and a controller circuit configured to detect an anomaly of a second container filled with pharmaceutical product in agitation and inspection of the second container based at least in part on the collected information.

81. The system of claim 80, wherein the controller circuit is configured to generate quality control diagnostics based at least in part on the detected anomaly.

82. The system of any of claims 80-81, wherein the sensor circuitry includes one or more optical sensors configured to sense a light intensity, in a specific wavelength or wavelength range, irradiated on the second container, wherein the controller circuit is configured to detect the anomaly of the second container or the contained pharmaceutical product based at least in part on the sensed light intensity.

83. The system of claim 82, wherein the controller circuit is configured to detect light exposure of the pharmaceutical products contained in the containers based on the sensed light intensity.

84. The system of any of claims 80-83, wherein the sensor circuitry includes an acoustic sensor configured to collect acoustic data in response to manipulation of the second container, wherein the controller circuit is configured to detect a glass breakage event in a glass portion of the second container using the collected acoustic data.Docket No.6063.002WO1 85. The system of any of claims 80-84, wherein the sensor circuitry includes an embedded accelerometer configured to collect vibration data in response to manipulation of the second container, wherein the controller circuit is configured to detect a glass breakage event in a glass portion of the second container using the collected vibration data.

86. The system of any of claims 80-85, wherein the sensor circuitry includes an embedded accelerometer in an interior of the first container, the embedded accelerometer configured to collect container motion or position information during agitation of the first container, wherein the controller circuit is configured to detect an anomaly of a robotic system for agitation and inspection of containers based at least in part on the collected container motion or position information.

87. The system of claim 86, wherein the controller circuit is configured to: calculate one or more agitation parameters using the collected container motion or position information; and detect the anomaly of the second container or the contained pharmaceutical product using information acquired during the agitation of the second container in accordance with the calculated one or more agitation parameters.

88. The system of claim 87, wherein the one or more agitation parameters include at least one of a position, an orientation, a velocity, a displacement, or a range of movement of the first container.

89. The system of any of claims 87-88, wherein the controller circuit is configured to: make predictions of fluid dynamics characteristics of the pharmaceutical product by applying the collected container motion or position information to a computational fluid dynamics (CFD) model; andDocket No.6063.002WO1 calculate the one or more agitation parameters using the predictions of fluid dynamics characteristics.

90. The system of any of claims 87-89, wherein the controller circuit is configured to pre-process the collected container motion or position information including removing or attenuating a temporal drift, and to calculate the one or more agitation parameters using the pre-processed container motion or position information.

91. The system of any of claims 87-90, wherein the controller circuit is configured to program a robotic system with the calculated one or more agitation parameters, and to detect the anomaly of the second container or the contained pharmaceutical product using information acquired during a robotic agitation of the second container by the robotic system.

92. The system of any of claims 86-91, wherein the sensor circuitry includes one or more embedded optical sensors at the interior of the first container, the one or more embedded optical sensors configured to sense a light intensity, in a specific wavelength or wavelength range, irradiated on the first container, wherein the controller circuit is configured to synchronize collection of the container motion or position information or calibrate the container motion or position information using the collected optical information.

93. The system of any of claims 86-92, wherein the controller circuit is configured to: collect ergonomic information sensed by a plurality of spatially distributed imaging sensors from an operator agitating the first container; estimate pose or tracking motion of the operator using the collected ergonomic information; and detect the anomaly of the second container or the contained pharmaceutical product using information acquired during the agitation of the second container based on the estimated pose or the tracked motion of the operator.Docket No.6063.002WO1 94. A method of characterizing inspection and quality control of processes of pharmaceutical product manufacturing and packaging in containers, the method comprising: introducing agitation to a first container empty of pharmaceutical product, the first container associated with an embedded sensor platform including an embedded sensor in an interior of the first container, the embedded sensor selected from an optical sensor, an acoustic sensor, a vibration sensor, or a combination thereof; collecting sensor data by the embedded sensor during the agitation of the first container; and based at least in part on the collected sensor data, detecting an anomaly of a second container filled with pharmaceutical product in agitation and inspection of the second container based at least in part on the collected sensor data.

95. The method of claim 94, further comprising generating quality control diagnostics based at least in part on the detected anomaly of the second container or the contained pharmaceutical product.

96. The method of any one of claims 94-95, further comprising: collecting container motion or position information during agitation of the first container using an embedded accelerometer in an interior of the first container; and based at least in part on the collected sensor data, detecting an anomaly of a robotic system for agitation and inspection of containers based at least in part on the collected container motion or position information.

97. The method of any of claims 94-96, further comprising calculating one or more agitation parameters using the collected container motion or position information, wherein detecting the anomaly includes using information acquired during the agitation of the second container in accordance with the calculated one or more agitation parameters.Docket No.6063.002WO1 98. The method of claim 97, wherein the one or more agitation parameters include at least one of a position, an orientation, a velocity, an acceleration, a displacement, or a range of movement of the first container.

99. The method of any of claims 97-98, further comprising: making predictions of fluid dynamics characteristics of the pharmaceutical product by applying the collected container motion or position information to a computational fluid dynamics (CFD) model; and calculating the one or more agitation parameters using the predictions of fluid dynamics characteristics.

100. The method of any of claims 97-99, further comprising: pre-processing the collected container motion or position information including removing or attenuating a temporal drift; and calculating the one or more agitation parameters using the pre-processed container motion or position information.

101. The method of any of claims 94-100, further comprising: collecting optical information from the first container via one or more embedded optical sensors at the interior of the first container; and synchronizing collection of the first container motion or position information or calibrating the first container motion or position information using the collected optical information.

102. The method of any of claims 94-101, further comprising: collecting ergonomic information sensed by a plurality of spatially distributed imaging sensors from an operator agitating the first container; and estimating pose or tracking motion of the operator using the collected ergonomic information, wherein detecting the anomaly includes using information acquired during the agitation of the second container based on the estimated pose or the tracked motion of the operator.Docket No.6063.002WO1 103. The method of any of claims 94-102, wherein the collected sensor data include light intensity data, in a specific wavelength or wavelength range, irradiated on the second container sensed by one or more optical sensors, wherein detecting the anomaly includes determining light exposure of the pharmaceutical product based on the sensed light intensity.

104. The method of any of claims 94-103, wherein the collected sensor data include acoustic data sensed by an acoustic sensor in response to manipulation of the second container, wherein detecting the anomaly includes detecting a glass breakage event in a glass portion of the second container using the collected acoustic data.

105. The method of any of claims 94-104, wherein the collected sensor data include vibration data collected by the embedded accelerometer in response to diagnostic striking of the second container, wherein detecting the anomaly includes detecting a glass breakage event in a glass portion of the second container using the collected vibration data.

Images