Convolutional neural network soft-sensor controller for microfluidics

Abstract

Methods and systems for convolutional neural network-based, soft-sensor controller for microfluidics are described herein. In one aspect, a method can include: receiving image data of an output of a droplet generator; inputting the image data into a trained image classification model; determining, based on the image data and via the image classification model, a flow regime of the droplet generator, wherein the flow regime comprises one of: (a) a dripping regime, (b) a non-production regime, and (c) a jetting regime; and adjusting a parameter of the droplet generator when the flow regime comprises either (b) or (c).

Description

CONVOLUTIONAL NEURAL NETWORK SOFT-SENSOR CONTROLLER FOR MICROFLUIDICSCROSS-REFERENEC TO RELATED APPLICATIONS

[0001] This application claims priority to U.S. Patent Application No. 63 / 635,225 (titled “Convolutional Neural Network Soft-Sensor Controller for Microfluidics”) filed April 17, 2024, the contents of which are hereby incorporated by reference for any and all purposes.TECHNICAL FIELD

[0002] The present disclosure relates to the field of process control and to the field of microfluidics.BACKGROUND

[0003] Microfluidics enables the production and manipulation of multi-phasic mixtures such as gas bubbles, liquid droplets and multiple emulsions with unparalleled precision and control. Leveraging advanced techniques in micro / nano-fabrication, precise microchannels can be manufactured to control droplets and bubbles for a wide range of advanced applications.

[0004] For example, chemical reactions can be induced in single droplets, providing a unique platform to conduct high-throughput analyses and synthesis with minimal reagents use and reduced waste. Furthermore, droplet microfluidics facilitates the encapsulation of delicate biological materials such as cells and proteins under mild conditions that preserve their functionality and viability, which is particularly well-suited for the development of low-cost and highly efficient biomedical diagnostics and therapeutics. The precision and scalability of droplet microfluidics enables fabrication of functional particles such as microbubbles, microcapsules and nanoparticles with precisely designed morphology and functionality, enhancing disease diagnostics as well as controlled release and targeted delivery of various pharmaceutical actives.

[0005] Achieving consistent uniformity in the production of droplets and bubbles throughout the operation of microfluidic devices is crucial for harnessing the full benefits of this technology. The maintenance of uniformity is, however, challenged by several factors, which necessitates continuous user intervention to adjust flow rates and pressures,ensuring that the droplet and bubble production maintains the desired dimensions and properties. Even seemingly negligible changes in operating conditions can cause large fluctuations in performance of device output because of the sensitivity of flow behaviors of fluids at the microscopic scale. Over time, the performance of microfluidic devices may be further compromised by issues such as surface fouling, changes in wetting properties, channel clogging and the solvent-induced swelling of the microfluidic devices. These factors introduce additional layers of complexity in achieving and maintaining the uniformity of droplet and bubble generation, posing significant challenges to the scalability and reliability of microfluidic applications.

[0006] Developing autonomous microfluidic systems capable of self-adapting to changing conditions would enable the precise formation of a wide array of droplets and bubbles with complex composition and morphology without direct operator intervention. Such a capability enhances the efficiency and effectiveness of droplet and bubble microfluidics and leads to new applications that use the full potential of this versatile technology.

[0007] A few recent studies have demonstrated the ability to control droplet microfluidics, using technologies such as neural networks and reinforcement learning to gather insight on flow regimes in microfluidics over various flow conditions. Other techniques involve the use of impedance electronics embedded into the microfluidic device as a sensor for measuring microbubble diameter as a function of the voltage measured or measuring the interference pattern created by focusing a laser on droplets in the outlet channel using piezoelectric transducer. In addition, feedback sensors have been developed for precise control of flowrates and pressures in microfluidic devices off-chip. Despite these advances, many of these approaches use highly sophisticated sensing techniques that involve specialized equipment outside what is traditionally used in microfluidics, do not have the ability to measure process variables on-chip, and do not have the ability to control the system when it is disturbed into undesirable flow regimes, such as one in which bubbles or droplets are not produced. Accordingly, there is a long- felt need in the field for improved techniques for controlling microfluidic production.SUMMARY

[0008] Methods and systems for convolutional neural network-based, soft-sensor controller for microfluidics are described herein. In one aspect, a method can include: receiving image data of an output of a droplet generator; inputting the image data into a trained image classification model; determining, based on the image data and via the image classification model, a flow regime of the droplet generator, wherein the flow regime comprises one of: (a) a dripping regime, (b) a non-production regime, and (c) a jetting regime; and adjusting a parameter of the droplet generator when the flow regime comprises either (b) or (c).BRIEF DESCRIPTION OF THE DRAWINGS

[0009] In the drawings, which are not necessarily drawn to scale, like numerals may describe similar components in different views. Like numerals having different letter suffixes may represent different instances of similar components. The drawings illustrate generally, by way of example, but not by way of limitation, various aspects discussed in the present document. In the drawings:

[0010] FIG. 1. Control Schematic using CNN & image recognition for PID control of microfluidic bubble generation.

[0011] Fig 2. Representative training three flow regimes for CNN: (a) A liquid- dominated flow regime without bubble generation, (b) bubble generation in the dripping regime, and (c) air-dominated jetting regime.

[0012] FIG. 3. Process Reaction Curve for a first order plus dead time (FOPDT) response in the process variable as a response to a step change in the manipulated variable used for Ziegler-Nichols Open Loop Tuning.

[0013] FIG. 4. Diameter response (a) from open-loop step change in pressure from 41.4 kPa to 46.7 kPa (b) using Ziegler-Nichols tuning.

[0014] FIG. 5. Diameter response (a) from open-loop step change in flowrate from 1500 pL / hr to 1300 pL / hr (b) using Ziegler-Nichols tuning.

[0015] FIG. 6. Setpoint tracking diameter responses (a) while varying pressure (b) while at constant aqueous flowrate for increases in size setpoint.

[0016] FIG. 7. Setpoint tracking diameter responses (a) while varying aqueous flowrate (b) while at constant air pressure for decreases in size setpoint.

[0017] FIG. 8. Setpoint tracking diameter responses (a) while varying aqueous flowrate (b) while at constant air pressure for increases in size setpoint.

[0018] FIG. 9. Setpoint tracking diameter responses (a) while varying aqueous flowrate (b) while at constant air pressure for decreases in size setpoint.

[0019] FIG. 10. Time series of response to sharp disturbance knocking flow out of a bubble producing flow regime, (a) System that has been disturbed and is in a liquid dominated flow regime and no longer producing bubbles, (b) Moment breakthrough pressure is reached by CNN allowing bubbles to be produced, (c) Controller continued to reduce the pressure from the breakthrough pressure to reach the setpoint size.

[0020] FIG. 11. Diameter response (a) to maintain the setpoint value by varying the air pressure (b) to overcome a flowrate disturbance from 2000 pL / hr to 1800 pL / hr (c).

[0021] FIG. 12. Diameter response (a) to maintain the setpoint value by varying the aqueous flowrate (c) to overcome an air pressure disturbance from 35 kPa to 34 kPa (c).

[0022] FIG. 13. Long-term diameter response (a) of an open-loop control system with constant air pressure (b) at constant aqueous flowrate.

[0023] FIG. 14. Long-term diameter response (a) of a closed-loop control system with varying air pressure (b) at constant aqueous flowrate.

[0024] FIG. 15. Training and validation accuracy of CNN.

[0025] FIG 16: Control schematic using CNN & image recognition for PID control of microfluidic bubble generation.

[0026] FIG. 17: Ziegler-Nichols tuning for a diameter response (a) from open-loop step change in pressure from 18.7 kPa to 20.3 kPa (b) and for a diameter response (c) from open-loop step change in flowrate created by a step change in liquid pressure from 31.7 kPa to 29.6 kPa (d).

[0027] FIG. 18: Setpoint tracking diameter responses (a) while varying pressure (b) while at constant aqueous flowrate for increases in diameter setpoint. Setpoint tracking diameter responses (c) while varying pressure (d) while at constant aqueous flowrate for decreases in diameter setpoint.

[0028] FIG. 19: Setpoint tracking diameter responses (a) while varying liquid driving pressure (b) while at constant air pressure for decreases in diameter setpoint. Setpoint tracking diameter responses (c) while varying liquid driving pressure (d) while at constant air pressure for increases in diameter setpoint.

[0029] FIG. 20: Diameter responses (a) while varying air pressure (b) to overcome liquid-dominated flow by reaching the breakthrough pressure of 41.9 kPa and tapering down to 35.7 kPa to achieve a setpoint of 70 pm of large disturbance knocking flow out of a bubble producing flow regime. Diameter values of zero indicate no bubbles production, (c) System that has been disturbed and is in a liquid dominated flow regime - no longer producing bubbles, (d) Moment breakthrough pressure is reached by CNN allowing bubbles to be produced, (e) Controller continued to reduce the pressure from the breakthrough pressure to reach the setpoint diameter.

[0030] FIG. 21 : (a) Diameter response to maintain the setpoint value by varying the air pressure (b) to overcome a flowrate disturbance caused by a change in liquid driving pressure (c). (d) Diameter response to maintain the setpoint value by varying the liquid driving pressure (e) to overcome an air pressure disturbance (f).

[0031] FIG. 22: (a) Long-term diameter response of an open-loop control system with (b) constant air pressure at constant aqueous flowrate, (c) Long-term diameter response of a closed-loop control system with (d) varying air pressure at constant aqueous flowrate.

[0032] FIG. 23 : Example of underdamped pressure control parameters leading to oscillations in microfluidic control system using pressure control of dispersed nitrogen phase.

[0033] FIG. 24: Example of underdamped flowrate control parameters leading to oscillations in microfluidic control system using pressure driven flow of continuous liquid phase.

[0034] FIG. 25: One hour diameter control at 55 pm with vertical green line indicating time points where the CNN was active in recovery to bubble production. The CNN controlled the output pressure 0.74% of the time while PID control worked the remaining 99.26% of the time.

[0035] FIG. 26: Setpoint tracking diameter-increase response (a) while varying liquid driving pressure (b) while at constant air pressure for an increase in diameter setpoint for gas-in-oil bubbles.

[0036] FIG. 27: (a) gas-in-water and (b) gas-in-oil bubbles.DETAILED DESCRIPTION OF ILLUSTRATIVE EMBODIMENTS

[0037] The present disclosure may be understood more readily by reference to the following detailed description of desired embodiments and the examples included therein.

[0038] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art. In case of conflict, the present document, including definitions, will control. Preferred methods and materials are described below, although methods and materials similar or equivalent to those described herein can be used in practice or testing. All publications, patent applications, patents and other references mentioned herein are incorporated by reference in their entirety. The materials, methods, and examples disclosed herein are illustrative only and not intended to be limiting.

[0039] The singular forms “a,” “an,” and “the” include plural referents unless the context clearly dictates otherwise.

[0040] As used in the specification and in the claims, the term "comprising" can include the embodiments "consisting of' and "consisting essentially of.” The terms “comprise(s),” “include(s),” “having,” “has,” “can,” “contain(s),” and variants thereof, as used herein, are intended to be open-ended transitional phrases, terms, or words that require the presence of the named ingredients / steps and permit the presence of other ingredients / steps. However, such description should be construed as also describing compositions or processes as "consisting of' and "consisting essentially of' the enumerated ingredients / steps, which allows the presence of only the named ingredients / steps, along with any impurities that might result therefrom, and excludes other ingredients / steps.

[0041] As used herein, the terms “about” and “at or about” mean that the amount or value in question can be the value designated some other value approximately or about the same. It is generally understood, as used herein, that it is the nominal value indicated ±10% variation unless otherwise indicated or inferred. The term is intended to convey that similar values promote equivalent results or effects recited in the claims. That is, it is understood that amounts, sizes, formulations, parameters, and other quantities and characteristics are not and need not be exact, but can be approximate and / or larger or smaller, as desired, reflecting tolerances, conversion factors, rounding off, measurement error and the like, and other factors known to those of skill in the art. In general, an amount, size, formulation, parameter or other quantity or characteristic is “about” or“approximate” whether or not expressly stated to be such. It is understood that where “about” is used before a quantitative value, the parameter also includes the specific quantitative value itself, unless specifically stated otherwise.

[0042] Unless indicated to the contrary, the numerical values should be understood to include numerical values which are the same when reduced to the same number of significant FIGs. and numerical values which differ from the stated value by less than the experimental error of conventional measurement technique of the type described in the present application to determine the value.

[0043] All ranges disclosed herein are inclusive of the recited endpoint and independently of the endpoints. The endpoints of the ranges and any values disclosed herein are not limited to the precise range or value; they are sufficiently imprecise to include values approximating these ranges and / or values.

[0044] As used herein, approximating language can be applied to modify any quantitative representation that can vary without resulting in a change in the basic function to which it is related. Accordingly, a value modified by a term or terms, such as “about” and “substantially,” may not be limited to the precise value specified, in some cases. In at least some instances, the approximating language can correspond to the precision of an instrument for measuring the value. The modifier “about” should also be considered as disclosing the range defined by the absolute values of the two endpoints. For example, the expression “from about 2 to about 4” also discloses the range “from 2 to 4.” The term “about” can refer to plus or minus 10% of the indicated number. For example, “about 10%” can indicate a range of 9% to 11%, and “about 1” can mean from 0.9-1.1. Other meanings of “about” can be apparent from the context, such as rounding off, so, for example “about 1” can also mean from 0.5 to 1.4.

[0045] Further, the term “comprising” should be understood as having its open-ended meaning of “including,” but the term also includes the closed meaning of the term “consisting.” For example, a composition that comprises components A and B can be a composition that includes A, B, and other components, but can also be a composition made of A and B only. Any documents cited herein are incorporated by reference in their entireties for any and all purposes.

[0046] Any embodiment or aspect provided herein is illustrative only and does not limit the scope of the present disclosure or the appended claims. Any part or parts of any one ormore embodiments or aspects can be combined with any part or parts of any one or more other embodiments or aspects.

[0047] Microfluidics has emerged as a foundational process for creating highly uniform emulsions and bubbles. To enable integration of microfluidic platforms into industrial processes, achieving precise control over the size uniformity of microfluidic-generated bubbles and emulsions is crucial. Even if the external variables as flow rates or pressures are kept constant, microfluidic processes can be easily disturbed by unknown factors that would substantially compromise the uniformity of resulting emulsions and bubbles. In this disclosure, we provide a soft-sensor approach that combines a convolutional neural network (CNN) and an image recognition algorithm for feature extraction to detect both the flow regime and the size and uniformity of resulting bubbles. Although the disclosed technology is illustrated by reference to formation of air-in-liquid bubbles, it should be understood that the disclosed technology is not limited to bubbles and can used in virtually any microfluidics context, including, for example, emulsion production.

[0048] By using the CNN to detect flow regime, the disclosed approach can restore the bubble-producing flow regime in response to disturbances. Beyond self-recovery, our controller actively adjusts to minimize errors, maintain setpoints, mitigate disturbances, and ensure system stability over extended periods. 99.2% of bubbles produced during an 8-hour period remain within 5% of the setpoint with our controller taking action. By leveraging the soft sensor and artificial intelligence-assisted feedback control, our work presents a widely applicable approach for precise and automated control of microfluidics in diverse applications.

[0049] In this disclosure, we provide an autonomous microfluidics system that relies simply on microscopic imaging and is trained with a convolutional neural network (CNN) to return the process to the desired microbubble production flow regime. We produce gas bubbles in a flow focusing generator by applying a constant pressure for the gas phase and a constant flowrate for the aqueous phase using a syringe pump. Gas bubbles are selected over liquid droplets due to their greater size variability for various reasons including the compressibility of the gas phase, large interfacial tension, and significantly different viscosity of the two phases, necessitating enhanced control; however, the system can easily be adapted for droplets by switching out the pressure controller for another syringe pump for the dispersed phase. Our control system actively adjusts either the aqueous flowrate or gas pressure to ensure the bubbles being produced match the user-determined setpoint for the bubble size. As an illustrative example, without feedback control, only 2.1% of bubbles fell within 5% of the initial diameter value over an eight-hour period. Our autonomous control system achieved 99.2% setpoint accuracy within the same timeframe while also showing effective setpoint tracking and disturbance rejection. The disclosed approach can be used to control various geometries of microfluidic output, such as rods and discs, as but some examples.Methodolog

[0050] For illustration purposes, we use a well-established flow-focusing generator to produce gas bubbles. This geometry splits the continuous phase into two streams which subsequently surround and pinch off the dispersed phase at a cross junction, as shown in FIG. 1. The symmetry of the junction allows for more flexibility in the size and frequency of bubble generation. The immiscibility of the two phases forces bubbles to form through either a dripping or jetting mechanism. The dripping regime involves the periodic breakup of a fluid stream into bubbles due to capillary instability. This instability arises from the interplay of surface tension and viscous forces. The dripping frequency is governed by the capillary number (Ca), representing the ratio of viscous to capillary forces. The jetting regime involves the stretching of a fluid stream into an extended jet due to the dominance of inertial forces or viscous forces over capillary forces. In this regime, a more complex breakup behavior, governed by Rayleigh-Plateau instability, takes place resulting in polydisperse bubbles. Controlling the flows of the two fluid phases to maintain microfluidic generation in the dripping regime is critical to maintaining the uniformity of resulting droplets and bubbles.In this study, we use a flow-focusing geometry to produce nitrogen gas (dispersed phase) bubbles in an aqueous phase of 0.5 wt% sodium dodecyl sulfate (SDS) dissolved in DI water (continuous phase). The dispersed phase is injected into the device as pressurized nitrogen gas controlled by a differential pressure controller and the continuous phase is injected with a syringe pump as depicted in FIG. 1. All microfluidic devices in this study have undergone hydrophilic-surface treatment to ensure stable formation of gas bubbles using a 2 wt% polyvinyl alcohol (PVA) solution. Proportional-integral-derivative (PIP) control

[0051] PID control is a fundamental type of control and is widely used in industrial applications due to its simplicity and effectiveness. It has been widely studied, can address a wide range of process behaviors, and is straightforward to implement, making it a natural choice for control of microfluidics. PID feedback control operates by continuously comparing the desired setpoint to the process output, giving the error, e. The proportional term responds to the current error, the integral term accumulates past errors to eliminate steady-state discrepancies (offset), and the derivative term accounts for the rate of change of the error, as shown in Equation 1 u(t) = Kce(t + e(t)dt + KCTD+ cswhereKC is the proportional gain, TD is the integral time constant in minutes, TD is the derivative time constant in minutes, and csis the controller bias (actuating signal when e=0). These terms are combined to compute the controller output (»( / )), which adjusts the controller’s input, aiming to reduce the error and maintain stable and precise control of the process. In this study, two separate control schemes are explored: manipulating the pressure of the dispersed phase to control microbubble size and manipulating the flowrate of the continuous phase to control microbubble size. FIG. 1 depicts the control scheme employed when the pressure of the dispersed phase is manipulated for control of output bubble size, or when the continuous phase flowrate is manipulated to control bubble size.

[0052] Altering the PID tuning parameters Kc, TI, and TD can influence the response of the controller; increasing the proportional gain enhances responsiveness, but may lead to overshooting, while adjusting the integral and derivative time-constants affects the elimination of steady-state error and reduces the settling time, respectively. This method relies on measuring the process output in real-time. Numerous process parameters, including pressure and temperature, can be measured directly using commercially available real-time in-line sensors. However, microfluidic processes currently lack reliable, industry-proven inline sensor technologies that can directly measure the important process variables that need to be controlled such as size, shape, and flow regime. Thus, we use an artificial intelligence (Al)-based approach to create an indirect sensor, known as a soft-sensor, when controlling microfluidics.Soft sensors

[0053] Control systems rely on sensors to measure the movement of a process variable from a user-specified set point. When sensor variables are difficult to measure, soft- sensors can be used to estimate them. There are three types of soft-sensor models:knowledge-based models that rely on first principles, and black-box models that are data- driven, and hybrid models that combine the two. In this study, a black-box model consisting of a two-step process is used to estimate the flow regime and size of the process output, gas bubbles. The first step in our process is using a convolutional neural network (CNN) for image classification. In this study, a linear architecture CNN can be used to classify the microfluidic output into one of three regimes: liquid-dominated flow, airdominated flow, and microbubble flow, as shown in FIG. 2. The second step in our process is using a Hough image recognition algorithm for detection of the microbubbles. This algorithm is designed for feature extraction and gives an output of their location and size.

[0054] Our novel soft-sensor operates as follows: first, a CNN trained for image classification determines the current flow regime of the process from a snapshot of the device taken by a high-speed microscope camera. When the process is in a bubble producing dripping regime, an image recognition algorithm measures the size of the bubbles being produced. The combination of the CNN and image recognition is the soft- sensor output on which the controller acts to match the bubble size to the user-specified setpoint. When the process is not in a bubble producing flow regime, a direct controller action is taken to return the process to the production of gas bubbles. Our approach is superior to other microfluidic control techniques because of its ability to recover into the bubble-producing dripping regime when a disturbance causes the system to move into a flow regime in which bubbles are not being produced.Results and Discussion

[0055] Ziegler-Nichols tuning

[0056] Tuning the PID control parameters can achieve reliable controller responses. In this study, we use Ziegler-Nichols, open-loop response tuning rules, before making minor adjustments to achieve desired responses. Ziegler-Nichols tuning rules are suitable for systems characterized by first-order responses with a lag time, a feature observed in our system. This is achieved by turning off the controller and monitoring the response to a step change in the manipulated variable. As shown in the process reaction curve to a step change in FIG. 3, a tangent line through the inflection point is drawn to estimate the delay time (rd) and response time (r). From this response, u is the step change in the process input, such as gas pressure, and y is the change in the measured process variable, which isthe bubble size herein. These variables are used to calculate controller parameters, whereKcT; = 2.0rd, and TD= 0.5rdSystematic changes in tuning parametersare then made to achieve the desired slightly underdamped response from our controller.

[0057] A step pressure increase from 41.4 to 46.7 kPa causes an increase in diameter as shown in FIG. 4. Plotting the tangent line to the curve for the change in diameter, we can use Ziegler-Nichols tuning rules to obtain tuning parameters. The delay time is 1 second and the response time is 27 seconds, leading to a controller gain of 6.48 kPa / wm. integral time constant is 2 seconds, and derivative time constant is 0.5 seconds.

[0058] Instead of the gas pressure, it is also possible to control the flowrate of the continuous phase. For a flowrate step change from 1500 pL / hr to 1300 pL / hr the openloop tuning controller gain is -349.6 pL / hr, integral time constant is 6 seconds, and derivative time constant is 1.5 seconds calculated from FIG. 5. This response to flowrate change shows a more gradual change in diameter and occurs slower than responses for air pressure step changes.Setpoint tracking

[0059] In process control, achieving precise and dynamic control of process variables is essential. Setpoint tracking by PID control holds significant importance in optimizing process performance, ensuring close adherence to desired operating conditions and allowing for switches to new operating setpoints.

[0060] For many microfluidic processes, effective changes in bubble / droplet sizes are required for different applications. For example, the size of gas bubbles is crucial in determining their resonance frequency, particularly in applications where bubbles serve as a contrast agent in ultrasound sonography. In the context of microfluidic reactors, the size of droplets plays a crucial role in influencing the reaction rates and kinetics of associated chemical processes, thus controlling the size is paramount.

[0061] In PID control, the proportional term responds to the current error, the integral term addresses accumulated past errors, and the derivative term anticipates future errors. This combination allows the PID controller to regulate the process, reducing deviations from the setpoint, enhancing overall system stability and performance.

[0062] With our controller, we can dynamically manipulate either the continuous phase (aqueous flowrate), or the dispersed phase (air pressure) to reach a desired setpoint, output bubble size.

[0063] FIGs. 6-9 show the experimental controller actions after a diameter setpoint changes at time zero. Each color represents a different setpoint change trial: dashed lines represent the setpoint change, and the solid line is the process variable changing with time. The aqueous flowrate is kept constant, and the pressure is manipulated by the controller to reach the new setpoint in FIGs. 6 and 7 where the first shows the response for an increase in diameter setpoint and the latter represents a decrease in diameter setpoint. Both responses are adjusted to be slightly overdamped; that is, the system does not overshoot the new setpoint value before eventually settling down to the desired value.

[0064] Responses in all cases settle to the new setpoint value in under 100 seconds, similar to that seen in other microfluidic control studies. This type of response is found to be more reliable compared to underdamped systems that overshoot the setpoint. Underdamped systems tend to cause large oscillations in output size and have long settling times.

[0065] In contrast, tests are run with constant pressure while manipulating the aqueous flowrate to achieve the new setpoint as shown in FIGs. 8 and 9. The controller can achieve changes in setpoint for increases in diameter as shown in FIG. 8 and for decreases in diameter as shown in FIG. 9. Again, these responses are underdamped, but due to the known long response times caused by adjusting flowrate using a widely used commercial syringe pump, the response curves are more oscillatory in nature. Tuning parameters are known to work best close to the parameters they were tuned at; for example, FIG. 8 shows overshoot for larger setpoint changes and underdamped responses for smaller setpoint changes near the tuning range.Disturbance rejection

[0066] Disturbance rejection can overcome potential disruptions during process operation. This is important in intricate microfluidic processes in which minute variations can have large impacts on the process outputs due to inherently-small length scales. Such disturbances in microbubble production occur due to changes in air pressure, aqueous phase flowrate, fouling, clogging, changes in wetting properties, and external factors that cannot be anticipated.

[0067] For example, a random physical vibration such as one produced by motion of a person by the experimental set-up can significantly impact the uniformity of the resulting bubbles. Effective disturbance rejection in process control systems involves theimplementation of robust control strategies capable of mitigating the adverse effects of these external influences.

[0068] Our control system is superior to existing microfluidic controllers because its CNN architecture allows it to recover from disturbances that would otherwise move it to non-bubble generating flow regimes as shown in FIG. 10. In FIG. 10a there are no bubbles being produced at a pressure of 29.3 kPa, so the controller linearly increases the pressure until bubbles are made and the controller can obtain an error for PID pressure control.

[0069] The onset of bubble production occurs 21.5 seconds later at a breakthrough pressure of 41.9 kPa shown in FIG. 10b. Now that bubbles are being produced and the controller can measure an error, PID control takes over and reduces the pressure to 35.7 kPa to reach the intended setpoint shown in FIG. 10c. This recovery is difficult to predict using only basic PID control because without bubbles being created, there is no way to obtain the error. Although neural networks for microfluidics control have been previously reported, the controllers are trained in the bubble making regime only and do not account for the complex nature of bubble breakup and elevated pressures or flowrates often needed to breakthrough and induce breakup.

[0070] In addition to large disturbances, our controller also works to overcome disturbances small enough to only create error in the bubble size. As mentioned, two separate control schemes can be employed: altering pressure to regain the setpoint after a disturbance in flowrate as shown in FIG. 11 and manipulating flowrate to maintain setpoint after a disturbance in pressure as shown in FIG. 12. The former shows a slightly overdamped response as the pressure slowly decreases without overshoot to regain the diameter setpoint after the flowrate disturbance. The latter is an underdamped response and shows an oscillatory response in the flowrate to return to the setpoint after a disturbance in the pressure is imposed.

[0071] Another important aspect of disturbance rejection is the ability to remain stable for long periods of time. A typical operating shift in various manufacturing industries in the US and many countries is eight hours, during which many changes in operating conditions can occur. FIG. 13 shows performance of a gas bubble generation process left unattended without control measures (i.e., the flowrate and the pressure are kept constant). Over extended durations, frequent disruptions in flow conditions lead to significant variations in the output bubble size. Remarkably, only 2.16% of the produced bubbles fallwithin 5% of the initial bubble diameter. The exact cause of these disruptions is unknown, necessitating the implementation of a control system to counteract them, as they cannot be systematically eliminated from the process.

[0072] This drastic variability is avoided with control action taking place, as shown in FIG. 14. Despite disturbances, the controller dynamically adjusts the pressure to restore the setpoint. Throughout the eight-hour period, our controller achieved 99.2% accuracy, with bubbles deviating by no more than 5% from the setpoint. Notably, the pressure required to satisfy the setpoint must be increased gradually by over 50%; while we do not fully understand the physical origin of such an adjustment in the pressure, this result nevertheless highlights the importance of feedback control to enable stable and robust microfluidic manufacturing.Conclusions

[0073] Microfluidic devices offer precise control for producing droplets and bubbles crucial for various industries. Transitioning from laboratory to industrial-scale operations poses challenges such as disturbances, fouling, and changes in device performance, necessitating continuous monitoring and adjustments to be made to manipulated variables to maintain the user-determined setpoint. Integrating feedback control systems can enhance product uniformity and reduce the labor-intensive tasks associated with process maintenance, addressing a need in scaling up microfluidic processes for industrial applications. As explained herein, the disclosed results show that PID control is a robust feedback control mechanism. It relies on using a soft-sensor to allow for error measurements using artificial intelligence where physical measurements are not reliable. Our convolutional neural network driven soft-sensor identifies flow regimes giving the controller the ability to regain a bubble producing flow regime if a disturbance knocks it into a different flow regime. In addition to self-recovery, our controller takes control actions to reduce the errors to maintain setpoints, counteract disturbances, and to stabilize a system over long time. Our controller shows over 99% of bubbles produced during an 8- hour production period fall within 5% of the setpoint in contrast to only 2.16% of bubbles falling within the same range when control action is not implemented. Leveraging a combination of machine learning and image recognition software, soft-sensors enable feedback control in droplet-based microfluidic systems, enhancing control over the size, shape, and functionality of microfluidic-generated emulsions.MethodsMicrofluidic fabrication and operation

[0074] Photomasks for microfluidic geometries were purchased from Artnet Pro, Inc. Silicon wafers were cleaned with IPA and DI water before oxygen plasma cleaning (Anatech) for enhanced bonding. SU-8 2025 photoresist was spin coated onto the silicon wafer before soft-baking at 65°C for 3 minutes then 95°C for 6 minutes. The wafer was exposed to 160 —2 at 365 nm intensity cm (ABM). After exposure, the wafer was postbaked for 2 minutes at 65°C and then for 6 minutes at 95°C. Lastly, the silicon wafer was gently agitated for 8 minutes in SU-8 developer. PDMS (SYLGARD 184) was mixed in a 10: 1 weight ratio of elastomer to curing agent. The mixture was degassed in a vacuum chamber for 1 hour to remove all bubbles before curing for 1 hour in an 80°F oven. The resulting elastomer mold was cut from the master and oxygen plasma bonded to a glass slide. 2 wt% PVA was surface coated onto the PDMS for a hydrophilic coating.

[0075] The aqueous phase in all experiments was 0.5 wt% SDS dissolved in DI water. The aqueous phase was administered with a Harvard Apparatus PHD Ultra syringe pump. Compressed nitrogen (Airgas) was used for the dispersed phase and was controlled using a differential pressure controller (Alicat). Images used for the control scheme are taking on a Nikon eclipse TE200 inverted microscope with 3 different high-speed cameras, a Photron Mini AX-200, a Phantom Vision Research v7.3, and a Phantom Vision Research v611 proving the adaptability of this approach across multiple microfluidic setups.Neural network architecture and training

[0076] A sequential 19-layer CNN was created using the Keras API inside of Tensorflow. The architecture contained 4 convolution layers using 3 x 3 convolutions and 2 x 2 max pooling. The remaining structure was flattened with a dropout layer set at 50%. There were two dense layers, one with rectified linear activation and the final layer with softmax activation function. The neural net was trained with 128 x 600 resolution images. 43,554 images were used for neural net training and the net was validated with an additional 4,839 images. 100% accuracy was achieved for both training and validation sets within 5 epochs as shown in FIG. 15.Image recognition and feature extraction

[0077] Image recognition was completed using Matlab image recognition functions.Since the bubbles in this work are circular, the function imfindcircles was used. This built-in function uses a Hough transform to isolate features and extract their location and size. Hough transforms are techniques used in computer vision and image analysis for feature extraction.9 This is used to measure the size, position, and uniformity of every bubble present in an image as part of the soft-sensor output.Convolutional neural network augmented soft-sensor for autonomous microfluidic production of uniform bubbles

[0078] Microfluidics has emerged as a foundational process for creating highly uniform emulsions and bubbles. To enable integration of microfluidic platforms into industrial processes, achieving precise control over the size uniformity of microfluidic-generated bubbles and emulsions is crucial. Even if the external variables such as flow rates or pressures are kept constant, microfluidic processes can be easily disturbed by unknown factors that would substantially compromise the uniformity of resulting emulsions and bubbles. In this study, we introduce a two-step soft-sensor approach that combines a convolutional neural network (CNN) and an image recognition algorithm for feature extraction to detect both the flow regime and the size and uniformity of resulting bubbles. By using a CNN to detect flow regimes, the controller described herein is able to restore the bubble-producing flow regime in response to disturbances. Beyond self-recovery, the controller actively adjusts to minimize errors, maintain setpoints, mitigate disturbances, and ensure system stability over extended periods. 99.2% of bubbles produced during an 8-hour period remain within 5% of the setpoint with our controller acting. By leveraging the soft sensor and artificial intelligence-assisted feedback control, the present disclosure presents a widely applicable approach for precise and automated control of microfluidics in diverse applications.Introduction

[0079] Microfluidics enables the production and manipulation of multi-phasic mixtures such as gas bubbles, liquid droplets and multiple emulsions with unparalleled precision and control. Leveraging advanced techniques in micro / nano-fabrication, precise microchannels can be manufactured to control droplets and bubbles for a wide range of advanced applications. For example, chemical reactions can be induced in single droplets, providing a unique platform to conduct high-throughput analyses and synthesis with minimal reagents use and reduced waste. Furthermore, droplet microfluidics facilitates the encapsulation of delicate biological materials such as cells and proteins under mildconditions that preserve their functionality and viability, which is particularly well-suited for the development of low-cost and highly efficient biomedical diagnostics and therapeutics. The precision and scalability of droplet microfluidics enables fabrication of functional particles such as microbubbles, microcapsules and nanoparticles with precisely designed morphology and functionality, enhancing disease diagnostics as well as controlled release and targeted delivery of various pharmaceutical actives.

[0080] Achieving consistent uniformity in the production of droplets and bubbles throughout the operation of microfluidic devices is crucial for harnessing the full benefits of this technology. The maintenance of uniformity is, however, challenged by several factors, which necessitates continuous user intervention to adjust flow rates and pressures, ensuring that the droplet and bubble production maintains the desired dimensions and properties. Even seemingly negligible changes in operating conditions, such as the instrumental uncertainty from pressure / flow controllers or syringe pumps can cause fluctuations in performance of device output because of the sensitivity of flow behaviors of fluids at the microscopic scale. Over time, the performance of microfluidic devices may be further compromised by issues such as surface fouling, changes in wetting properties, channel clogging and the solvent-induced swelling of the microfluidic devices. These factors introduce additional layers of complexity in achieving and maintaining the uniformity of droplet and bubble generation, posing significant challenges to the scalability and reliability of microfluidic applications.

[0081] Developing autonomous microfluidic systems capable of self-adapting to changing conditions would enable the precise formation of a wide array of droplets and bubbles with complex composition and morphology without direct operator intervention. The realization of such a capability will enhance the efficiency and effectiveness of droplet and bubble microfluidics and simultaneously lead to new applications that leverage the full potential of this versatile technology.

[0082] A few recent studies have demonstrated the ability to control droplet microfluidics, using technologies such as neural networks and reinforcement learning to gather insight on flow regimes in microfluidics over various flow conditions. Other techniques involve the use of impedance electronics embedded into the microfluidic device as a sensor for measuring microbubble diameter as a function of the voltage measured or measuring the interference pattern created by focusing a laser on droplets inthe outlet channel using piezoelectric transducer. The Vision Development Module within Lab VIEW (National Instruments™) has been used for droplet detection and control using a virtual instrument. In addition, feedback sensors have been developed for precise control of flowrates and pressures in microfluidic devices off-chip. Despite these advances, many of these approaches use highly sophisticated sensing techniques that involve specialized equipment not traditionally used in microfluidics, do not have the ability to easily measure process variables on-chip, and do not have the ability to control the system when it is disturbed to reach different flow regimes, such as one in which bubbles or droplets are not produced.

[0083] In this study, we introduce an autonomous microfluidics system that relies simply on microscopic imaging and is trained with a convolutional neural network (CNN) to return the process to the desired microbubble production flow regime. The system is able to control the size of microfluidic bubble production using only commercially available pressure controllers, a microscope, and a high-speed camera, all of which are commonly employed in microfluidic setups. We generate gas bubbles in a flow-focusing device by applying pressure to the dispersed gas phase using a high-pressure nitrogen canister. Additionally, we use pressure-driven flow, also from a high-pressure nitrogen canister, to pressurize a liquid reservoir, thereby pushing the liquid into the microfluidic chip as the continuous phase. Pressure driven flow of the aqueous phase is chosen over commercially available syringe pumps because of their significantly reduced response times without periodic fluctuations. Gas bubbles are selected over liquid droplets due to their greater size variability for various reasons including the compressibility of the gas phase, large interfacial tension, and significantly different viscosity of the two phases, necessitating enhanced control.

[0084] The control system actively adjusts either the liquid driving pressure or gas pressure to ensure the bubbles being produced match the user-specified setpoint for the bubble diameter while showing effective setpoint tracking, disturbance rejection, and stability over an eight-hour period. Potentially, the approach can be trained to control the shape of particles produced by microfluidics such as rods and discs, and higher-order geometries such as double emulsions and Janus droplets.Materials and Methods

[0085] We use a well-established flow-focusing generator geometry to produce gas bubbles. This geometry splits the continuous phase into two streams which subsequently surround and pinch off the dispersed phase at a cross junction, as shown in FIG. 16. The symmetry of the junction allows for more flexibility in the size and frequency of bubble generation. The immiscibility of the two phases forces bubbles to form through either a dripping or jetting mechanism. The dripping regime involves the periodic breakup of a fluid stream into bubbles due to capillary instability. This instability arises from the interplay of surface tension and viscous forces. The dripping frequency is governed by the capillary number (Ca), representing the ratio of viscous to capillary forces. The jetting regime involves the stretching of a fluid stream into an extended jet due to the dominance of inertial forces or viscous forces over capillary forces. Regardless of the break-up mechanism, bubbles are grouped into a single class, termed the bubble generation regime. This classification is used because the high-speed camera captures the images of produced bubbles away from the flow-focusing junction.

[0086] In this study, we use a flow-focusing geometry to produce nitrogen gas (dispersed phase) bubbles in an aqueous phase of 0.5 wt% sodium dodecyl sulfate (SDS) dissolved in DI water (continuous phase). The dispersed phase is injected into the device as pressurized nitrogen gas controlled by a differential pressure controller and the continuous phase is injected by using a differential pressure controller to adjust the pressure in a pressurized liquid reservoir and thus drive flow into the microfluidic device as depicted in FIG. 16. All microfluidic devices in this study have undergone hydrophilic- surface treatment to ensure stable formation of gas bubbles using a 2 wt% polyvinyl alcohol (PVA) solution.

[0087] PID control is one of the most fundamental types of control and is widely used in industrial applications due to its simplicity and effectiveness. It has been widely studied, can address a wide range of process behaviors, and is straightforward to implement, making it a natural choice for control of microfluidics. PID feedback control operates by continuously comparing the desired setpoint to the process output, giving the error, e. The proportional term responds to the current error, the integral term accumulates past errors to eliminate steady-state discrepancies (offset), and the derivative term accounts for the rate of change of the error, as shown in Equation 1.where Kcis the proportional gain, T, is the integral time-constant in minutes, TDis the derivative time constant in minutes, and csis the controller bias (actuating signal when e=0). These terms are combined to compute the controller output (u(t)), aiming to reduce the error and maintain stable and precise control of the process. In this study, two separate control schemes are explored: manipulating the pressure of the dispersed air phase to control microbubble diameter and manipulating the liquid driving pressure of the continuous phase to control microbubble diameter as illustrated in the control scheme of FIG. 16.

[0088] Altering the PID tuning parameters Kc, TI, and TDcan significantly influence the response of the controller; increasing the proportional gain enhances responsiveness, but may lead to overshooting, while adjusting the integral and derivative time-constants affects the elimination of steady-state error and reduces the settling time, respectively. This method has been used in many applications, but it relies on measuring the process output in real-time. Numerous process parameters, including pressure and temperature, can be measured directly using commercially available real-time in-line sensors. However, microfluidic processes currently lack reliable, industry -proven inline sensor technologies that can directly measure the important process variables that need to be controlled such as size, shape, and flow regime. Thus, we use an artificial intelligence (Al)-based approach to create an indirect sensor, known as a soft-sensor, when controlling microfluidics.

[0089] When sensor variables are difficult to measure, soft-sensors can be used to estimate them. There are three types of soft-sensor models: knowledge-based models that rely on first principles, black-box models that are data-driven, and hybrid models that combine the two. In this study, a black-box model consisting of a two-step process is used to estimate the flow regime and diameter of the process output, gas bubbles. The first step in our process is using a convolutional neural network (CNN) for image classification. CNNs have played a crucial role in the development and advancement of computer vision and artificial intelligence (Al). In this study, a linear architecture CNN is used to classify the microfluidic output into one of three regimes: liquid-dominated flow, air-dominated flow, and bubble generation, as shown in FIG. 2. The second step in our process is using aHough image recognition algorithm for detection of the microbubbles. This algorithm is designed for feature extraction and gives outputs of their location and diameter.

[0090] Our novel soft-sensor operates as follows: first, a CNN trained for image classification determines the current flow regime of the process from a snapshot of the device taken by a high-speed microscope camera. When the process is in a bubble producing regime, an image recognition algorithm measures the diameter of the bubbles being produced. The combination of the CNN and image recognition is the soft-sensor output on which the controller acts to match the bubble diameter to the user-specified setpoint. When the process is not in a bubble producing flow regime (either liquid- dominated or air-dominated flow), a direct controller action is taken to return the process to the production of gas bubbles. Our two-step approach is superior to other microfluidic control techniques because of its ability to return to the bubble generation regime when a disturbance causes it to move into a flow regime in which bubbles are not being produced. Our controller acts in real-time and can obtain measurements and adjust setpoints with an average sampling frequency of 108 milliseconds.

[0091] Photomasks for microfluidic geometries were purchased from Artnet Pro, Inc. Silicon wafers were cleaned with IPA and DI water before oxygen plasma cleaning (Anatech) for enhanced bonding. SU-8 2025 photoresist was spin coated onto the silicon wafer before soft-baking at 65°C for 3 minutes, then at 95°C for 6 minutes. The wafer was exposed to 160 mJ / cm2at 365 nm intensity (ABM). After exposure, the wafer was postbaked for 2 minutes at 65°C and then for 6 minutes at 95°C. Lastly, the silicon wafer was gently agitated for 8 minutes in SU-8 developer. PDMS (SYLGARD 184) was mixed in a 10: 1 weight ratio of elastomer to curing agent. The mixture was degassed in a vacuum chamber for 1 hour to remove all bubbles before curing for 1 hour in an 80°F oven. The resulting elastomer mold was cut from the master and oxygen plasma bonded to a glass slide. 2 wt% PVA was surface coated onto the PDMS for a hydrophilic coating.

[0092] The aqueous phase in all experiments was 0.5 wt% SDS dissolved in DI water. The aqueous phase was administered using a differential pressure controller (Alicat) to pressurize a liquid reservoir to drive flow into the device. Compressed nitrogen (Airgas) was used for the dispersed phase and was controlled using a differential pressure controller (Alicat). Images used for the control scheme were taken on a Nikon eclipse TE200 inverted microscope with 3 different high-speed cameras, a Photron Mini AX-200, aPhantom Vision Research v7.3, and a Phantom Vision Research v611 proving the adaptability of this approach across multiple microfluidic setups.

[0093] A standard sequential 19-layer CNN was created using the Keras API inside of TensorFlow. The architecture was designed by TensorFlow for classifying images into 3 categories for the game rock-paper-scissors. This model was chosen because our dataset also has 3 classes of images and because this architecture is simple and performs well. The architecture contained 4 convolution layers using 3 x 3 convolutions and 2 x 2 max pooling. The remaining structure was flattened with a dropout layer set at 50%. There were two dense layers, one with rectified linear activation and the final layer with softmax activation function. The neural net was trained with 128 x 600 resolution images over 5 epochs with batch size of 100. In total, 48,393 images were collected for training the neural network via high-speed imaging. The images spanned all three classes: liquid- dominated, air-dominated, and the bubble generation regime with bubble sizes ranging from 30 - 100 pm. The training images were split 90% for training, and 10% for cross- validation of the model. 100% accuracy was achieved for both training and validation sets within 5 epochs as shown in FIG. 15.

[0094] The CNN was trained only on images in the gas-in-water system previously mentioned; however, because of the similarity in refractive index, the CNN is found to be effective at classifying images for gas-in-oil bubbles as well. New results show the effectiveness of setpoint tracking for gas-in-oil bubbles shown in Supplementary Information Section 7. New training images are needed for systems in which the refractive index of the dispersed phase differs greatly from that of air, i.e. liquid-liquid systems.

[0095] Image recognition was completed using MATLAB image recognition functions. Since the bubbles in this work are circular, the function imfmdcircles was used. This built- in function uses a Hough transform to isolate features and extract their location and size. Hough transforms were first patented by the U.S. Atomic Energy Commission in 1962 for detecting particles in bubble chambers and its generalized algorithm is still used widely in computer vision and image analysis for feature extraction. This algorithm is used to measure the size, position, and uniformity of every bubble present in an image as part of the soft-sensor output. Hough transforms are an established method for geometric object detection software, and the imfmdcircles algorithm developed by MATLAB is known forits sub-pixel accuracy. A stage micrometer was used for calculating the ratio between pixels and micrometers (pm).Results and Discussion

[0096] Tuning the PID control parameters is required to achieve reliable controller responses. In this study, we tune a single-input, single-output (SISO) controller to control the output diameter of bubbles made in a microfluidic device by varying either the air pressure of the dispersed phase, or the liquid driving pressure of the continuous phase. Ziegler-Nichols open-loop response tuning rules are used to acquire initial tuning parameters before trial-and-error adjustments are made to obtain tuning parameters that result in the desired overdamped responses. Ziegler-Nichols tuning is appropriate because the system responses exhibit first-order plus dead-time behavior (FOPDT). Tuning parameters are obtained without the controller while monitoring the response to a step change in the manipulated variable. As shown in the process reaction-curve in FIG. 3, a tangent line through the inflection point is drawn to estimate the delay time (rd) and response time (T). Here, u is the step change in the process input, such as gas pressure, and y is the change in the measured process variable, which is the bubble diameter herein.These variables are used to calculate controller parameters, where Kc= 1.2 >Ti =2.0 Tdand TD= 0.5 Td. Small changes in tuning parameters are then made to achieve the desired slightly overdamped response from our controller.

[0097] For a step pressure increase from 18.7 to 20.3 kPa, an increase in diameter is shown in FIG. 18a-b. Plotting the tangent line to the curve, the delay time is 1.1 second and the response time is 1 second. These yield Kc = 0.125 kPa / pm, TI= 2.2 seconds, and TD= 0.55 seconds. Instead of the gas pressure, the flowrate of the continuous phase is manipulated by varying the liquid driving pressure. For a driving pressure step change from 31.7 kPa to 29.6 kPa, the delay time is 1.2 seconds, and the response time is 2.3 seconds, yielding c = -0.21 kPa / pm, TI= 2.4 seconds, and TD= 0.6 seconds, as shown in FIG. 18c-d. Detailed information on tuning with Ziegler-Nichols open loop method can be found in Supplementary Information Section 6.

[0098] Setpoint tracking by PID control holds significant importance in optimizing process performance, ensuring close adherence to desired operating conditions and allowing for switches to new operating setpoints. For many microfluidic processes, effective changes in bubble / droplet diameters are required for different applications. Forexample, the gas bubble diameter is crucial in determining its resonance frequency, particularly in applications where bubbles serve as a contrast agent in ultrasound sonography. In the context of microfluidic reactors, the droplet diameter plays a crucial role in influencing the reaction rates and kinetics of associated chemical processes.

[0099] With our controller, we can dynamically manipulate either the continuous phase (aqueous flowrate via liquid driving pressure), or the dispersed phase (air pressure) to reach a desired setpoint, output bubble diameter. FIGS. 18-19 show the experimental controller actions after a diameter setpoint changes at time zero. Each color represents a different setpoint change: dashed lines for setpoint changes, and solid lines for process variable changes. The liquid driving pressure (aqueous flowrate) is kept constant, and the air pressure is manipulated by the controller to reach the new setpoint in FIG. 18, where the first shows the response for an increase in the diameter (3a-b) setpoint and the latter for a decrease in the diameter setpoint (3c-d). Both responses are adjusted to be slightly overdamped; that is, having small overshoot before settling to the desired value, with all responses settling to the new setpoint in under 100 seconds - similar to those in other microfluidic control studies. This response is more reliable compared to underdamped systems having large oscillations in diameter and long settling times. Underdamped tuning parameter responses are shown in FIGS. 23 and 24.

[0100] We also perform tests with constant pressure while manipulating the aqueous flowrate via liquid driving pressure to achieve the new setpoint as shown in FIG. 19. The controller can achieve changes in setpoint for increases in diameter as shown in FIG. 19a- b and for decreases in diameter as shown in FIG. 19c-d. Again, these responses are overdamped, not allowing any overshoot, and it is seen that response times are considerably faster using pressure driven flow for the aqueous phase than widely used commercially available syringe pumps seen in FIGS. 8 and 9.

[0101] Disturbance rejection is needed to overcome all potential disruptions during process operation. This is especially important in intricate microfluidic processes in which minute variations can have large impacts on the process outputs due to inherently-small length scales. Such disturbances in microbubble production occur due to changes in air pressure, aqueous phase flowrate, fouling, clogging, changes in wetting properties, and external factors that cannot be anticipated. For example, a random physical vibration such as one produced by motion of a person near the microfluidic set-up can significantlyimpact the uniformity of the resulting bubbles. Feedback control was selected for microfluidics because it provides a robust solution to managing unpredictable disturbances that cannot be fully anticipated or systematically eliminated. Unlike feedforward control, which requires precise knowledge of all potential disturbances, feedback control continuously monitors the system's output and automatically adjusts to counteract any deviations. This real-time correction ensures that the effects of disturbances are rapidly mitigated, enhancing the system's stability and efficiency without the need for ongoing human intervention.

[0102] Our control system is superior to many microfluidic controllers because its CNN architecture allows it to recover from large disturbances that would otherwise move to non-bubble generating flow regimes, as shown in FIG. 20. FIG. 20a tracks the diameter of bubbles being produced at pressures shown in FIG. 20b in response to a large disturbance. In FIG. 20c, there are no bubbles being produced at a pressure of 29.3 kPa, so the controller linearly increases the pressure until bubbles are generated and the controller obtains an error for PID diameter control. The onset of bubble production occurs 21.5 seconds later at a breakthrough pressure of 41.9 kPa shown in FIG. 20d. Now that bubbles are being produced and the controller can measure an error, PID control takes over and reduces the pressure to 35.7 kPa to reach the intended setpoint shown in FIG. 20e. Basic PID control cannot achieve this transition to bubble creation because without bubbles, there is no way to obtain the current error. A recovery of flow regime video is shown in Video S5. Although neural networks for microfluidics control have been reported, their controllers are trained in the bubble generating regime only and do not account for the complex nature of bubble breakup at elevated pressures or flowrates often needed to achieve breakthrough and induce breakup.

[0103] To track the CNN's activation frequency in disturbance recovery, a one-hour test is conducted at a constant setpoint, beginning with a dispersed phase pressure of 0 kPa. The CNN increases the pressure to initiate bubble production, after which PID control maintains the setpoint. The CNN activates if disturbances push the system into liquid- or air-dominated flow regimes. Over the hour, PID control maintained the setpoint 99.2% of the time, but the CNN's disturbance recovery is crucial for sustained bubble production, as shown in FIG. 27. This experiment demonstrates the controller's ability tostart from a zero pressure condition using the CNN to drive the pressure up into the bubble production regime.

[0104] In addition to large disturbances, our controller overcomes disturbances small enough to create error only in the bubble diameter. As mentioned, two separate control schemes can be employed: altering dispersed phase pressure to regain the setpoint after a disturbance in flowrate as shown in FIG. 21a-c and manipulating flowrate by changing liquid driving pressure to maintain setpoint after a disturbance in pressure as shown in FIG. 21d-f. The former shows a slightly overdamped response as the pressure slowly decreases without overshoot to regain the diameter setpoint after the flowrate disturbance. The pressure response is able to regain the setpoint in under 20 seconds. The latter is also a slightly overdamped response to return the diameter to the setpoint following a sharp increase in dispersed phase pressure that causes the diameter to increase quickly above the setpoint. The controller increases the liquid driving pressure and thus the aqueous flowrate returns the bubbles to the setpoint in only 20 seconds.

[0105] Another important aspect of disturbance rejection is the ability to remain stable for long periods of time. A typical operating shift in manufacturing industries in the US and many countries is eight hours during which many changes in operating conditions can occur. FIG. 22a-b shows performance of a gas bubble generation process left unattended without control measures (i.e., the flowrate and the pressure are kept constant). Over extended durations, frequent disruptions in flow conditions lead to significant variations in the output bubble size. Remarkably, only 2.16% of the produced bubbles fall within 5% of the initial bubble diameter. The exact cause of these disruptions is unknown, necessitating the implementation of a control system to counteract them, as they cannot be systematically eliminated from the process.

[0106] This drastic variability is avoided with control action, as shown in FIG. 22c-d. Despite disturbances, the controller adjusts the pressure to maintain the setpoint. Throughout the eight-hour period, our controller achieved 99.2% accuracy, with bubbles deviating by no more than 5% from the setpoint. Notably, the pressure required to satisfy the setpoint must be increased gradually by over 50%; while we do not fully understand the physical origin of such an adjustment in the pressure, this result nevertheless highlights the importance of feedback control to enable stable and robust microfluidic manufacturing.Conclusion

[0107] Microfluidic devices offer precise control for producing droplets and bubbles crucial for various industries. Transitioning from laboratory to industrial-scale operations poses challenges presented by disturbances, fouling, and changes in device performance. These necessitate continuous monitoring and adjustments to manipulated variables that maintain user-specified setpoints. The integrating feedback controllers herein enhance product uniformity and reduce the labor-intensive tasks associated with process maintenance, addressing a critical need in scaling-up microfluidic processes for industrial applications. Our experimental results show that PID control is a resilient feedback control mechanism, relying on a soft-sensor to obtain error measurements using artificial intelligence in the face of unreliable physical measurements. Our CNN-driven, soft-sensor identifies flow regimes enabling the controller to regain bubble-generation flow regimes when shifted by disturbances to undesired regimes. In addition to self-recovery, our controller reduces errors while maintaining setpoints, countering disturbances, and stabilizing operation over long times. Our controller permits over 99% of bubbles produced during 8-hours to fall within 5% of setpoint diameters; in contrast to only 2.16% when control action is not implemented. Leveraging a combination of machine learning and image recognition software, soft-sensors herein enable feedback control in dropletbased microfluidic systems, potentially enhancing control over the size, shape, and functionality of microfluidic-generated emulsions.Supplementary Information

[0108] FIGS. 23 and 24 shows underdamped control parameters leading to increased oscillations in microfluidics.

[0109] FIG. 25 shows when CNN is active in control for flow regime recovery.

[0110] In this study, a high-speed camera is used to collect images across 3 classes of operation in a flow-focusing microfluidic device: liquid dominated flow, air-dominated flow, and bubble generation, shown in FIG. 2. In total, 48,393 images were collected across the three classes. In the bubble generation regime, images were collected at various conditions leading to the generation of bubbles in a range from 30-100 pm, ensuring the CNN will be trained to find bubbles in a large range of sizes. After the dataset was collected, the images were split automatically so that 90% of each class was designated for the training dataset, and the remaining 10% were designated for the validation dataset.This method is known as cross-validation and is commonly employed to avoid overfitting the model. The images obtained for training were collected immediately downstream of the flow-focusing junction and were 128x600 resolution images. This size was chosen because it was the smallest width image to capture the entire channel, and the longest length so that the most information could be collected for images inside the channel.

[0111] The CNN used in this study is a classification neural network with 19 sequential layers developed originally for classification of images for the game rock-paper-scissors. Training of the neural network was performed using a hosted GPU in Google Colab using the TensorFlow framework. After importing the dataset, the model was trained using 5 epochs and a batch size of 100 images. Because of the size of the dataset, the model took 10+ hours to train; however, 100% accuracy was achieved in both training and validation at the end of 5 epochs seen in FIG. 15. While it is not typical for validation accuracy to outperform the training accuracy, it can occur because of dropout layers that can introduce noise, data augmentation to rotate images to diversify the dataset, or overfitting. The source of the lower training accuracy was not explored because the model performed very well and the overall simplicity of the images being classified.

[0112] Ziegler-Nichols tuning parameters are obtained using the open-loop graphical method developed in 1942. The open-loop method uses the results from the process reaction curve to estimate parameters as shown in FIG. 3. To tune using this method, while the controller is inactive, a step change is introduced in a process variable (u) while the output (y) is measured. When plotted, a tangent through the inflection point of the output curve gives the delay time (id) and response time (T) when it crosses the initial output value (y i) and final output value (y?). For this study, the output value is the diameter of the bubbles being generated. Two different controllers were tuned for this study, one for controlling the pressure of the gas in the dispersed phase (FIG. 17a-b) and the other for controlling the driving pressure of the liquid in the continuous phase (FIG. 17c-d).

[0113] For complete tuning, the Ziegler-Nichols method was used to obtain the values for change in input pressure, change in diameter, the delay time, and the response time to get a first guess at the tuning parameters. Then, trial-and-error tests were performed near the obtained tuning parameters until the desired slightly overdamped response was obtained.

[0114] Setpoint tracking results for gas bubble generation in HFE oil containing 2 wt% Krytox were obtained using a flow-focusing device with identical geometry and a silanized surface treatment. These results demonstrate the applicability of the CNN in both gas-in-water and gas-in-oil bubbles, due to the minimal difference in refractive index between the two continuous phases without the need for re-training the neural net.

[0115] Comparison of the images in FIG. 32 for the gas-in-water bubbles (a) and the gas-in-oil bubbles (b) reveals minimal visual differences, which can be attributed to the small relative differences in the refractive indices of the continuous phases, thus the CNN works for both systems without re-training.EXEMPLARY EMBODIMENTS

[0116] The following embodiments are exemplary only and do not serve to limit the scope of the present disclosure of the appended claims. It should be understood that any part of any one or more Embodiments can be combined with any part of any other one or more Embodiments.Embodiment 1

[0117] A method, comprising: receiving image data of an output of a droplet generator; inputting the image data into a trained image classification model; determining, based on the image data and via the image classification model, a flow regime of the droplet generator, wherein the flow regime comprises one of: (a) a dripping regime, (b) a nonproduction regime, and (c) a jetting regime; and adjusting a parameter of the droplet generator when the flow regime comprises either (b) or (c).

[0118] Embodiment 2

[0119] The method of Embodiment 1, further comprising determining, based on the image data and via the image classification model, whether the droplet generator is failing to produce microfluidic bubbles.

[0120] Embodiment 3

[0121] The method of Embodiment 1 or 2, further comprising determining, based on the image data and via the image classification model, whether a channel of the droplet generator is filled with a liquid.

[0122] Embodiment 4

[0123] The method of any of Embodiments 1-3, further comprising determining, based on the image data and via the image classification model, whether a channel of the droplet generator is filled with a stream of gas encircled by a layer of liquid between the stream of gas and a sidewall of the channel.

[0124] Embodiment 5

[0125] The method of any of Embodiments 1-4, wherein the parameter comprises any one or more of a flow rate, a pressure, a capillary number, a Weber number, or an amount of a surfactant.

[0126] Embodiment 6

[0127] The method of any of Embodiments 1-5, wherein the droplet generator comprises a flow focusing generator.

[0128] Embodiment 7

[0129] The method of any of Embodiments 1-6, wherein the droplet generator produces at least one of microfluidic bubbles or an emulsion when the flow regime is a dripping regime.

[0130] Embodiment 8

[0131] The method of any of Embodiments 1-7, wherein the flow regime is a dripping regime, and further comprising: determining, based on the image data and via the image classification model, a first characteristic value of the output of the droplet generator; comparing the first characteristic value to a first target characteristic value; and adjusting a parameter of the droplet generator such that the first characteristic value of the output of the droplet generator approaches the first target characteristic value.

[0132] Embodiment 9

[0133] The method of Embodiment 8, wherein the first characteristic comprises a first cross-sectional dimension of the output or a wall thickness of the output.

[0134] Embodiment 10

[0135] The method of Embodiment 8 or 9, further comprising: determining, based on the image data and via the image classification model, a second characteristic value of the output of the droplet generator; comparing the second characteristic value to a second target characteristic value; and adjusting a parameter of the droplet generator such that the second characteristic value of the output of the droplet generator approaches the second target characteristic value.

[0136] Embodiment 11

[0137] The method of any of Embodiments 8-10, wherein the second characteristic comprises a second cross-sectional dimension of the output or a wall thickness of the output.

[0138] Embodiment 12

[0139] A method, comprising: receiving image data of an output of a droplet generator; inputting the image data into a trained image classification model; determining, based on the image data and via the image classification model, a first characteristic value of the output of the droplet generator; comparing the first characteristic value to a first target characteristic value; and adjusting a parameter of the droplet generator such that the first characteristic value of the output of the droplet generator approaches the first target characteristic value.

[0140] Embodiment 13

[0141] The method of Embodiment 12, wherein the parameter comprises any one or more of a flow rate, a pressure, a capillary number, a Weber number, or an amount of a surfactant.

[0142] Embodiment 14

[0143] The method of Embodiments 12 or 13, wherein the droplet generator comprises a flow focusing generator.

[0144] Embodiment 15

[0145] The method of any of Embodiments 12-14, wherein output of the droplet generator comprises at least one of microfluidic bubbles or an emulsion.

[0146] Embodiment 16

[0147] The method of any of Embodiments 12-15, wherein the first characteristic comprises a first cross-sectional dimension of the output or a wall thickness of the output.

[0148] Embodiment 17

[0149] The method of any of Embodiments 12-16, further comprising: determining, based on the image data and via the image classification model, a second characteristic value of the output of the droplet generator; comparing the second characteristic value to a second target characteristic value; and adjusting a parameter of the droplet generator such that the second characteristic value of the output of the droplet generator approaches the second target characteristic value.

[0150] Embodiment 18

[0151] The method of any of Embodiments 12-17, wherein the second characteristic comprises a second cross-sectional dimension of the output or a wall thickness of the output.

[0152] Embodiment 19

[0153] The method of any of Embodiments 12-18, wherein the classification model comprises a convolutional neural network.

[0154] Embodiment 20

[0155] A system configured to perform the method of any of Embodiments 1-19, comprising: an imager configured to collect image data of an output of a droplet generator; a trained classification model configured to classify the image data; and a controller configured to adjust a parameter of the droplet generator.

Claims

What is Claimed:

1. A method, comprising: receiving image data of an output of a droplet generator; inputting the image data into a trained image classification model; determining, based on the image data and via the image classification model, a flow regime of the droplet generator, wherein the flow regime comprises one of:(a) a dripping regime,(b) a non-production regime, and(c) a jetting regime; and adjusting a parameter of the droplet generator when the flow regime comprises either (b) or (c).

2. The method of claim 1, further comprising determining, based on the image data and via the image classification model, whether the droplet generator is failing to produce microfluidic bubbles.

3. The method of claim 1, further comprising determining, based on the image data and via the image classification model, whether a channel of the droplet generator is filled with a liquid.

4. The method of claim 1, further comprising determining, based on the image data and via the image classification model, whether a channel of the droplet generator is filled with a stream of gas encircled by a layer of liquid between the stream of gas and a sidewall of the channel.

5. The method of claim 1, wherein the parameter comprises any one or more of a flow rate, a pressure, a capillary number, a Weber number, or an amount of a surfactant.

6. The method of claim 1, wherein the droplet generator comprises a flow focusing generator.

7. The method of claim 1, wherein the droplet generator produces at least one of microfluidic bubbles or an emulsion when the flow regime is a dripping regime.

8. The method of claim 1, wherein the flow regime is a dripping regime, and further comprising: determining, based on the image data and via the image classification model, a first characteristic value of the output of the droplet generator; comparing the first characteristic value to a first target characteristic value; and adjusting a parameter of the droplet generator such that the first characteristic value of the output of the droplet generator approaches the first target characteristic value.

9. The method of claim 8, wherein the first characteristic comprises a first cross- sectional dimension of the output or a wall thickness of the output.

10. The method of any one of claims 8-9, further comprising: determining, based on the image data and via the image classification model, a second characteristic value of the output of the droplet generator; comparing the second characteristic value to a second target characteristic value; and adjusting a parameter of the droplet generator such that the second characteristic value of the output of the droplet generator approaches the second target characteristic value.

11. The method of claim 10, wherein the second characteristic comprises a second cross-sectional dimension of the output or a wall thickness of the output.

12. A method, comprising: receiving image data of an output of a droplet generator; inputting the image data into a trained image classification model;determining, based on the image data and via the image classification model, a first characteristic value of the output of the droplet generator; comparing the first characteristic value to a first target characteristic value; and adjusting a parameter of the droplet generator such that the first characteristic value of the output of the droplet generator approaches the first target characteristic value.

13. The method of claim 12, wherein the parameter comprises any one or more of a flow rate, a pressure, a capillary number, a Weber number, or an amount of a surfactant.

14. The method of claim 12, wherein the droplet generator comprises a flow focusing generator.

15. The method of claim 12, wherein output of the droplet generator comprises at least one of microfluidic bubbles or an emulsion.

16. The method of claim 12, wherein the first characteristic comprises a first cross- sectional dimension of the output or a wall thickness of the output.

17. The method of any one of claims 12-16, further comprising: determining, based on the image data and via the image classification model, a second characteristic value of the output of the droplet generator; comparing the second characteristic value to a second target characteristic value; and adjusting a parameter of the droplet generator such that the second characteristic value of the output of the droplet generator approaches the second target characteristic value.

18. The method of claim 17, wherein the second characteristic comprises a second cross-sectional dimension of the output or a wall thickness of the output.

19. The method of any one of claims 12-19, wherein the classification model comprises a convolutional neural network.

20. A system configured to perform the method of any of claims 1-19, the system comprising: an imager configured to collect image data of an output of a droplet generator; a trained classification model configured to classify the image data; and a controller configured to adjust a parameter of the droplet generator.

Images