Microfluidic-based fiber formation method and fiber formation system

Abstract

The microfluidic-based fiber formation method and system employs computer vision and deep learning systems and methods to enable non-contact sensing, analysis, and monitoring of key operating parameters within the microfluidic bridge printhead on the 3D bioprinter. The embodiments may use object detection and / or semantic segmentation to facilitate sensing, analysis, and monitoring. Deep learning can use convolutional neural networks to localize and analyze the flow of different biological materials within the microchannels, as well as to identify the operation of various microfluidic printhead-on-chip components that may affect the final quality of the printed tissue. The printed tissue can include single-material fibers, including hollow fibers and more complex coaxial layered fibers.

Description

[Technical field]

[0001] CROSS-REFERENCE TO RELATED APPLICATIONS This application claims the benefit of U.S. Provisional Application No. 63 / 238,028, filed September 7, 2021, the entire contents of which are incorporated herein by reference.

[0002] Aspects of the invention relate to quality control methods and apparatus for monitoring microfluidic bridge printhead performance. More specific aspects relate to methods and apparatus for monitoring microfluidic bridge performance in three-dimensional (3D) bioprinting platforms. Such platforms can produce either single fibers or coaxial multi-layered hydrogel fibers with distinct fiber core and shell components, depending on the printhead used. Even more specific aspects relate to computer vision and machine learning systems that provide real-time visual images of the printhead nozzle and of the fiber being formed within the nozzle to quantify fiber geometric features, such as fiber diameter and fiber core and shell concentricity. [Background technology]

[0003] The cell-loaded fibers (usually containing a core and a shell) are the building blocks of 3D bioprinted tissues. A specific and consistent fiber structure is required to maintain therapeutic cell viability, tissue function, and protection from the immune system of the host into which the tissue is implanted.

[0004] One aspect of the fiber structure takes into account the diffusion of oxygen and nutrients into the inner cell-containing region of the fiber. Such diffusion can be a function of the thickness of the fiber's shell material. A thicker shell can result in a significant reduction in oxygen and nutrients, as well as therapeutic cell starvation, cell death, and ultimately loss of function. Therefore, maintaining shell dimensions below a certain thickness is important to ensure cell viability and function. On the other hand, maintaining shell dimensions above a certain thickness may be important to protect the cell payload from host immune attack. These two conflicting dimensions form a shell thickness window that not only provides immune protection but also ensures the maintenance of cell function. The window may be very narrow.

[0005] In addition to the concentricity of the fiber core and shell, the overall fiber diameter may be related to the host's ability to accept the fiber. Fibers with diameters less than 1 mm may cause a stronger fibrotic response, making maintenance of a fiber diameter greater than 1 mm desirable to reduce unnecessary fibrosis. A consistent fiber diameter during tissue fabrication is also important. Bioprinted tissues consist of multiple layers. If the fiber diameter is not well controlled, each layer of the resulting tissue may be non-uniform. Furthermore, repeated non-uniformities may exacerbate errors in fiber thickness. As a result, the overall macrostructure of the bioprinted tissue may lose fidelity and suffer from reduced mechanical integrity and mechanical function. Thus, it can be seen that for a bioprinted tissue to be reliable enough for clinical use, the concentricity of the core and shell, as well as the fiber diameter, must be consistent.

[0006] Unfortunately, misconcentricity can occur during bioprinting of coaxial fibers for a variety of reasons, with the fabrication of printheads to produce such fibers being one source of problems. In particular, conventional fabrication of such printheads requires bonding multiple stacks of layers, e.g., transparent polydimethylsiloxane (PDMS) layers or glass layers. Poor alignment of the layers during bonding can also lead to poor alignment of channels and valves, which in turn can negatively affect material flow during fiber formation, leading to a lack of concentricity. Another consideration is that biomaterial properties, such as viscosity under varying pressures during printing, can affect not only fiber diameter but also the alignment of the central axes of the fiber's core (middle) and shell (outer).

[0007] Still further, material anomalies such as air bubbles and, in the case of cell-loaded fibers, cell clusters can also lead to non-concentricity. Figures 1A-1F show respective cross-sectional views taken along the length and diameter of a representation of a fiber with different locations of the core inside the shell of the fiber. In Figure 1A, fiber 100 has a center axis 110, a core 120, a core outer edge 130, a shell 140, and a shell outer edge 150. Corresponding numbering applies to Figures 1B-1F. Figure 1A is an example of a fiber with concentricity between the core 120 and the shell 140. Figures 1B-1F are different examples of non-concentricity.

[0008] It would be desirable to provide a system that not only monitors fiber production but also provides a feedback mechanism to correct for non-concentricities identified during macrofluidic crosslinking. Additionally, it would be desirable to provide a system that teaches a 3D bioprinting system to produce consistently concentric fibers with consistent core and shell diameters to facilitate the generation of acceptable 3D tissues. Summary of the Invention

[0009] The present invention addresses the above-mentioned problems in the art by integrating computer vision and deep learning into a three-dimensional (3D) bioprinting platform, thereby enabling direct monitoring of the movement and flow of multiple different materials within a microfluidic crosslinking printhead, including, for example, cell-laden hydrogels and other crosslinkable materials. In an embodiment, the present invention includes one or more cameras as part of a computer vision system and method for imaging the simultaneous flow of multiple different materials within the microfluidic crosslinking printhead. Further embodiments include a light emitting diode (LED) or LED array, each of which illuminates the transparent features and internal fibers of the microfluidic printhead. In an embodiment, one or more mirrors may be substituted for one of the cameras.

[0010] Embodiments of the present invention allow for non-contact sensing of biomaterial characteristics as they flow out of the microfluidic bridges to form fibers for 3D biprinting. Embodiments also then allow for automated analysis that allows for the derivation of quantitative measures of printed fiber properties. These measures can serve as quality control and / or quality assurance parameters.

[0011] Aspects of the invention integrate computer vision and intelligent / deep / machine learning into a 3D bioprinting platform to directly monitor the motion and co-flow of different materials, including one or more hydrogel materials, in a microfluidic bridge device. In embodiments, the system of the invention allows for non-contact sensing of fiber formation. Embodiments of the system of the invention provide automated analysis to derive quantitative measures of printed fiber properties as quality control and / or quality assurance parameters. In some embodiments, such quantitative measures include real-time high-level quantification of biological material (e.g., cells) throughout the fiber during printing of the biological material for qualitative assessment of fiber quality with respect to consistency of biological material throughout the printed fiber. In some embodiments, such quantitative measures include real-time high-level quantification of other objects within the material flow, such as, for example, particulates.

[0012] In embodiments, different machine learning tools can be used. For example, convolutional neural networks (CNNs) can be used for object detection. As another example, semantic segmentation can be used to monitor and analyze different properties of the print head and the resulting fabric during printing.

[0013] In embodiments, the output of various machine learning tools can be fed back in real time to the 3D bioprinting platform to adjust the pressure and / or displacement and subsequent flow of material within the microfluidic channels of the print head to correct for diameter and / or non-concentricity, thus ensuring consistent production of high quality fibers and minimizing loss of expensive biomaterial and cellular influx in the bioprinted fibers.

[0014] In an embodiment, the material stream comprises at least one crosslinkable material, and preferably at least one hydrogel, and optionally further comprises at least one biological material, such as a cell population in a biocompatible material. In an embodiment, the cell population comprises or is selected from the group consisting of a single cell suspension, a cell aggregate, a cell spheroid, a cell organoid, or a combination thereof. In an embodiment, the material stream comprises particulates.

[0015] In embodiments, the cell population comprises cells from endocrine and exocrine glands selected from the group consisting of pancreas, liver, thyroid, parathyroid, pineal, pituitary, thymus, adrenal, ovary, testis, enteroendocrine cells, stem cells, stem cell-derived cells, or cells engineered to secrete a bioactive agent of interest. In embodiments, the cell population releases cell-derived extracellular vesicles in the form of exosomes containing therapeutic proteins or nucleic acids. In embodiments, the biocompatible material is selected from alginate, collagen, decellularized extracellular matrix, hyaluronic acid, PEG, fibrin, gelatin, GEL-MA, silk, chitosan, cellulose, PCL, PLA, POEGMA, and combinations thereof.

[0016] In some embodiments, the microfluidic print head may use pressure to control material flow in fiber production, hi other embodiments, the microfluidic print head may use material displacement to control material flow in fiber production.

[0017] Among other things, embodiments of the present invention enable real-time inspection of various geometric features of the multi-layer fiber being produced. Among the resulting advantages is avoiding the need to incorporate expensive and / or complex pressure sensors or other microelectromechanical systems (MEMS) based technology into the printhead to monitor valve and channel pressures and to detect defects, anomalies, and / or faults in the process and printed fiber.

[0018] In one aspect, the invention provides a microfluidic bridge printhead material flow sensing system including a microfluidic bridge printhead, a material flow including at least one crosslinkable material, a camera system for monitoring the material flow through the microfluidic bridge printhead and providing streaming images of the material flow, and a computer system for determining physical properties of a printed fiber resulting from crosslinks generated by the material flow by analyzing the material flow represented in the streaming images, the computer system including a machine learning based system that compares the streaming images of the material flow to user established material flow parameters corresponding to physical properties of the printed fiber within predetermined tolerances and records the material flow parameters of the material flow and the results of the comparison.

[0019] In an embodiment, the microfluidic bridge print head includes one or more transparent channels and the camera system monitors material flow through at least one of the one or more transparent channels, preferably the microfluidic bridge print head includes a transparent nozzle or metering and dispensing channel.

[0020] In an embodiment, the camera system includes a first camera positioned at a first angle relative to at least one of the one or more transparent channels and a second camera positioned at a second, different angle relative to at least one of the one or more transparent channels, hi an embodiment, the first camera and the second camera are at a right angle relative to each other.

[0021] In an alternative embodiment, the camera system includes a camera and a plurality of mirrors arranged to provide a first view and a second, different view for at least one of the one or more transparent channels, and the camera receives images of the first and second views. In an embodiment, the second view is orthogonal to the first view. In an embodiment, the plurality of mirrors includes three mirrors arranged to provide the first and second views. In an embodiment, the plurality of mirrors includes two mirrors, one of which is rotatable to alternately provide the first and second views to the camera.

[0022] In an exemplary embodiment, the microfluidic system includes a plurality of transparent channels and the camera system includes a plurality of equal pairs of first and second cameras, each of which is positioned perpendicular to one another, and each of the plurality of pairs of first and second cameras monitors material flow through a different respective one of the plurality of transparent channels.

[0023] In an embodiment, the machine learning based system identifies one or more deviations in the material flow from material flow parameters established by a user. In an embodiment, and in response to the one or more identified deviations, the machine learning based system identifies whether it is necessary to adjust the material flow parameters. In an embodiment, the machine learning based system adjusts the material flow parameters in response to a cumulative deviation exceeding a predetermined amount. In an embodiment, the machine learning based system adjusts the material flow parameters to maintain a physical property of the printed fiber within a predetermined tolerance. In an embodiment, the physical property includes a diameter of the bioprinted fiber. In an embodiment, the physical property includes a concentricity of layers within the bioprinted fiber.

[0024] In embodiments, the machine learning based system performs object detection and / or semantic segmentation of streaming images of the material stream. In embodiments, the object detection and / or semantic segmentation enables detection of the location of one or more objects within the material stream. In embodiments, the object detection and / or semantic segmentation enables visual estimation of the shape and / or size of one or more objects within the material stream. In embodiments, the object detection and / or semantic segmentation enables visual estimation of the general amount and / or distribution of biological material (e.g., cell populations) within the material stream.

[0025] In an embodiment, the microfluidic device includes a three-dimensional (3D) bioprinting print head and the system includes a 3D bioprinting system for fabricating the bioprinted fibers. In an embodiment, the 3D bioprinting print head includes a plurality of channels for selectively providing respective ones of a plurality of materials in a material stream.

[0026] In an embodiment, the at least one crosslinkable material comprises a hydrogel. In an embodiment, the material stream comprises at least one biological material, preferably the at least one biological material comprises a cell population. In an embodiment, the cell population comprises a single cell suspension, a cell aggregate, a cell spheroid, a cell organoid, or a combination thereof, or is selected from the group consisting of a single cell suspension, a cell aggregate, a cell spheroid, a cell organoid, or a combination thereof. In an embodiment, the material stream further comprises a particulate. In an embodiment, the material stream further comprises a dye, a pigment, or a colloid. In an embodiment, the presence of cells in the material stream serves as a contrast agent to facilitate measurement of physical properties of the bioprinted fiber.

[0027] In an exemplary embodiment, a biomaterial containing cells flows through each channel to produce a bioprinted fiber. In an embodiment, the bioprinted fiber is a coaxially layered hydrogel fiber. In an embodiment, the bioprinted fiber includes a core hydrogel material and a shell hydrogel material around the core hydrogel material, where the core hydrogel material is concentrically disposed within the shell hydrogel material within a predetermined tolerance.

[0028] In one embodiment, the computer system uses the results of the comparison to control material flow by adjusting the displacement of material within the microfluidic device. In an embodiment, the system further includes a displacement controller responsive to the results of the comparison to control material flow and displacement of material through the microfluidic device during printing of the printed fibers.

[0029] In another embodiment, the computer system uses the results of the comparison to control the material flow by adjusting the pressure of the material flow in the microfluidic device, In an embodiment, the system further includes a pressure controller responsive to the results of the comparison to control the material flow and pressure through the microfluidic device during printing of the printed fibers.

[0030] In an embodiment, the machine learning based system is selected from the group consisting of a convolutional neural network (CNN), a long short-term memory (LSTM) network, a recurrent neural network (RNN), a recurrent convolutional neural network (RCNN), or a combination of an RNN and a CNN. In an embodiment, the machine learning based system includes a graphics processing unit (GPU).

[0031] In embodiments, the system further includes a light emitting diode (LED) or LED array for illuminating one or more of the transparent channels. In some embodiments, the system includes one LED or LED array for each of the cameras. In some embodiments, each LED or LED array is disposed behind a respective camera. In alternative embodiments, each LED or LED array is disposed on an opposite side of the transparent channel from a respective camera.

[0032] In another aspect, the invention provides a method for monitoring material flow through a microfluidic bridge printhead, the method comprising: acquiring streaming images of the material flow through a microfluidic bridge printhead using a camera system; and determining physical properties of the printed fiber resulting from crosslinks generated by the material flow by analyzing the material flow represented in the streaming images, where determining comprises using a machine learning based system to compare the streaming images of the material flow to user established material flow parameters corresponding to the physical properties of the printed fiber within a predetermined tolerance range, and in response to determining, controlling the material flow to maintain the physical properties of the printed fiber within the predetermined tolerance range. Preferably, acquiring comprises acquiring streaming images through one or more transparent channels of the microfluidic bridge printhead.

[0033] In an embodiment, acquiring includes positioning a first camera in the camera system at a first angle relative to at least one of the one or more transparent channels and positioning a second camera in the camera system at a second, different angle relative to at least one of the one or more transparent channels, In an embodiment, positioning includes positioning the first camera and the second camera at a right angle relative to each other.

[0034] In an alternative embodiment, acquiring includes disposing a camera in a camera system to provide a first view of at least one of the one or more transparent channels, and disposing a plurality of mirrors to provide a second, different view of at least one of the one or more transparent channels. In an embodiment, the second view is orthogonal to the first view. In an embodiment, disposing includes disposing three mirrors to provide the second view. In an embodiment, disposing includes disposing two mirrors to alternately provide the first and second views to the camera, one of the mirrors being rotatable to alternately provide the first and second views to the camera.

[0035] In an embodiment, the comparing includes identifying one or more deviations in the material flow from a user-established material flow parameter. In an embodiment, the method further includes determining whether the one or more deviations in the material flow exceed a predetermined amount, and adjusting one or more of the user-established material flow parameters in response to determining to maintain a physical property of the printed fiber within a predetermined tolerance. In an embodiment, the physical property includes a diameter of the bioprinted fiber. In an embodiment, the physical property includes a concentricity of the bioprinted fiber.

[0036] In an embodiment, the determining further comprises performing object detection and / or semantic segmentation of the streaming images of the material stream using a machine learning based system. In an embodiment, the object detection and / or semantic segmentation enables detection of the location of one or more objects within the material stream. In an embodiment, the object detection and / or semantic segmentation enables visual estimation of the shape and / or size of one or more objects within the material stream. In an embodiment, the object detection and / or semantic segmentation enables visual estimation of the general amount and / or distribution of biological material (e.g., cell populations) within the material stream.

[0037] In an embodiment, acquiring includes acquiring streaming images through one or more transparent channels in a three-dimensional (3D) bioprinting printhead in the microfluidic device, the 3D bioprinting printhead producing the bioprinted fibers. In an embodiment, monitoring includes monitoring a plurality of channels in the 3D bioprinting printhead, the plurality of channels selectively providing respective ones of the plurality of materials to the material stream.

[0038] In an embodiment, the at least one crosslinkable material comprises a hydrogel. In an embodiment, the material stream further comprises at least one biological material, preferably the at least one biological material comprises a cell population. In an embodiment, the cell population comprises a single cell suspension, a cell aggregate, a cell spheroid, a cell organoid, or a combination thereof, or is selected from the group consisting of a single cell suspension, a cell aggregate, a cell spheroid, a cell organoid, or a combination thereof. In an embodiment, the material stream further comprises a particulate. In an embodiment, the material stream further comprises a dye, a pigment, or a colloid. In an embodiment, the presence of cells in the material stream serves as a contrast agent to facilitate measurement of physical properties of the bioprinted fiber.

[0039] In an exemplary embodiment, a biomaterial containing cells flows through each channel to produce a bioprinted fiber. In an embodiment, the bioprinted fiber is a coaxially layered hydrogel fiber. In an embodiment, the bioprinted fiber includes a core hydrogel material and a shell hydrogel material around the core hydrogel material, where the core hydrogel material is concentrically disposed within the shell hydrogel material within a predetermined tolerance.

[0040] In one embodiment, controlling the flow of material comprises controlling the displacement of the material within the microfluidic device, hi another embodiment, controlling the flow of material comprises controlling the pressure of the flow of material within the microfluidic device.

[0041] In an embodiment, the machine learning based system is selected from the group consisting of a convolutional neural network (CNN), a long short-term memory (LSTM) network, a recurrent neural network (RNN), a recurrent convolutional neural network (RCNN), or a combination of an RNN and a CNN. In an embodiment, the machine learning based system includes a graphics processing unit (GPU).

[0042] In embodiments, the method further includes disposing light emitting diodes (LEDs) or LED arrays to illuminate one or more of the transparent channels. In some embodiments, the method includes disposing one LED or LED array, respectively, at each of the cameras. In some embodiments, the method includes disposing each LED or LED array behind a respective camera. In alternative embodiments, the method includes disposing each LED or LED array on an opposite side of the transparent channel from a respective camera.

[0043] In an embodiment, the method further includes identifying one or more defects in the material, such as, for example, clogs and / or air bubbles, by analyzing the material flow represented in the streaming images using a machine learning based system to perform object detection and / or semantic segmentation on the streaming images.

[0044] In an embodiment, the method further includes using a machine learning based system to perform object detection and / or semantic segmentation on the streaming imagery to provide a general amount and / or distribution of one or more objects within the material stream, preferably the one or more objects include biological material, such as cells.

[0045] In an embodiment, the method further comprises analyzing whether the core hydrogel material is concentrically disposed within the shell hydrogel material.

[0046] The accompanying drawings, which are incorporated in and form a part of this specification, illustrate embodiments of the present disclosure and, together with the description, further serve to explain the principles of the disclosure and to enable those skilled in the art to make and use the disclosure. [Brief description of the drawings]

[0047] [Figure 1] A to F show cross-sectional views of different fibers.

[0048] [Figure 2A] FIG. 1 is a high-level block diagram of a system in accordance with one or more embodiments. [Figure 2B] FIG. 2 is a more detailed diagram of a camera vision system in accordance with one or more embodiments. [Figure 2C] FIG. 2 is a more detailed diagram of a camera vision system in accordance with one or more embodiments. [Figure 2D] FIG. 2 is a more detailed diagram of a camera vision system in accordance with one or more embodiments.

[0049] [Diagram 3] 3 shows an example of a display for viewing the output of the system of FIG. 2.

[0050] [Figure 4] 1A and 1B show valve configurations for the provision of materials used in producing fibers according to one or more embodiments.

[0051] [Diagram 5] 1 is a high-level flowchart of the operation of one or more embodiments.

[0052] [Figure 6] A-D show photographs and graphs related to the training of the neural network to perform the case study.

[0053] [Figure 7] A and B show the training results.

[0054] [Figure 8A] Photographs and plots related to the case studies are presented. [Figure 8B] Photographs and plots related to the case studies are presented. [Figure 8C] Photographs and plots related to the case studies are presented. [Figure 8D] Photographs and plots related to the case studies are presented. [Figure 8E] Photographs and plots related to the case studies are presented. [Figure 8F] Photographs and plots related to the case studies are presented. [Figure 8G] Photographs and plots related to the case studies are presented. [Figure 8H] Photographs and plots related to the case studies are presented.

[0055] [Figure 8I] 1 shows an exemplary image of a blockage that impedes fiber production.

[0056] [Figure 8J] 1 shows an exemplary image of air bubbles impeding fiber production.

[0057] [Figure 9A] Training results related to the case study depicted in Figures 8A-H are shown. [Figure 9B] Training results related to the case study depicted in Figures 8A-H are shown.

[0058] [Figure 10A] Photographs and plots for further case studies are provided. [Figure 10B] Photographs and plots for further case studies are provided. [Figure 10C] Photographs and plots for further case studies are provided. [Figure 10D] Photographs and plots for further case studies are provided. [Figure 10E] Photographs and plots for further case studies are provided. [Figure 10F] Photographs and plots for further case studies are provided. [Figure 10G] Photographs and plots for further case studies are provided. [Figure 10H] Photographs and plots for further case studies are provided. [Figure 10I] Photographs and plots for further case studies are provided.

[0059] [Figure 11A] 10A-10I show training results related to the case study depicted in FIG. [Figure 11B] 10A-10I show training results related to the case study depicted in FIG.

[0060] [Figure 12A] FIG. 3 is a screenshot of the display for viewing the output of the system in FIG. 2 taken at two different time points and showing controlling the shell diameter of the bioprinted fiber to a desired diameter. [Figure 12B] FIG. 3 is a screenshot of the display for viewing the output of the system in FIG. 2 taken at two different time points and showing controlling the shell diameter of the bioprinted fiber to a desired diameter.

[0061] [Figure 13] Screenshots of a video recording of the bioprinted fiber from three time points (0, 30, 90 seconds) during its printing (left) and corresponding high-level quantification of the biological material (right).

[0062] [Figure 14] 13 shows an image of a segment of a fiber containing biological material similar to that shown in FIG. 13 in the context of a portion of the entire bioprinted fiber reconstructed from a video recording of the fiber during printing.

[0063] [Figure 15] A shows an image of the core and shell, B to E show images of different core positions within the shell. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0064] Certain exemplary aspects of the systems, devices and methods according to this invention are described herein in connection with the following description and the accompanying figures. These aspects are indicative, however, of but a few of the various ways in which the principles of the invention may be employed, and the present invention is intended to include all such aspects and their equivalents. Other advantages and novel features of the present invention may become apparent from the following detailed description when considered in conjunction with the drawings.

[0065] In the following detailed description, many specific details are set forth to provide a thorough understanding of the present invention. In other instances, structures, interfaces, and processes are not described in detail so as not to unnecessarily obscure the present invention. However, it will be apparent to those skilled in the art that those specific details disclosed herein are not required to practice the present invention and do not represent limitations on the scope of the present invention, except as set forth in the claims. It is intended that no part of this specification be interpreted as resulting in a denial of any portion of the full scope of the present invention. Although specific embodiments of the present disclosure have been described, these embodiments are similarly not intended to limit the full scope of the present invention.

[0066] FIG. 2A shows a high-level block diagram of elements that can be integrated into an embodiment of the present invention. The elements of FIG. 2 can be integrated into a 3D bioprinter platform. One example is the RX1™ bioprinter from Aspect Biosystems. Looking more closely at FIG. 2A, two cameras 210, 215 forming part of a computer vision system can be positioned respectively behind and to one side of a transparent printhead nozzle 220, perpendicular to an axis 225 that passes vertically through the nozzle (axis 225 is enlarged in FIG. 2A for clarity).

[0067] As used herein, "transparent" means sufficiently translucent to allow light to pass through the microchannel and / or nozzle structure and to permit observation of the material within the microchannel and / or nozzle. In the case of a multi-layer fiber comprising a core and one or more shells, the nozzle is sufficiently transparent to permit visual distinction between the core and one or more shells.

[0068] In one embodiment, the cameras 210, 215 are positioned at a 90 degree angle relative to each other. Depending on the nozzle placement and configuration, either alone or within a bioprinting system, different angles may be acceptable or preferred. Additionally, according to different embodiments, the resolution of the cameras 210, 215 may vary. In some implementations, a resolution of 480p may be sufficient. In other implementations, higher resolution may be desirable. Higher resolutions may be anticipated in the future. Interlaced video may provide acceptable video quality in some implementations.

[0069] In one embodiment, the cameras 210, 215 may support a resolution of 4K (2160 x 3840 pixels) at 30 FPS. Other resolutions and frame rates may be appropriate. An embodiment may use an M2 lens with a focal length of 11.9 mm in an 8MP configuration (the designation "M12" refers to the size of the mount on the lens). The M12 is a type of lens that has a different focal length and may have different F-stop values. Other lenses with different mount sizes (e.g., M4-M10) may be suitable. Also, other types of mounts may be suitable, such as C-mount or CS-mount. Other configurations other than 8MP may be appropriate.

[0070] Figure 2B shows a perspective view of the placement of cameras 210, 215 relative to printhead nozzles 220, and also shows an axis 225 that passes perpendicularly through the printhead nozzles 220. Figure 2C shows a front-on view of the same placement.

[0071] In some embodiments, the cameras are positioned so that each lens can be approximately 26 mm from the center of the nozzle of the print head to optimize focus and magnification within the field of view. Different focal length lenses with different f-stops and different fields of view can allow different positioning. In one configuration, two cameras are positioned at right angles to each other on two sides of the nozzle to produce two orthogonal views of the nozzle. In one embodiment, additional lighting can be provided to illuminate the nozzle from either behind each camera or from the opposite side of each camera, or both. The lighting can include a light emitting diode (LED) or LED array 230 for the front camera 210, or an LED or LED array 235 for the side camera 215. The LED or LED array 230 can be positioned on-axis with the camera lens to project light (in one embodiment, white light) to illuminate the nozzle. The light can pass through a narrow circular or polygonal hole, or slit (not shown) to help focus the light more clearly to where imaging is to be performed. With these types of optical arrangements, the edge of the inner nozzle (the inner diameter the fiber is formed in) and the fiber being produced (shell and core, in the case of concentric fibers) can be visible in the camera view.

[0072] In one embodiment, one or more mirrors may be positioned to provide a desired view to a camera, which may be any of the example cameras 210 described herein. In FIG. 2D, light sources 270, 275, which may be LED light sources, pass light through the nozzle 220. The light source 270 provides light (dotted line) that hits a mirror 252 and is reflected off a further mirror 256 and then to a camera 260, which includes a camera structure 262 and a lens 264. The light source 275 provides light (solid line) that hits another mirror 254 and is reflected through (which may be a dichroic mirror) to reach the camera 260. In one embodiment, the mirrors and light sources are positioned to pass light at right angles through the nozzle 220.

[0073] With two orthogonal images of the printhead nozzle 220 and the fiber being produced inside the nozzle, it is possible to obtain three values: 1) With a known nozzle inner diameter and known distances from the lens and nozzle center, it is possible to measure the number of pixels within the nozzle inner diameter and therefore determine the horizontal distance each pixel represents. 2) Assuming the fiber cross-section is elliptical, it is possible to take the vector sum of the fiber diameter and / or fiber core diameter from two orthogonal views to determine upper and lower limits for the true diameter of the fiber and core. 3) Using information about the location of the left and right edges of the fiber and core, it is possible to continuously calculate the degree of concentricity of the core to the overall fiber diameter by taking the ratio of the center of the core to the overall fiber diameter. Using the same principle, it is possible to continuously calculate the concentricity of the fiber or core to the inner nozzle.

[0074] Furthermore, with a given flow rate and measured fiber diameter, it is also possible to calculate fiber velocity, which is another important feedback for optimizing the print speed: if the nozzle moves too fast or too slow relative to the stage, the fidelity of the fiber and printed structures will be adversely affected.

[0075] The machine learning module 240 receives data streams or captured images from the front camera 210 and the side camera 215. In an embodiment, the module 340 may use one or more GPU processors with multiple cores to facilitate the calculations necessary for the machine learning system to perform rapid computational analysis of the data streams or captured images, and the training of models that provide feedback for controlling material flow in the bioprinting system. In an embodiment, model training may include testing and validation to facilitate optimization and inference with the models being trained.

[0076] The machine learning module 240 may interact with a computer system (main computer) 250, which may also perform several user and system interaction functions. For user interaction, the computer system 250 may provide an appropriate graphical display and graphical user interface for interaction with various other components, such as the cameras 210, 215, the machine learning module, and the bioprinting system itself (of which the print head nozzles 250 are of course a part).

[0077] Part of the control that computer system 250 performs includes monitoring fiber concentricity and access to a control system in the bioprinting system to regulate material flow, whether by controlling pressure in the various microfluidic printhead channels described below or by controlling displacement of material through the channels. In an embodiment, the control may include switching on or off or otherwise regulating the opening and closing of pneumatic valves on the printhead. In an embodiment, part of the control that computer system 250 performs includes object detection and / or semantic segmentation. In an embodiment, object detection and / or semantic segmentation allows for a visual estimation of the general amount and / or distribution of biological material (e.g., cell populations) within the material stream.

[0078] The computer system 250 may also enable reading of printhead-specific information views of each feed from the cameras 210, 215, enable recording and / or loading of one or both of the feeds, and may adjust camera / video parameters such as contrast, color, brightness, and sharpness, among others.

[0079] FIG. 3 shows a representative image of a display according to one embodiment of the software and accompanying user interface to enable concentricity monitoring and allow the control system to adjust the microfluidic channel valves and pressures. In an embodiment, the display may show live streaming of video camera images, automated segmentation results, test measurements of fiber properties and concentricity, and / or measured parameters for recognized objects (e.g., biological materials). In other embodiments, channel conditions such as clogs or bubbles, or instability may be displayed. Some conditions may suggest corrective actions to the user, such as agitating the bioink to reduce clumping, or purging the channel to remove air bubbles. Other display possibilities may include quantitative information about fiber properties. In one embodiment, a confidence interval assessment from a machine learning system may be displayed, indicating the degree of certainty in the segmentation or classification of a particular image.

[0080] FIG. 4A shows an exemplary configuration of one type of print head 400 that can provide control over separate hydrogels containing different biomaterials or cells, thus enabling the generation of fibers of different single materials during bioprinting. The Aspect Biosystems DUO™ print head has a structure corresponding to that shown in FIG. 4A. In FIG. 4A, the different materials are provided through line 410 (connected to valve 412) and line 420 (connected to valve 422), respectively. A buffer can be provided through line 430 (connected to valve 422). A cross-linking material can be provided through line 440 (connected to valve 442). The bioprinted fibers are produced at outlet 445.

[0081] FIG. 4B shows an exemplary configuration of another type of print head 450 that can provide control over the formation of coaxial layered hydrogel fibers, where the fibers are composed of a core and a shell composed of different hydrogel materials, with or without cells, respectively. The Aspect Biosystems CENTRA™ print head has a structure corresponding to that shown in FIG. 4B. In FIG. 4B, the core material can be provided through lines 460, 465 (connected to respective valves 462, 467). The shell material can be provided through lines 470, 475 (connected to valves 472, 477). The buffer material can be provided through line 480 (connected to valve 482). The cross-linking material can be provided through line 490 (connected to valve 492). The bioprinted coaxial fiber is produced at outlet 495.

[0082] Referring back to FIG. 2A, the rear and side cameras 210, 215 may be pointed at the outlet 445 or in front of the outlet 495 of FIGS. 4A and 4B to provide video of the fiber as it is produced.

[0083] Depending on the type of material flow control being used, either a pressure controller or a displacement controller can allow for control of the flow and pressure or displacement of the biomaterial through the print head during 3D printing. With reference to Figures 4A and 4B, the pressure of the biomaterial through lines 410, 420, 460, 465, 470, and 475 can be controlled in a variety of ways, including through control of valves 412, 422, 462, 467, 472, and 477 associated with the lines.

[0084] 5 is a high-level flow chart of the implementation and use of computer vision and machine learning tools to provide automatic feedback control. At 510, the camera of the computer vision system is controlled to record video images of material passing through the print head nozzles. These images are acquired through various types of operation of the 3D biprinting system. To generate data for a training set for the machine learning model, for example, the system may be operated with known materials and parameters to obtain known results. Other data sets may also be generated for use in testing and evaluation of the trained system.

[0085] At 520, the collected data may be separated into a training set and a test / evaluation set as described above. At 530, when focusing on the training set, features of interest may be labeled to facilitate focusing on them during training. At 540, the appropriately labeled training set may be applied to a deep learning model to train the deep learning model and evolve the model to make it generate correct results from the training data. In some deep learning models, forward propagation of results may be used. In other models, back propagation may be used.

[0086] In one embodiment, for example, a convolutional neural network (CNN) having a UNET network architecture may be utilized. This type of network is known to provide favorable results when dealing with images of very complex structures with poorly defined boundaries (e.g., in the field of medical imaging). The nozzle of a bioprinter may be more limited in the ability of a camera vision system to see well through the nozzle and reach different biomaterials. Those skilled in the art will appreciate that different varieties of CNNs, such as, for example, long short-term memory (LSTM) networks or recurrent CNNs (RCNNs), may be effectively used. In one embodiment, a recurrent neural network (RNN) may be used alone or in combination with a CNN.

[0087] At 550, the training results may be analyzed to identify various features of interest, such as, for example, valve location, fiber dimensions, concentricity of the fiber core and fiber shell, etc. The results may then be classified according to the features of interest, level of accuracy, etc. At 560, a test / validation set may be used to evaluate the performance of the trained network. If the model is satisfactory at 570, then the completed trained model may be deployed at 580. If the model is not satisfactory, then the training set may be modified at 575 and control may be returned to 530 for further training. Modification of the training set may be informed by the nature of the results obtained with previous training sets.

[0088] Material flow:

[0089] Aspects of the present invention include material streams that can be used to print fiber structures for advantageous use as biomaterials. As used herein, "biomaterial" refers to natural or synthetic substances that are useful for building or replacing tissues, such as human tissues, with or without living cells. In the field of bioprinting, the term "biomaterial" is often synonymous with the term "bioink."

[0090] The material stream generally includes at least one crosslinkable material, such as, for example, hydrogels, including but not limited to, alginate, chitosan, PEGDA, PEGTA, hyaluronic acid (HA), HAMA, collagen, CollMA, gelatin, gelMA, agarose, gellan, fibrin (fibrinogen), PVA, etc., or any combination thereof, and non-hydrogels, including but not limited to, PCL, PLGA, PLA, etc., or any combination thereof. In preferred embodiments, the material stream includes at least one hydrogel. Non-limiting examples of hydrogels include alginate, agarose, collagen, fibrinogen, gelatin, chitosan, hyaluronic acid-based gels, or any combination thereof. A variety of synthetic hydrogels are known and can be used in embodiments of the systems and methods provided herein. For example, in some embodiments, one or more hydrogels form at least a portion of the structural basis of the three-dimensional structure to be printed. In some embodiments, the hydrogel has the ability to support the growth and / or proliferation of one or more cell types, which may be dispersed within the hydrogel or added to the hydrogel after it has been printed in a three-dimensional configuration.

[0091] In embodiments, the hydrogels can be crosslinked by chemical crosslinkers. For example, hydrogels containing alginate can be crosslinked in the presence of divalent cations such as calcium chloride (CaCl2), hydrogels containing chitosan can be crosslinked using polyvalent anions such as sodium tripolyphosphate (STP), hydrogels containing fibrinogen can be crosslinked in the presence of enzymes such as thrombin, and hydrogels containing collagen, gelatin, agarose, or chitosan can be crosslinked in the presence of heat or basic solutions.

[0092] In embodiments, hydrogel fibers may be produced by a precipitation reaction achieved via extraction of a solvent from an input material upon exposure of the input material to a crosslinker material that is miscible with the input material. Non-limiting examples of input materials that form fibers via a precipitation reaction include collagen and polylactic acid (PLA). Non-limiting examples of crosslinking materials that enable precipitation-mediated hydrogel fiber formation include polyethylene glycol (PEG) and alginate. Crosslinking of the hydrogel increases the hardness of the hydrogel and, in some embodiments, allows for the formation of a solidified hydrogel.

[0093] In some embodiments, the hydrogel comprises alginate. Alginate forms a solidified colloidal gel (high water content gel or hydrogel) when contacted with divalent cations. Any suitable divalent cation may be used to form a solidified hydrogel with input materials comprising alginate. In the alginate ion affinity series Cd2+>Ba2+>Cu2+>Ca2+>Ni2+>Co2+>Mn2+, Ca2+ has optimal characteristics and is most commonly used to form alginate gels (Ouwerx, C. et al., Polymer Gels and Networks, 1998, 6(5):393-408). Studies have shown that calcium alginate gels, the so-called "egg box" model, form through cooperative binding of Ca2+ ions by poly-G blocks on adjacent polymer chains (ISP Alginates, Section 3: Algin-Manufacture and Structure, in Alginates: Products for Scientific Water Control, 2000, International Specialty Products: San Diego, pp. 4-7). G-rich alginates tend to form thermally stable, strong but brittle Ca gels, whereas M-rich alginates tend to form less thermally stable, weaker but more elastic gels. In some embodiments, the hydrogel comprises depolymerized alginate.

[0094] In some embodiments, hydrogels can be crosslinked using a free radical polymerization reaction, which creates covalent bonds between molecules. Free radicals can be generated by exposing a photoinitiator to light (often UV light) or by exposing the hydrogel precursor to a chemical source of free radicals, such as ammonium persulfate (APS) or potassium peroxodisulfate (KPS) in combination with N,N,N,N-tetramethylethylenediamine (TEMED) as initiator and catalyst, respectively. Non-limiting examples of photocrosslinkable hydrogels include methacrylated hydrogels, such as hyaluronic acid methacrylate (HAMA), gelatin methacrylate (GEL-MA), or polyethylene (glycol) acrylate-based (PEG-acrylate) hydrogels, which are used in cell biology because they are inert to cells. Polyethylene glycol diacrylate (PEG-DA) is commonly used as a scaffold in tissue engineering because polymerization occurs rapidly at room temperature, requires low energy input, has a high water content, is elastic, and can be customized to include various biomolecules.

[0095] In an embodiment, the material stream includes a non-biodegradable polymer. In an example, the input material may be a synthetic polymer, such as polyvinyl acetate (PVA). In an embodiment, the material stream may include hyaluronic acid (HA).

[0096] In an embodiment, the material stream includes particulates, and as used herein, "particulates" refers to immiscible particles, typically comprised of polymers, metals, or other inorganic materials, ranging from about 0.1 μm to about 100 μm in size. Particulates may be symmetrical (e.g., spherical, cubic, etc.), although this is not a requirement. Particulates having an aspect ratio of 2:1 or greater may be considered microrods or microfibers.

[0097] Additional Ingredients:

[0098] Material streams according to embodiments of the present invention may include any of a wide variety of natural or synthetic polymers that support the viability of biological cells, including, for example, alginate, laminin, fibrin, hyaluronic acid, poly(ethylene)glycol-based gels, gelatin, chitosan, agarose, or combinations thereof. In embodiments, the subject compositions are physiologically compatible, i.e., promote cell growth, cell differentiation, and cell communication. In certain embodiments, the input material includes one or more physiological matrix materials, or combinations thereof. By "physiological matrix material" is meant biological materials found in natural mammalian tissues. Non-limiting examples of such physiological matrix materials include fibronectin, thrombospondin, glycosaminoglycans (GAGs) (e.g., hyaluronic acid, chondroitin-6-sulfate, dermatan sulfate, chondroitin-4-sulfate, or keratin sulfate), deoxyribonucleic acid (DNA), adhesive glycoproteins, and collagens (e.g., collagen I, collagen II, collagen III, collagen IV, collagen V, collagen VI, or collagen XVIII).

[0099] Collagen provides tensile strength to most tissues, with multiple collagen fibrils with diameters of about 100 nm joining together to generate strong multi-coil fibers with diameters of about 10 μm. The biomechanical function of a particular tissue construct is provided in an oriented manner through the alignment of collagen fibers. In some embodiments, the input material comprises collagen fibrils. The input material comprising collagen fibrils can be used to create a fibrous structure that is formed into a tissue construct. By adjusting the diameter of the fibrous structure, the orientation of the collagen fibrils can be controlled to direct the polymerization of the collagen fibrils in a desired manner.

[0100] For example, previous studies have shown that microfluidic channels of different diameters can direct the polymerization of collagen fibrils to form fibers oriented along the length of the channel, but only when the channel diameter is 100 μm or less (Lee et al., 2006). Primary endothelial cells grown in these oriented matrices were shown to align in the direction of the collagen fibers. In another study, Martinez et al. demonstrated that 500 μm channels in cellulose bead scaffolds can direct collagen and cell alignment (Martinez et al., 2012). By adjusting the diameter of the fibers, the orientation of collagen fibers within the fiber structure can be controlled. Thus, the fiber structure and the collagen fibers therein can be patterned to create tissue constructs with the desired configuration of collagen fibers that are essential to impart desired biomechanical properties to the 3D printed structure.

[0101] Cell populations:

[0102] In embodiments, the cell population comprises or is selected from the group consisting of single cell suspension, cell aggregate, cell spheroid, cell organoid, or combination thereof.The flow material according to embodiments of the present invention can incorporate any mammalian cell type, including but not limited to stem cells (e.g., embryonic stem cells, adult stem cells, induced pluripotent stem cells), embryonic cells, endoderm cells (e.g., lung cells, liver cells, pancreatic cells, gastrointestinal cells, or urogenital tract cells), mesoderm cells (e.g., kidney cells, bone cells, muscle cells, endothelial cells, or cardiac cells), ectoderm cells (skin cells, nervous system cells, pituitary cells, or eye cells), stem cell-derived cells, or any combination thereof.

[0103] For example, fluid materials may be derived from cells of the pancreas (alpha, beta, delta, epsilon, gamma), liver (hepatocytes, Kuppfer cells, stellate cells, sinusoidal endothelial cells, bile duct cells), thyroid gland (follicular cells), pineal gland (pineal cells), pituitary gland (growth hormone cells, lactotrophs, gonadotropes, corticotrophs, and thyrotropes), thymus (thymocytes, thymic epithelial cells, thymic stromal cells), adrenal gland (cortical cells, chromaffin cells), ovary, and the like. (granulosa cells), testis (Leydig cells), gastrointestinal tract (enteroendocrine cells - intestine, stomach, pancreas), cells from endocrine and exocrine glands including fibroblasts, chondrocytes, meniscal fibrochondrocytes, bone marrow stromal (stem) cells, embryonic stem cells, mesenchymal stem cells, induced pluripotent stem cells, differentiated stem cells, tissue-derived cells, smooth muscle cells, skeletal muscle cells, cardiac muscle cells, epithelial cells, endothelial cells, myoblasts, chondroblasts, osteoblasts, osteoclasts, and any combination thereof.

[0104] The cells can be obtained from a donor from the same species as the recipient (allogeneic), from a different species than the recipient (xenogeneic), or from the recipient (autologous). Specifically, in embodiments, the cells can be obtained from a suitable donor, such as a human or animal, or from the subject into which the cells are to be transplanted. Mammalian species include, but are not limited to, humans, monkeys, dogs, cows, horses, pigs, sheep, goats, cats, mice, rabbits, and rats. In one embodiment, the cells are human cells. In other embodiments, the cells can be xenogeneic, such as from dogs, cats, horses, monkeys, or any other mammal.

[0105] In some embodiments, at least one biomaterial comprises a cell population that expresses / secretes one or more endogenous bioactive agent(s), such as, for example, insulin, glucagon, ghrelin, pancreatic polypeptide, Factor VII, Factor VIII, Factor IX, alpha 1-antitrypsin, angiogenic factors, growth factors, hormones, antibodies, enzymes, proteins, exosomes, etc. As described herein, endogenous bioactive agents include substances that cells naturally produce in a biological context (e.g., insulin release in response to elevated glucose concentrations). Endogenous bioactive agents can constitute therapeutic agents in the context of the present disclosure.

[0106] In some embodiments, the fluid material may include genetically engineered cells that secrete a particular factor. In embodiments, it is within the scope of this disclosure that the cell populations described above may include engineered cells (e.g., genetically engineered cells) that secrete a particular factor. The cells may also be from an established cell culture line, or may be cells that have been genetically engineered and / or modified to obtain a desired genotype or phenotype. In some embodiments, tissue fragments may also be used, which may provide several different cell types within the same structure.

[0107] Genetic modification techniques applicable to the present disclosure include recombinant DNA (rDNA) technology (Stryjewska et al., Pharmacologial Reports. 2013; 65:1075), cell modification based on the use of targeted nucleases (e.g., meganucleases, zinc finger nucleases (ZFNs), transcription activator-like effector nucleases (TALENs), CRISPER-associated nuclease Cas9 (CRISPER-Cas9), and the like (Lim et al., Nature Communications. 2020; 11:4043; Stoddard BL, Structure. 2011; 19(1):7-15; Gaj et al., Trends Biotechnol. 2013; 31(7):397-405; Hsu et al., Cell. 2014; 157(6):1262; Miller et al., Nat Biotechnol. 2010;29(2):143-148), cell modification based on the use of site-specific recombination using a recombinase system (e.g., Cre-Lox) (Osborn et al., Mol Ther. 2013;21(6):1151-1159; Hockemeyer et al., Nat Biotechnol. 2009;27(9):851-857; Uhde-Stone et al., RNA. 2014;20(6):948-955; Ho et al., Nucleic Acids Res. 2015;43(3):e17; Sengupta et al., Journal of Biological Engineering. 2017;11(45):1-9), and the like. In some embodiments, some combination of the above techniques may be used for cell modification.

[0108] The disclosure encompasses modified cells capable of producing one or more therapeutic agents, including, but not limited to, a protein, a peptide, a nucleic acid (e.g., DNA, RNA, mRNA, siRNA, miRNA, nucleic acid analogs), a peptide nucleic acid, an aptamer, an antibody or fragment or portion thereof, an antigen or epitope, a hormone, a hormone antagonist, a growth factor or recombinant growth factor and fragments and variants thereof, a cytokine, an enzyme, an antibiotic or antimicrobial compound, an anti-inflammatory agent, an antifungal agent, an antiviral agent, a toxin, a prodrug, a small molecule, a drug (e.g., a drug, a dye, an amino acid, a vitamin, an antioxidant), or any combination thereof.

[0109] Working Example In the following examples, convolutional neural networks were trained to monitor and analyze different properties of the print head and biomaterial during the gelation and extrusion processes for both object detection and semantic segmentation. Several case studies were performed using the Aspect Biosystems RX1™ bioprinting platform. The examples demonstrate the use of computer vision for in-motion detection and localization and state detection of valves, segmentation of anomalies and bubbles in microchannels, and analysis of single material fiber properties, as well as analysis of more complex coaxially layered hydrogel fibers.

[0110] To aid in segmenting and identifying the flow of biomaterials through the microfluidic printhead, food dyes were used for the purposes of the following case study examples and were added to the bioink in the bioprinter to allow visual identification of material boundaries. To facilitate bioprinting and microfluidic experiments with cells and biomaterials, a bioink material suitable for cells visible under ambient lighting conditions has also been developed. This material was subsequently used to perform printing experiments with bioinks containing real cells.

[0111] In additional embodiments, it is possible to segment and identify material streams even when the material is transparent. As fibers form in the print head through cross-linking, their edges become more distinct due to differences in refractive index with the surrounding material. The edges of the fibers are imaged while illuminating a light source directly in-line, from the opposite side of the nozzle, towards a camera. Distinguishable edges are relevant features that can be used to train machine learning algorithms to identify fiber dimensions through segmentation.

[0112] From these examples and the accompanying description, the advantages of computer vision and deep learning for accurately monitoring the performance and operation of microfluidic printheads in 3D bioprinters can be appreciated. In particular, it will be appreciated that computer vision systems used with 3D bioprinters enable accurate feedback and non-contact sensing, thus enabling future opportunities for closed loop control to achieve performance optimization that would not otherwise be possible.

[0113] To develop the case studies, preliminary work was carried out to identify potential uses in the case studies and to train the neural network on the potential uses.

[0114] In the context of microfluidic-based 3D printing, in one embodiment, the valves can be used to control the flow of fluids consisting of different biomaterials or cells in the microchannels of a microfluidic printhead. This allows the use of different fluids, alone or in combination, to generate complex fiber and tissue structures from a single microfluidic device. When the valve is pressurized with air pressure, all flow is restricted through the microchannel, which corresponds to the closed state. When the valve is relaxed with air pressure, all flow is allowed through the microchannel, which corresponds to the open state. In one embodiment, the valves can be configured such that when the valve is relaxed with air pressure, the valve is closed, and when the valve is pressurized with air pressure, the valve is open.

[0115] Because microfluidic devices are typically made from transparent materials, there is a visible change in the appearance of the valve due to the expansion of the walls as they open and close. This visible change allows computer vision to detect the operational state of the valve by monitoring the physical appearance of the valve during operation.

[0116] Object detection networks can be used not only to classify different objects, but also to localize objects within a larger image. The following example shows the results of an evaluation of one embodiment of a computer vision and deep learning system for detecting and monitoring the operational state of each valve in a microfluidic printhead. For the purposes of the following example, a previously established convolutional neural network was chosen.

[0117] For real-time detection, inference speed is important. Therefore, for the purposes of the following examples, a single-shot detector (SSD) was the meta-architecture chosen. When implementing a single-shot detector network, the feature extractor may vary depending on the application and type of object that needs to be detected. Selecting the most suitable feature detector often leads to evaluating different networks for their performance.

[0118] Valve localization and state detection was achieved using an object detection convolutional neural network. Video of the Aspect Biosystems DUO™ microfluidic print head was collected on an Aspect Biosystems RX1™ bioprinter during operation using the built-in camera to generate a dataset for training and evaluation. Video was recorded at 480p and frames were extracted for labeling. Valves were individually labeled based on their location and whether they were open or closed. An example of a labeled image can be seen in Figures 6A and 6B. Figure 6A shows a bounding box around the valves in the print head. Figure 6B shows the same bounding box with the open and closed states of the valves further identified. Figures 6C and 6D show the running estimates of the operational states of valves 1 and 2, respectively, during the print session. In this example, valve 1 was open for approximately 3 seconds to generate fibers. Valve 2 remained closed.

[0119] Data augmentation was performed by introducing randomized contrast, brightness, and reflectance to increase the size and variety of images used for training. The resulting images were split into two datasets. The first training dataset with approximately 1500 images was used to train the object detection network. The second training set with approximately 250 images was a validation dataset used to evaluate the performance of the trained network before deployment.

[0120] Three single-shot detector (SSD) neural networks, namely SSD-MobilenetV2, SSD-InceptionV2, and SSD-ResNet50, were trained to determine the most suitable one for deployment. To aid in training, pre-trained parameter weights on common objects in the context (COCO) dataset were used as initial training points. The training loss function consisted of two components. The first component was a Smooth L1 localization loss to quantify the localization error between the predicted exposed boxes and the ground truth boundary boxes. This can be seen in equation (1) and equation (2).

number

[0121] In these equations, x t corresponds to the ground truth bounding box min / max coordinates for a particular object, and x p corresponds to the min / max coordinates of the bounding box predicted from the SSD network.

[0122] The second component was a weighted focal loss to quantify the error on the class prediction corresponding to the proposed bounding box. The equation for the weighted focal loss can be seen in equation (3).

number

[0123] In equation (3), p t corresponds to the SSD network output of the correct class. γ is the modulation coefficient (1-p t ) is used to control α t is a class weighting scale defined by the user to emphasize the detection of a particular class. For training, γ is set to 2 and α twas set to 0.75 for the positive class (i.e., open and closed valve states) and 0.25 for the negative class (i.e., background class prediction). The final loss, which utilizes both the localization and classification loss functions, can be seen in equation (4).

number

[0124] Training of the networks was performed using the Adam optimizer as described in Kingma, Diederik & Ba, Jimmy, Adam: adam: A Method for Stochastic Optimization, International Conference on Learning Representations (2014), incorporated herein by reference. All three of the above neural networks were evaluated by determining their classification and localization accuracy on the validation database. Classification accuracy was determined using Equation (5).

number

[0125] Accuracy is based on true positives (TP), true negatives (TN), false positives (FP), and false negatives (FN) for each class. Localization accuracy was determined by calculating the Intersection over Union (IoU) score for correct predictions. IoU is the sum of the predicted bounding box (x p ) and the ground truth bounding box (x t ) divided by the area of ​​their union.

number

[0126] The loss used for training was a pixel-wise entropy loss function. To calculate the loss, the final activations from the UNET network were converted into probability scores corresponding to each class. This was done using the softmax function, which can be seen in equation (7) for a network identifying K classes.

number

[0127] In equation (7), p t is the activation value of the t-th class for a particular pixel in the image. A softmax function is used to process the activation values ​​from the UNET network for each pixel.

[0128] The classification cross entropy was calculated as shown in equation (8).

number

[0129] In equation (8), p t is the true class activation value for a particular pixel. The activation values ​​of other classes are not taken into account. α p are the class weights for rescaling the loss to penalize misclassification of a particular class. The weights used for training were 1, 1.235, and 1.35 for background, fibers, and bubbles, respectively. Cross entropy was used to quantify the error between the true class and the predicted class for every pixel comprising the image. The Adam optimizer described above was used to train the network and minimize the loss function.

[0130] The trained UNET network was evaluated on the validation dataset by computing both the average IoU and average FI scores over all classes. The IoU score was calculated via equation (5), and the FI score was calculated via equation (9).

number

[0131] Precision and recall were calculated for each class as shown in equation (10) and equation (11).

number

[0132] All three networks were trained on the same dataset and then examined on a validation dataset to evaluate their performance and determine which one was most suitable for deployment.

[0133] The training results, classification accuracy for both valve states, and the average IoU score for correct classification can be seen in Figures 7A and 7B. A classification was considered correct if the IoU score was 0.5 or higher. The classification accuracy for localized and open valves was very similar for all three networks. However, SSD-ResNet50 had the best accuracy when classifying the closed valve state. SSD-ResNet50 was also able to minimize the loss function without overfitting the best as seen by the training curves. Therefore, based on the results, it was determined that the SSD-ResNet50 network had the best performance out of the three. Other non-limiting examples of CNNs that may be used herein include ImageNet, COCO, Cityscapes, PASCAL VOC, and ADE20K, as well as MSRF-Net, UACANet-L, ResUNet+++TTA, UNETR, SwinUnet, Unet+, DC-UNET, and KiU-Net.

[0134] Example 1 - Single Material Fiber In this example, a 3D bioprinting system using an Aspect Biosystems DUO™ microfluidic printhead was used.

[0135] To perform single material fiber analysis and anomaly detection, a semantic segmentation network was utilized to localize the flow of biomaterial and air bubbles within the microfluidic print head during operation. The segmentation network used for this case study was the UNET network. The dataset for fiber analysis and anomaly detection was created using the same videos and images captured for valve state detection, as described above with reference to Figures 6A-6D. However, in this case, pixels were labeled if they corresponded to a specific biomaterial or air bubble within the print head. The remainder of the pixels were labeled as background. Figure 8A shows an example of a labeled image showing biomaterial 810 or air bubbles 820 within a microchannel. Figure 8B shows a close-up view of the print head extrusion region 830 of Figure 8A, indicated by the bounding box 835. Figure 8C shows an approach to calculate fiber diameter by estimating the location of the extruded fiber edge.

[0136] The same videos collected to perform the valve detection and condition monitoring described above with respect to Figures 6A-6D were used to examine the performance of the proposed computer vision system for fiber analysis and anomaly detection. The videos were processed at 30 FPS and a running estimate of the extruded fiber diameter, as well as the detection of any air bubbles or anomalies within the print head microchannels, were recorded for analysis and evaluation.

[0137] The images were enhanced using randomized reflectance, brightness, and contrast, and the resulting enhanced images were split into training and validation datasets of approximately 900 and 150 images, respectively.

[0138] FIG. 8D shows two valves. In this particular example, only one valve was open to provide biomaterial from the microchannel associated with that valve. The other valve was left closed since the material in that valve's corresponding microchannel was not being used. FIG. 8D shows a segmentation from one frame captured during printing. The frame shows the same structure as in FIG. 6A and FIG. 6B. FIG. 8D shows biomaterial 850. There is an extrusion region 860 with a bounding box 865 drawn around the extrusion region 860. FIG. 8E shows a close-up of the extrusion region 860 within the bounding box of FIG. 8D.

[0139] Figure 8F shows a running estimate of the extruded fiber diameter during operation of the system. Corresponding to Figure 6C, Figure 8F shows the fiber diameter during a period of approximately 3 seconds when the valve controlling the flow of biomaterial is open.

[0140] Using the results of the biomaterial segmentation, the geometric properties of the extruded fibers, including the diameter, were monitored and determined. This process was accomplished by estimating the edge boundaries of the segmented biomaterial within the extrusion region. Once the edge boundaries were determined, the fiber diameter was calculated based on the average distance between the fiber edges. The estimate was then converted from pixels to microns using a pre-calibrated gain based on the position of the camera relative to the printhead.

[0141] During printing, the biomaterial fiber was extruded for approximately 3 seconds. Using the segmentation results in the extruded region, the fiber diameter was estimated to be approximately 4 pixels. To evaluate the accuracy and effectiveness of the computer vision system for fiber analysis, the printed fibers were measured under a microscope after each printing session. The measured fiber diameter was then compared to the diameter estimated from the computer vision system. The fiber diameter was estimated with high accuracy.

[0142] The diameter of the extruded fiber is a very important property that directly impacts the final quality of the printed fiber and therefore must be monitored carefully. Instability and drift in fiber diameter can lead to poor print quality, unpredictable results, and unnecessary sample-to-sample variability. Using one embodiment of the computer vision system of the present invention, the end-to-end latency of the computer vision system ranged from 23-28 ms, enabling the kind of high frame rates required for real-time analysis.

[0143] In addition to the presence and flow of biomaterial within the microfluidic printhead, other factors such as the presence of foreign objects or anomalies within the microfluidic channels of the printhead can affect the final quality of the printed structures. Clumps of cells and biomaterial can block channels in the printhead and thus restrict fluid flow. Bubbles that arise due to oil-soluble gases within the biomaterial and cells can also disrupt the bioink during extrusion. In many cases, these bubbles tend to remain in the inactive channels. However, in some cases, the bubbles may nucleate and mix with the bioink in the active channels. FIG. 8G shows an example of an air bubble 875 that disrupts the biomaterial 870 and thus the generation of fibers. FIG. 8H is a close-up of the bubble concentration in FIG. 8G.

[0144] When this bubble blockage occurs, the consistency of the printed biomaterial is significantly affected and thus may negatively impact the final quality of the tissue. In most cases, the bubbles move more quickly and in any case are difficult to distinguish visually, making it impossible to recognize with the naked eye when the bubbles are nucleating and affecting the generated fibers. However, the segmentation network allows for easy identification and localization of the bubbles in the live camera feed, allowing for quality control for improved operation and reliability. FIG. 8I shows an additional exemplary back-illuminated visible light (left) and corresponding colored overlay (mask, right) image of a clog that blocks fiber generation. FIG. 8J shows an additional exemplary back-illuminated visible light (left) and corresponding colored overlay (mask, right) image of a bubble that blocks fiber generation. In an embodiment, the machine learning-based system disclosed herein can identify clogs, bubbles, and other types of print defects or print failures in response to images provided by one or more cameras described herein, as illustrated in FIGS. 8I and 8J.

[0145] Figure 9A shows the training loss decreasing over time. Figure 9B shows the IoU and F1 scores comparing the training and validation datasets used, showing scores above 98%, indicating satisfactory performance of the trained network. Class weighting was necessary to compensate for significant class imbalance during training due to the large number of background pixels in all of the images.

[0146] Example 2: Coaxial Layered Hydrogel Fibers Coaxial layered fibers may be part of a complex biological tissue and may consist of separate regions containing different biomaterials, cells and growth factors, placing stringent requirements on the structural geometry to ensure acceptable final tissue quality and function.

[0147] In this example, the UNET network was also used to segment and analyze the geometric properties of more complex coaxially layered fibers generated from a 3D bioprinting system using the Aspect Biosystems CENTRA™ microfluidic printhead, which allows for coaxially layered or hollow perfused fiber formation, as used in the previous case study. Such fibers allow for the 3D patterning of tissues with integrated perfusable vasculature, and also allow for engineered isolation of core fiber cells from the external environment.

[0148] In this example, a coaxially layered fibrinogen solution dyed red was printed and the resulting extruded fibers were analyzed at 15 Hz. The end-to-end latency of the computer vision system ranged from 14 to 16 ms. This low latency was made possible by using inference acceleration libraries and specialized GPU-accelerated hardware for deep learning, such as Nvidia's TensorRT library, exemplified by the Nvidia Jetson Xavier system.

[0149] By simultaneously processing orthogonal projections of the nozzle, cross-sectional profiles of the bioprinted fibers could be generated to provide good visualization of the extruded fibers in real time. The cross-sectional profiles were used to qualitatively analyze the axial symmetry of the fibers as well as to identify any potential defects or inconsistencies that may lead to poor structural fidelity and inhomogeneity among multiple printed samples.

[0150] The diameter and layer thickness of the shell and core, as well as the concentricity of the core inside the shell, are very important to achieve the desired biological function, such as controlled perfusion when printing coaxially layered biomaterials. The described computer vision system according to the embodiment can be fully integrated into the 3D printing platform for real-time analysis of various geometric properties of the extruded fibers to detect any inconsistencies and malfunctions that may affect the final quality of the tissue. Furthermore, the computer vision system allows for non-contact feedback, again allowing the opportunity to develop closed-loop controllers that would otherwise be impossible to achieve.

[0151] In this example, a 1080p video was recorded of the nozzles on the print head while the fibers were being formed and extruded. Images were extracted from the video and pixels were labeled as corresponding to the core material, the shell material, or the background. The images were then cropped to create a 256x256 pixel image focused around the fibers. The images were again enhanced through randomized brightness, contrast, and reflectance. A training set of approximately 2000 images and a validation data set of approximately 500 images were obtained.

[0152] 10A and 10B show images of a print head nozzle 1000 having a coaxial layered hydrogel fiber 1010. FIG. 10C highlights the shell biomaterial 1014 and the core biomaterial 1016 within the fiber 1010. FIG. 10D shows the various parameters involved in calculating the degree of misalignment or non-concentricity between the central axis of the core and the central axis of the shell in a coaxial bioprinted fiber. The calculation takes into account, among other things, the diameter of the core and the diameter of the shell, as well as the distance on either side of the circumference of the shell.

[0153] The loss function for training was the same pixel-wise classification cross-entropy function shown in equation (6). A softmax function as shown in equation (5) was also used to process the activation output from the UNET network. The class weights implemented for training were 1, 1.15, and 1.25 for background, shell material, and core material, respectively. Again, the Adam optimizer referenced above was used to train the network.

[0154] From the segmentation results, the layer thicknesses of the shell and core, as well as the alignment of the core within the shell of the fiber, were determined by estimating the edge boundaries of the shell and core through analysis of the segmentation results. Using the edge boundaries, the layer thicknesses of the different materials comprising the extruded fiber were determined as shown in FIG. 10D. The alignment of the core, λ, was also calculated using Equation (12).

number

[0155] In Equation 12, Δ1 and Δ2 correspond to the parameters shown in Figure 10D. A value of 0 corresponds to perfect alignment with the specified viewing plane, while positive and negative values ​​correspond to misalignment to the right and left, respectively.

[0156] To demonstrate potential integration into a portable platform, the proposed computer vision system was deployed using an Nvidia Jetson Xavier AGX system as the machine learning engine 240 in Figure 2. While deployed on the Nvidia system, the trained network was optimized using Nvidia's TensorRT software development kit (SDK) to reduce network latency and improve throughput. Inference and post-processing of the segmentation results were performed exclusively on the Nvidia system. Commands via the main computer 250 were sent to the Nvidia system. Data on the geometric properties of the extruded fiber were sent back to the main computer via bidirectional Transmission Control Protocol (TCP) communication shown in Figure 2. Two cameras (210 and 215 in Figure 2) were positioned at right angles to each other around the extrusion nozzle 220. The camera feeds around the extrusion nozzle were collected through USB communication with the Nvidia system and analyzed simultaneously. The analysis was performed at 15 FPS. Using geometric properties calculated from both camera feeds, the axial symmetry and core concentricity of the fiber were visualized in real time for better insight into the overall structural fidelity of the fiber.

[0157] Figure 10E shows an example of the segmentation results for one of the captured images. Using the segmentation results, the boundary edges of the shell and core materials were located as seen in Figure 10F. Despite the fact that the regions are not easily separable by the naked eye, the segmentation network is able to distinguish between the shell and core materials and identify their boundaries with high accuracy and confidence.

[0158] Figures 10G and 10H show current estimates of layer thickness and core alignment of the extruded fiber, with Figure 10G showing diameter as a function of time and Figure 10H showing elapsed time. Figure 10I shows the generated cross-sectional profile of the extruded fiber at different stages during operation using the geometric characteristics shown in Figure 10G. Figure 10I shows the progressive circularity of the core and shell cross-sections as well as the progressive concentricity of the core and shell.

[0159] Figure 11A shows the training loss decreasing over time. Figure 11B shows the IoU and F1 scores comparing the training and validation datasets used, showing scores above 85%, indicating satisfactory performance of the trained network. As mentioned above and as in Example 1, class weighting was necessary to compensate for class imbalance during training due to the large proportion of background pixels in the images.

[0160] Example 3: Feedback control of fiber diameter This example demonstrates feedback control of a bioprinted fiber comprising a core and a shell, where the diameter of the shell is adjusted in real time to form a bioprinted fiber with a desired shell diameter.

[0161] At the beginning of the bioprinting process, the shell outer diameter (OD) and core inner diameter (ID) were set to 1.0 mm and 0.5 mm, respectively. Figure 12A shows that the machine learning based system disclosed herein detected that the shell diameter was 0.80 mm and the shell channel pressure was 149 mBar. Figure 12B shows that the shell channel pressure was automatically adjusted and slowly increased to 237 mBar to increase the shell diameter to 1.02 mm over 13 seconds.

[0162] Example 4: Cell mass analysis This example demonstrates the ability to monitor approximate cell quantity and / or location in real time during printing of a biofiber via the systems and methods disclosed herein, enabling qualitative whole-fiber analysis.

[0163] FIG. 13 shows three images and corresponding high-level quantification of cells within a bioprinted fiber during printing. Each of the images is a representative screenshot taken from a 2-minute video of the bioprinting process at a selected time point (0 s, 30 s, 90 s). As can be seen from the images and corresponding cell quantification, the total amount of cell content may vary over the course of the printing process as a function of various factors. For example, the cell amount is lower at 30 s into the printing process compared to the amount observed at 0 s and 90 s. For each of the graphs on the right side of FIG. 13, the fiber diameter is shown. Since closed-loop control was not utilized to control the fiber diameter, the fiber diameter varies in this experiment. The x-axis in FIG. 13 corresponds to the column number, with each column being one pixel wide. The y-axis represents the sum of the pixels in each column where cellular material is identified. The light orange color in the graphs indicates the fiber diameter.

[0164] The method illustrated in FIG. 13 allows for general qualitative analysis of entire fibers, including cellular material. For example, object detection and / or semantic segmentation implemented in a machine learning system according to an embodiment allows for the identification of the proximity of cellular material to the fiber edge, thereby providing an indication of how centered the cellular material is within the fiber. Fibers with cellular material too close to the fiber edge (i.e., cellular material butting against the fiber edge) may be rejected for use due to potential immune system recognition of the cellular material. Additionally or alternatively, object detection and / or semantic segmentation implemented in a machine learning system of the present disclosure allows for the identification of fibers with reduced cellular content in the form of low volume through the fiber, gaps, or other significant variations in the cellular material. In an embodiment, object detection and / or semantic segmentation as implemented in a machine learning system according to an embodiment allows for the identification of higher quality fibers, such as fibers with more consistent cellular content in terms of, for example, location (i.e., how centered the cellular material is relative to the fiber) and / or cell volume throughout the fiber (i.e., substantial lack of gaps or other variations).

[0165] FIG. 14 shows an exemplary back-illuminated visible (left) image of a segment of a bioprinted fiber during printing, and additional qualitative analysis with a corresponding colored overlay (mask, right) image (top), with the segment indicated by a dashed box (bottom). As seen in the bottom image, the cellular content is substantially consistent throughout the fiber portion shown. In one embodiment, object detection and / or semantic segmentation implemented in a machine learning system according to one embodiment also enables this type of qualitative analysis.

[0166] FIG. 15A shows an image of the core and shell from one camera's field of view, according to one embodiment. The image shows the core centered within the shell along one axis. In one embodiment, a corresponding image from another camera's field of view may show the core relative to the shell along another axis, allowing for determination of the concentricity of the core within the shell. FIGS. 15B-15E are images of different core positions within the shell, taken at successive times as material (such as bioink) flows through the print head. Successive images, or in fact a stream of images, can show cell tracking as a velocity vector field. This approach can provide flow estimates from one or more of the cameras, which can be used to gauge fiber quality print time. This flow estimate can be used to ensure correct flow (e.g., flow in the right direction, or flow that is not stopped for any reason). In an embodiment, the flow estimate can be used to help determine if there is homogenous flow (which may be reflected from the core and shell images), to help center the cellular material within the fiber, and / or to help estimate the print flow overall.

[0167] All patent and non-patent references cited herein are expressly incorporated herein by reference in their entirety for all purposes.

Claims

1. 1. A microfluidic bridge printhead material flow sensing system comprising: a microfluidic bridge printhead; a camera system for monitoring a material flow through the microfluidic cross-linking printhead and providing streaming images of the material flow, the material flow comprising at least one cross-linkable material, preferably the at least one cross-linkable material comprising a hydrogel; and a computer system for determining physical properties of printed fibers resulting from crosslinks created by the material streams by analyzing the material streams represented in the streaming images; and Equipped with the computer system comprising a machine learning based system that compares the streaming images of the material stream to user-established material flow parameters corresponding to the physical properties of printed fibers within predetermined tolerances and records the material flow parameters of the material stream and the results of the comparison.

5. The microfluidic bridge printhead material flow sensing system.

2. 10. The system of claim 1, wherein the microfluidic bridge print head comprises one or more transparent channels, and the camera system monitors material flow through at least one of the one or more transparent channels, preferably wherein the microfluidic bridge print head comprises a transparent nozzle or dispensing channel.

3. (i) the camera system comprises a first camera positioned at a first angle relative to at least one of the one or more transparent channels and a second camera positioned at a second, different angle relative to at least one of the one or more transparent channels, preferably the first camera and the second camera being perpendicular to each other, and / or the microfluidic system comprises a plurality of transparent channels and the camera system comprises a plurality of equal pairs of first and second cameras, each first and second camera in each pair being positioned at right angles to each other, each of the plurality of pairs of first and second cameras monitoring material flow through a different respective one of the plurality of transparent channels, or 2. The system of claim 1, wherein the camera system comprises a camera and a plurality of mirrors, the mirrors arranged to provide a first view and a second distinct view of the at least one of the one or more transparent channels, the camera receiving images of the first and second views, preferably the second view being orthogonal to the first view, and / or the plurality of mirrors comprising three mirrors arranged to provide the first and second views.

4. (i) the machine learning based system identifies one or more deviations of the material flow from material flow parameters established by the user, and preferably, in response to the one or more identified deviations, the machine learning based system identifies whether the material flow parameters need to be adjusted, and / or, the machine learning based system adjusts the material flow parameters in response to accumulated deviations exceeding a predetermined amount. (ii) further comprising adjusting the material flow parameters to maintain physical properties of the printed fibers within the predetermined tolerances; and / or (iii) The system of claim 1, wherein the machine learning based system performs object detection and / or semantic segmentation of the streaming images of the material flow, preferably wherein the object detection and / or the semantic segmentation enables detection of the position of one or more objects within the material flow, preferably wherein the object detection and / or the semantic segmentation enables visual estimation of the shape, size, and / or quantity of the one or more objects within the material flow.

5. 10. The system of claim 1, wherein the microfluidic bridge printhead comprises a three-dimensional (3D) bioprinting printhead, and the system comprises a 3D bioprinting system for producing bioprinted fibers. (i) the 3D bioprinting print head comprises a plurality of channels for selectively providing respective plurality of materials to the material stream; (ii) the physical property comprises a diameter of the bioprinted fiber; (iii) the physical properties include the concentricity of the bioprinted fiber; (iv) the bioprinted fiber is a coaxial layered hydrogel fiber; (v) the bioprinted fiber comprises a core hydrogel material and a shell hydrogel material surrounding the core hydrogel material, the core hydrogel material being concentrically disposed within the shell hydrogel material within a predetermined tolerance; and / or (vi) the presence of cells within the material stream acts as a contrast agent to facilitate measurement of physical properties of the bioprinted fiber.

7. The system of claim 1 , wherein the material stream further comprises at least one biological material, preferably wherein the at least one biological material comprises a cell population. (i) the cell population comprises a single cell suspension, a cell aggregate, a cell spheroid, a cell organoid, or a combination thereof, or is selected from the group consisting of a single cell suspension, a cell aggregate, a cell spheroid, a cell organoid, or a combination thereof; (ii) the material stream further comprises particulates; (iii) the material stream further comprises a dye, pigment, or colloid; (iv) flowing the cell-laden biomaterial through the respective channels to produce the bioprinted fiber; (v) the bioprinted fiber is a coaxial layered hydrogel fiber; (vi) the bioprinted fiber comprises a core hydrogel material and a shell hydrogel material surrounding the core hydrogel material, the core hydrogel material being concentrically disposed within the shell hydrogel material within a predetermined tolerance; and / or (vii) the presence of cells within the material stream acts as a contrast agent to facilitate measurement of physical properties of the bioprinted fiber. (i) the computer system uses the results of the comparison to control the material flow by adjusting the displacement of material within the microfluidic bridge printhead. (ii) further comprising a displacement controller responsive to the result of the comparison to control the material flow and material displacement through the microfluidic bridge printhead during printing of the printed fiber; (iii) the computer system uses the results of the comparison to control the material flow by adjusting the pressure of the material flow within the microfluidic bridge printhead. (iv) further comprising a pressure controller responsive to the result of the comparison to control the material flow and pressure through the microfluidic bridge printhead during printing of the printed fiber. (v) the machine learning based system is selected from the group consisting of a convolutional neural network (CNN), a long short-term memory (LSTM) network, a recurrent neural network (RNN), a recurrent convolutional neural network (RCNN), or a combination of RNN and CNN; and / or (vi) The system of claim 1, wherein the machine learning-based system comprises a graphics processing unit (GPU).

10. (i) further comprising light emitting diodes (LEDs) or LED arrays for illuminating one or more of said transparent channels, preferably one LED or LED array for each of said cameras, preferably each LED or LED array being located behind a respective camera, or each LED or LED array being located behind a respective camera, or 4. The system of claim 3, wherein (ii) the plurality of mirrors includes two mirrors, one of the mirrors being rotatable to alternately present the first and second views to the camera.

11. 1. A method for monitoring material flow through a microfluidic bridge printhead, comprising: acquiring streaming images of a material flow through a microfluidic cross-linking printhead using a camera system, the material flow comprising at least one cross-linkable material, the at least one cross-linkable material comprising a hydrogel; determining physical properties of the printed fiber resulting from crosslinks created by the material flow by analyzing the material flow represented in the streaming images, wherein determining includes using a machine learning based system to compare the streaming images of the material flow to user-established material flow parameters corresponding to the physical properties of the printed fiber within predetermined tolerances; The method comprising:

12. The method of claim 11 , wherein said acquiring comprises acquiring the streaming image through one or more transparent channels of the microfluidic bridge-printhead.

13. 12. The method of claim 11, wherein said acquiring comprises positioning a first camera within said camera system at a first angle relative to said at least one of said one or more transparent channels and positioning a second camera within said camera system at a second, different angle relative to said at least one of said one or more transparent channels, preferably said positioning comprising positioning said first camera and said second camera at a right angle to each other.

14. 12. The method of claim 11, wherein said acquiring comprises: positioning a camera in said camera system to provide a first view of said at least one of said one or more transparent channels; and positioning a plurality of mirrors to provide a second, different view of said at least one of said one or more transparent channels, preferably said second view being orthogonal to said first view.

15. (i) said positioning includes positioning three mirrors to provide said second view. (ii) further comprising arranging light emitting diodes (LEDs) or LED arrays to illuminate one or more of said transparent channels, preferably comprising one LED or LED array for each of said cameras, preferably comprising each LED or LED array behind its respective camera, preferably comprising each LED or LED array on an opposite side of the transparent channel from its respective camera; or 15. The method of claim 14, wherein (iii) said positioning comprises positioning two mirrors to alternately provide the first and second views to the camera, one of the mirrors being rotatable to alternately provide the first and second views to the camera.

16. (i) said comparing includes identifying one or more deviations of said material flow from material flow parameters established by said user, preferably further including determining whether said one or more deviations of said material flow exceed a predetermined amount, and adjusting one or more of said material flow parameters established by said user in response to said determining to maintain said physical properties of said printed fiber within said predetermined tolerance range; (ii) said determining further comprises using said machine learning based system to perform object detection and / or semantic segmentation of said streaming images of said material stream, preferably said object detection and / or said semantic segmentation enabling detection of the position of one or more objects within said material stream, and / or said object detection and / or said semantic segmentation enabling visual estimation of the shape, size, and / or quantity of said one or more objects within said material stream; (iii) the microfluidic bridge printhead comprises a three-dimensional (3D) bioprinting printhead, and the acquiring comprises acquiring streaming images through one or more transparent channels in the 3D bioprinting printhead, the 3D bioprinting printhead producing bioprinted fibers, and preferably the monitoring comprises monitoring a plurality of channels in the 3D bioprinting printhead, the plurality of channels selectively providing respective plurality of materials to the material stream; and / or (vi) the physical property comprises concentricity of the bioprinted fiber.

17. The method of claim 11 , wherein the material stream further comprises at least one biological material, preferably wherein the at least one biological material comprises a cell population.

18. (i) The cell population comprises a single cell suspension, a cell aggregate, a cell spheroid, a cell organoid, and / or a microparticle, or is selected from the group consisting of a single cell suspension, a cell aggregate, a cell spheroid, a cell organoid, and / or a microparticle. (ii) the material in the material stream further comprises a dye, pigment, or colloid; (iii) the bioprinted fiber is a coaxial layered hydrogel fiber; (vi) the bioprinted fiber comprises a core hydrogel material and a shell hydrogel material surrounding the core hydrogel material, the core hydrogel material being concentrically disposed within the shell hydrogel material within a predetermined tolerance; and / or 20. The method of claim 17, wherein (v) the presence of cells within the material stream acts as a contrast agent to facilitate measurement of physical properties of the bioprinted fiber.

19. 12. The method of claim 11 , further comprising, in response to determining, controlling the material flow to maintain the physical properties of the printed fiber within the predetermined tolerance, preferably wherein controlling the material flow comprises controlling a displacement of material within the microfluidic device or wherein controlling the material flow comprises controlling a pressure of the material flow within the microfluidic device.

20. (i) The machine learning based system is selected from the group consisting of a convolutional neural network (CNN), a long short-term memory (LSTM) network, a recurrent neural network (RNN), a recurrent convolutional neural network (RCNN), or a combination of an RNN and a CNN. (ii) the machine learning-based system comprises a graphics processing unit (GPU); (iii) identifying one or more defects in the material flow by analyzing the material flow represented in the streaming images using the machine learning based system to perform object detection and / or semantic segmentation on the streaming images, preferably wherein the one or more defects are clogs and / or bubbles; (vi) providing a general amount and / or distribution of one or more objects within the material stream, preferably wherein the one or more objects comprise biological material; and / or 12. The method of claim 11, further comprising: (v) analyzing whether the core hydrogel material is concentrically arranged within the shell hydrogel material.

Images