Systems and methods for stitching frames in a video captured in bioprinting

Abstract

In a bioprinting system and method, a bioprinter may include one or more reservoirs, a printhead, and a print surface on which the bioprinting is dispensed. An optical system may collect imaging data of a plurality of cells within a material flow while passing through and / or exiting from the bioprinting system, wherein the passage of the plurality of cells through the bioprinting system creates a printed fiberduring and / or after. A computing system may identify a first frame in the imaging data captured at time t-n and a second frame in the imaging data captured at time t, generating, via a machine learning algorithm executing on the imaging data, a plurality of segmented regions in the imaging data, wherein each segmented region in the plurality of segmented region defines a boundary around a different set of cell aggregates or cells in the plurality of cells, determining, for combining with the first frame and based on the plurality of segmented regions, a portion of the second frame that is substantially non-overlapping with the first frame, determining, based on the plurality of segmented regions, one or more cells or cell aggregates in the plurality of cells that are located along an edge of the first frame or an edge of the portion of the second frame, identifying a seam where the first frame and the portion of the second frame are to be combined based at least on locations of the one or more cells or cell aggregates, combining the first frame and the portion of the second frame along the seam to generate at least a portion of a stitched image of the printed fiber, and determining one or more parameters associated with the plurality of cells, the material flow, or the bioprinting system based on the stitched image.

Description

SYSTEMS AND METHODS FOR STITCHING FRAMES IN A VIDEO CAPTURED IN BIOPRINTINGCROSS-REFERENCE TO RELATED APPLICATIONS

[0001] This application claims priority to U.S. Provisional Patent Application No. 63 / 690,779, filed September 4, 2024, the contents of which are incorporated by reference herein.TECHNICAL FIELD OF DISCLOSURE

[0002] Aspects of the present invention relate to quality control methods and apparatus for monitoring a three-dimensional (3D) bioprinting system. More particular aspects relate to methods and apparatus for using machine learning approaches to analyze a video of bioprinted materials and stitching frames in a video captured during bioprinting. The stitched video can be used to determine characteristics of the bioprinted materials, such as a quantity of cells within the bioprinted materials.BACKGROUND

[0003] Accurately assessing cellular characteristics of a bioprinted material is critical, especially when the bioprinted material is to be used for therapeutic purposes. For example, an accurate assessment of cell quantity within the bioprinted fiber is needed for a proper therapeutic dose determination. In some approaches, videos captured during the bioprinting process can be used for assessing cellular characteristics of the bioprinted material. However, in such approaches, using the raw video often results in inaccurate assessments of cell characteristics due to low quality or inaccuracies in the imagery.SUMMARY

[0004] The present disclosure addresses the foregoing problems in the art by integrating computer vision and deep learning into a three-dimensional (3D) bioprinting system, thereby enabling an accurate assessment of characteristics of a plurality of cells within a material flow during and / or after bioprinting. In embodiments, the materials may include, e.g., cell-laden hydrogels and other11105427751\1\AMERICAScross-linkable materials. In embodiments, the different stages of the bioprinting system may include one or more reservoirs, a printhead, and a print surface.

[0005] Embodiments of the invention enable contactless sensing of the characteristics of a plurality of cells as they flow through and / or exit the bioprinting system, during and / or after printing. Embodiments also enable automated analyses, which in turn enable derivation of quantitative measures of properties of the plurality of cells. These measures can serve as quality control and / or quality assurance parameters. In some embodiments, such quantitative measures include real-time quantitation of biological material (e.g., cells) during and / or after printing thereof, for qualitative assessment of cell quality in terms of the consistency of biological material. In some embodiments, such quantitative measures include real-time quantitation of other objects within a material flow.

[0006] In various embodiments, a machine learning system segments image frames in the imaging data captured by the bioprinting system into segmented regions. The segmented representation of a given frame identifies, for each pixel in the image frame, a segmented region with which the pixel is associated. The segmented representation may identify segmented regions corresponding to individual cells, fragments, or cell aggregates within a material flow or a printed fiber. In certain situations, objects captured in an image frame, such as individual cells, fragments, or cell aggregates, may be overlapping, such that a given object may be partially or fully occluded by another object. The machine learning system detects occluded objects in the imaging data and then extrapolates boundaries around the occluded portions based on the boundary of the segmented region corresponding to the non-occluded portions. Such extrapolation allows for a more accurate segmented representation of objects, e.g., cell aggregates, in an image frame.

[0007] In various embodiments, the de-occluded segmented representation is used to determine one or more characteristics associated with the biological material (e.g., cells, fragment, and / or cell aggregates) within a material flow or other characteristics of the bioprinted material. These characteristics include, but are not limited to, a cell count (individual cells and / or cell aggregates), cell and / or cell aggregate morphology, cell aggregate area, volume, pIEQ (islet cell mass), diameter variations and spatial distribution of cell aggregates, and homogeneity of cell aggregates across a bioprinted material.21105427751\1\AMERICAS

[0008] In various embodiments, the bioprinting system includes various optical systems for capturing videos during the bioprinting process. The frames of the video are stitched together in a manner that reduces or eliminates the amount of repeated imagery across frames while maintaining the structural accuracy of biomaterial depicted in those frames. Reducing or eliminating the amount of repeated imagery and maintaining the structural accuracy of biomaterial depicted in the frames is important for downstream operations on the stitched image, such as image segmentation and cell counting, to yield accurate results.

[0009] In various embodiments, the determined characteristics of a bioprinted material can be used for many different purposes, including, but not limited to, determining a therapeutic dose associated with the bioprinted material, generating training data to improve the performance of the machine learning system with further training and modifications, and making adjustments to other aspects to bioprinting system to improve the quality of bioprinted material.

[0010] In embodiments, the material flow comprises at least one cross-linkable material, and preferably at least one hydrogel, optionally further comprising at least one biological material, e g. a plurality of cells in a biocompatible material. In embodiments, the plurality of cells is selected from the group comprising or consisting of a single-cell suspension, cell aggregates, cell clusters, cell agglomerates, cell spheroids, cell organoids, or combinations thereof. In embodiments, the material flow comprises micro-particles.

[0011] In embodiments, the plurality of cells comprises cells from endocrine and exocrine glands selected from the group consisting of pancreas, liver, thyroid, parathyroid, pineal gland, pituitary gland, thymus, adrenal gland, ovary, testis, enteroendocrine cells, stem cells, stem cell-derived cells or cells engineered to secrete a biologically active agent of interest. In embodiments, the plurality of cells releases cell-derived extracellular vesicles in the form of exosomes containing a therapeutic protein or nucleic acid. In embodiments, the biocompatible material is selected from alginate, collagen, decellularized extracellular matrices, hyaluronic acid, PEG, fibrin, gelatin, GEL-MA, silk, chitosan, cellulose, PCL, PLA, POEGMA, and combinations thereof.

[0012] In embodiments, the printhead may comprise one or more transparent channels, and an optical system may monitor material flow through at least one of the one or more transparent channels, preferably wherein the printhead comprises a transparent nozzle or dispensing channel.31105427751\1\AMERICASIn embodiments, the printhead may be an extrusion printhead or, a co-extrusion printhead, including but not limited to multi-material extrusion, multi-axial extrusion, or melt-extrusion, an inkjet printhead, an electrowriting printhead, or a microfluidic printhead, including but not limited to on-chip crosslinking, microfluidic stereolithography or microfluidic spinning.

[0013] Generally, the printhead will comprise one or more inputs fluidically connected to one or more reservoirs, one or more dispensing channels for depositing the material flow onto the print surface, and preferably one or more transparent channels or regions. The cross-linkable material within the material flow may be cross-linked within the printhead, while dispensing from the printhead, and / or after deposition on the print surface, as is known in the art. The print surface may further comprise, e.g. a cross-linker bath.

[0014] In embodiments, the optical system may comprise one or more microscopes, one or more still cameras, and / or one or more video cameras. There may be an optical system at each stage of the bioprinting system, from the reservoir to the printhead to the print surface.

[0015] In an exemplary embodiment, the printhead may comprise a plurality of transparent channels and the optical system may comprise an equal plurality of microscopes, still cameras, and / or video cameras.

[0016] In embodiments, the device comprises a three-dimensional (3D) bioprinting printhead. In embodiments, the 3D bioprinting printhead comprises a plurality of channels to selectively provide a respective plurality of materials for the material flow.

[0017] In embodiments, the at least one cross-linkable material comprises a hydrogel. In embodiments, the material flow further comprises at least one biological material; preferably wherein said at least one biological material comprises a plurality of cells. In embodiments, the plurality of cells is selected from the group comprising or consisting of a single-cell suspension, cell aggregates, cell clusters, cell agglomerates, cell spheroids, cell organoids, or combinations thereof. In embodiments, the material flow further comprises microparticles. In embodiments, the material flow further comprises dyes, pigments or colloids. In embodiments, the presence of cells in the material flow acts as a contrast agent to facilitate measurement of physical properties of the bioprinted fibers.41105427751\1\AMERICASBRIEF DESCRIPTION OF THE DRAWINGS

[0018] The accompanying drawings, which are incorporated herein and form a part of the specification, illustrate embodiments of the present disclosure and, together with the description, further serve to explain the principles of the disclosure and to enable a person skilled in the relevant art to make and use the disclosure.

[0019] FIG. l is a high level diagram of a bioprinting system according to an embodiment;

[0020] FIG. 2 is a high level flowchart illustrating different stages before, during, and after passage of a plurality of cells through different stages of a bioprinting system according to an embodiment;

[0021] FIG. 3 shows a detailed diagram of the computing system of FIG. 1 configured to implement one or more aspects of various embodiments;

[0022] FIG. 4 shows a detailed diagram of the segmentation engine and the occlusion detection engine 324, according to various embodiments;

[0023] FIG. 5A illustrates an example of a segmented representation with segmented regions corresponding to occluded cell aggregates, according to various embodiments;

[0024] FIG. 5B illustrates an example of edge detection, according to various embodiments;

[0025] FIG. 5C illustrates an example of boundary extrapolation, according to various embodiments;

[0026] FIG. 6 illustrates a detailed diagram of the characterizing engine, according to various embodiments;

[0027] FIG. 7 illustrates an example of cell quantification operation, according to various embodiments;

[0028] FIG. 8A illustrates the results of the cell quantification operation in FIG. 7 relative to a target pIEQ dose, according to various embodiments;51105427751\1\AMERICAS

[0029] FIG. 8B illustrates the results of the cell quantification operation in FIG. 7 relative to a target pIEQ dose and an average pIEQ dose, according to various embodiments;

[0030] FIG. 8C illustrates the results of the cell quantification operation in FIG. 7 relative to a DNA-based quantification, according to various embodiments;

[0031] FIG. 9 illustrates a detailed diagram of the frame stitching engine, according to various embodiments;

[0032] FIG. 10A illustrates an example of frame stitching, according to various embodiments;

[0033] FIG. 10B is a detailed illustration of the stitched image fiber, according to various embodiments;

[0034] FIG. 11 is a flow diagram of method steps for performing stitching of frames of a video capturing a bioprinting process, according to various embodiments;

[0035] FIG. 12 shows a partial screenshot of a stitched image of a printed fiber, according to various embodiments;

[0036] FIG. 13 shows an image of a post-print full device, according to various embodiments;

[0037] FIG. 14 illustrates the results of dose estimation of bioprinted fibres containing aggregated pancreatic islet cells, according to various embodiments; and

[0038] FIG. 15 illustrates the results of cell counting for three BTT samples containing aggregated Retinal Pigment Epithelium (RPE-EPO) cells, according to various embodiments.DETAILED DESCRIPTION

[0039] Certain illustrative aspects of the systems, apparatuses and methods according to the present invention are described herein in connection with the following description and the accompanying figures. These aspects are indicative, however, of but a few of the various ways in which the principles of the invention may be employed and the present invention is intended to include all such aspects and their equivalents. Other advantages and novel features of the invention61105427751\1\AMERICASmay become apparent from the following detailed description when considered in conjunction with the figures.

[0040] In the following detailed description, numerous specific details are set forth in order to provide a thorough understanding of the invention. In other instances, well known structures, interfaces and processes have not been shown in detail in order not to unnecessarily obscure the invention. However, it will be apparent to one of ordinary skill in the art that those specific details disclosed herein need not be used to practice the invention and do not represent a limitation on the scope of the invention, except as recited in the claims. It is intended that no part of this specification be construed to effect a disavowal of any part of the full scope of the invention. Although certain embodiments of the present disclosure are described, these embodiments likewise are not intended to limit the full scope of the invention.

[0041] FIG. 1 shows a high level diagram of a bioprinting system 100 according to an embodiment. The bioprinting system 100 includes a plurality of stages, labeled for ease of reference as reservoir 110 (which may be one or more reservoirs), printhead 120, and print surface 130 (which may be any surface on which material from printhead 120 may be deposited). In embodiments, the printhead 120 may be an extrusion or co-extrusion printhead, including but not limited to multi-material extrusion (e.g., Liu et al. Adv Mater. 2017 Jan;29(3): 10.1002), multi- axial extrusion, or melt-extrusion (e.g., Loewner et al., Frontiers in Bioengineering and Biotechnology 2022 DOI 10.3389 / fbioe.2022.896719), an inkjet printhead (e.g. Lal Roy, Lab on a Chip 2021 DOI 10.1039 / dllc00524c), an electrowriting printhead (e.g., Kade and Dalton, Advanced Healthcare Materials 2020 DOI 10.1002 / adhm.202001232), or a microfluidic printhead, including but not limited to on-chip crosslinking, microfluidic stereolithography (e.g., Miri et al., Adv Mater. 2018 Jul; 30(27): e!800242) or microfluidic spinning (e.g., Lee et al. Nature Materials 2011 10:877-883). In preferred embodiments, the bioprinting system employs on-chip crosslinking as disclosed in U.S. Patent No. 11,046,930, PCT / CA2014 / 050556, PCT / CA2018 / 050315, PCT / CA2018 / 050315, PCT / CA2018 / 050315 and PCT / CA2022 / 051855, the disclosures of which are expressly incorporated by reference herein.

[0042] As shown, each of the stages has associated with it respective optical systems, which provide imaging data for cells and cell aggregates at each stage. For example, reservoir 110 may71105427751\1\AMERICAShave associated with it optical system 112, which may comprise microscopes, still cameras, and / or video cameras to obtain images, at appropriate angles, of a plurality of cells in the reservoir, to characterize that plurality of cells before printing. Those cells may be individual cells; they may be what are termed fragments; or they may be cell aggregates. Similarly, printhead 120 may have associated with it optical system 122, which also may comprise microscopes, still cameras, and / or video cameras to obtain images, at appropriate angles, of cells in the printhead, for example, in a nozzle of the printhead as described in the above-referenced US provisional and PCT applications, to characterize that plurality of cells during printing. Those cells similarly may be individual cells; they may be what are termed fragments; or they may be cell aggregates. Likewise, print surface 130 may have associated with it optical system 132, which likewise may comprise microscopes, still cameras, and / or video cameras to obtain images, at appropriate angles, of cells on the print surface, to characterize that plurality of cells after printing. Those cells likewise may be individual cells; they may be what are termed fragments; or they may be cell aggregates. In various embodiments, optical systems may be included in the bioprinting system 100 for only a subset for the stages. Furthermore, an optical system may be shared by one or more stages.

[0043] In embodiments, bioprinting system 100 may be under control of computing system 150, which may include one or more central processing units (CPUs), volatile memory, non-volatile memory, and non-transitory non-volatile storage. Computing system 150 may connect with one or more of optical systems 1 12, 122, and 132 through a variety of connections, either direct, or over a local area network (LAN), or through the cloud.

[0044] The computing system 150 may include a machine learning module 160. In embodiments, the machine learning module 160 may employ one or a plurality of graphical processing units (GPUs) with pluralities of cores to facilitate the necessary calculations for a machine learning model to perform rapid computational analysis of data streams or captured images, and training of the model that provides feedback to control material flow in the bioprinting system. In embodiments, the GPUs of machine learning module 160 may be part of computing system 150. In embodiments, the machine learning module 160 may use the memory and storage that computing system 150 has. In other embodiments, the machine learning module 160 may have its own volatile memory, non-volatile memory, and / or non-transitory non-volatile storage. In81105427751\1\AMERICASembodiments, the model training may involve testing and validation to facilitate optimization and inference by the model being trained.

[0045] In embodiments, machine learning module 160 may comprise a neural network selected from the group consisting of a convolutional neural network (CNN), a region-based CNN (R- CNN), a deformable convolutional network based model, a Mask R-CNN, a you only look once (YOLO) based model, and a transformer-based instance segmentation model to perform image segmentation on the imaging data received from optical systems 112, 122, and 132. In an embodiment, machine learning module 160 may interact with computing system 150 to perform a number of user-interactive functions and also system-interactive functions. For user interaction, the computing system 150 may provide an appropriate graphical display and graphical user interface for interaction with various other components such as the optical systems, the machine learning module 160, and the rest of bioprinting system 100.

[0046] In a cloud environment, machine learning module 160 and its user interface may be deployed as a cloud application, for example, as a login-accessible website, through which the users can view and interact with print videos online and see results of machine learning analysis. In such embodiments, the computing system 150 and machine learning module 160 may be hosted in the cloud.

[0047] Depending on the embodiment, computing system 150 may control the optical systems 112, 122, and 132 through one or more graphical user interfaces on one or more displays. Depending on the embodiment, optical systems 112, 122, and 132 may have their own control apparatus, whereby these optical systems, responsive to commands from the computing system 150, may position themselves appropriately relative to reservoir 110, printhead 120 and its associated feeds and nozzle(s), and print surface 130, respectively.

[0048] Part of the control that computing system 150 performs involves monitoring of access to control systems in the bioprinting system to adjust material flow, whether by control of pressure in various printhead channels (including, for example, microfluidic printhead channels) to be discussed below, or by control of displacement of material through the channels. In embodiments, control may involve toggling on or off or otherwise adjusting opening and closing of pneumatic valves on a printhead. In embodiments, part of the control that computing system 150 performs91105427751\1\AMERICASinvolves panoptic segmentation, semantic segmentation, instance segmentation, occlusion-aware segmentation, and / or image inpainting. In embodiments, the panoptic segmentation, semantic segmentation, instance segmentation, occlusion-aware segmentation, and / or image inpainting enables visual estimation of a general amount and / or distribution of biological material (e.g., plurality of cells) within a material flow or other characteristics of the bioprinted fibers.

[0049] Computing system 150 also may enable reading of the respective feeds from one or all of the optical systems to enable recording and / or loading of one or both of the feeds, and adjust viewing parameters such as contrast, tint, brightness, and sharpness, among others.

[0050] Depending on the embodiment, machine learning module 160 may characterize particles in bioprinting system 100, in a manner to be prescribed. Alternatively or in addition, machine learning module 160 may help provide focus to images from optical systems 112, 122, and 132 to aid in characterizing the particles. Alternatively or in addition, machine learning module 160 may provide outputs which can be fed back in real-time to the 3D bioprinting system in order to adjust pressure and / or displacement and subsequent flow of materials from the reservoir 110, to the printhead 120, to the print surface 130, to enable consistent production of quality tissues and minimize loss of expensive biomaterial and cell inputs.

[0051] FIG. 2 shows a high level view of the flows involved in the operation of bioprinting system 100 in conjunction with computing system 150 and machine learning module 160 according to an embodiment. At 210, cell quantification and characterization may be carried out before printing, for example, in a reservoir 110 of bioprinting system 100. At 212, cells may optionally be preprinted on a plate, e.g. to prompt aggregation. At this stage as well, the cells may be quantified and characterized. 212 is not essential, and may be omitted.

[0052] At 214, the cells are in a printhead, to be printed. Here, too, using the optical system, computing system, and the machine learning module 160 in FIG. 1, the cells in the printhead may be characterized. In an embodiment, comparisons may be made between the cells in the printhead and the cells in the microwells. At 216, the cells may be deposited into a printed device, for example, on the print surface 130 in FIG. 1. At 218, the device structure is characterized. At 220, before the resulting tissue is implanted, cell analysis may be done. Depending on the embodiment, the same machine learning module 160 in FIG. 1, or different machine learning systems, may be101105427751\1\AMERICASapplied at each stage 210-220, and the results at a given stage compared to results at other stages. In an embodiment, the results at a given stage may be compared to a reference to enable a determination of whether and / or how further training of the machine learning system should be carried out or whether any other modifications to the bioprinting system is needed.

[0053] FIG. 3 shows a detailed diagram of computing system 150 configured to implement one or more aspects of various embodiments. In one embodiment, computing system 150 includes a desktop computer, a laptop computer, a smart phone, a personal digital assistant (PDA), tablet computer, a computer server, or any other type of computing device configured to receive input, process data, and optionally display images, and is suitable for practicing one or more embodiments. Computing system 150 is configured to execute machine learning module 160. As shown, machine learning module 160 resides in memory 316 and includes an image pre-processing engine 318, a segmentation engine 320, a frame stitching engine 322, an occlusion detection engine 324, and a characterizing engine 326.

[0054] It is noted that the computing device described herein is illustrative and that any other technically feasible configurations fall within the scope of the present disclosure. For example, multiple instances of the image pre-processing engine 318, the segmentation engine 320, the occlusion detection engine 324, and the characterizing engine 326 could execute on a set of nodes in a distributed and / or cloud computing system to implement the functionality of machine learning module 160. In another example, the image pre-processing engine 318, the segmentation engine 320, the occlusion detection engine 324, and the characterizing engine 326 could execute on various sets of hardware, types of devices, or environments to adapt the image pre-processing engine 318, the segmentation engine 320, the occlusion detection engine 324, and the characterizing engine 326 to different use cases or applications. In a third example, image preprocessing engine 318, the segmentation engine 320, the occlusion detection engine 324, and the characterizing engine 326 could execute on different computing devices and / or different sets of computing devices.

[0055] In one embodiment, computing system 150 includes, without limitation, an interconnect (bus) 312 that connects one or more processors 302, an input / output (I / O) device interface 304 coupled to one or more input / output (I / O) devices 308, memory 316, a storage 314, and a network111105427751\1\AMERICASinterface 306. Processor(s) 302 may be any suitable processor implemented as a central processing unit (CPU), a graphics processing unit (GPU), an application-specific integrated circuit (ASIC), a field programmable gate array (FPGA), an artificial intelligence (Al) accelerator, any other type of processing unit, or a combination of different processing units, such as a CPU configured to operate in conjunction with a GPU. In general, processor(s) 302 may be any technically feasible hardware unit capable of processing data and / or executing software applications. Further, in the context of this disclosure, the computing elements shown in computing system 150 may correspond to a physical computing system (e.g., a system in a data center) or may be a virtual computing instance executing within a computing cloud.

[0056] I / O devices 308 include devices capable of providing input, such as a keyboard, a mouse, a touch-sensitive screen, a microphone, and so forth, as well as devices capable of providing output, such as a display device. Additionally, VO devices 308 may include devices capable of both receiving input and providing output, such as a touchscreen, a universal serial bus (USB) port, and so forth. I / O devices 308 may be configured to receive various types of input from an enduser (e.g., a designer) of computing system 150, and to also provide various types of output to the end-user of computing system 150, such as displayed digital images or digital videos or text. In some embodiments, one or more of I / O devices 308 are configured to couple computing system 150 to a network 330.

[0057] Network 330 is any technically feasible type of communications network that allows data to be exchanged between computing system 150 and external entities or devices, such as a web server or another networked computing device. For example, network 330 may include a wide area network (WAN), a local area network (LAN), a wireless (WiFi) network, and / or the Internet, among others.

[0058] Storage 314 includes non-volatile storage for applications and data, and may include fixed or removable disk drives, flash memory devices, and CD-ROM, DVD-ROM, Blu-Ray, HD-DVD, or other magnetic, optical, or solid-state storage devices. The image pre-processing engine 318, the segmentation engine 320, the occlusion detection engine 324, and the characterizing engine 326 may be stored in storage 314 and loaded into memory 316 when executed.121105427751\1\AMERICAS

[0059] Memory 316 includes a random-access memory (RAM) module, a flash memory unit, or any other type of memory unit or combination thereof. Processor(s) 302, I / O device interface 304, and network interface 306 are configured to read data from and write data to memory 316. Memory 316 includes various software programs that can be executed by processor(s) 302 and application data associated with said software programs, including the image pre-processing engine 318, the segmentation engine 320, the frame stitching engine 322, the occlusion detection engine 324, and the characterizing engine 326.

[0060] The image pre-processing engine 318 processes imaging data, such as still images or videos, captured by one or more of the optical systems included in the bioprinting system 100 such that downstream operations, such as image segmentation, cell quantization, etc., can be performed using those images. For example, the image pre-processing engine 318 can perform smoothing and noise reduction operations, such as Gaussian blurring, median filtering, or bilateral filtering, on imaging data captured by the optical systems to obtain better performance in downstream operations. In another examples, the image pre-processing engine 318 can perform color and / or contrast adjustment operations, noise reduction operations, background removal operations, image resizing operations, stitching / assembling operations, clustering operations, and / or normalization operations on the imaging data. In some embodiments, the image pre-processing engine 318 implements one or more machine learning models to perform the pre-processing operations on the imaging data. For example, the image pre-processing may implement a K-Nearest Neighbors (KNN) machine learning algorithm for clustering objects in the imaging data.

[0061] The segmentation engine 320 processes image frames in the imaging data (e.g., frames in a video) captured by one or more of the optical systems included in the bioprinting system 100 and segments the image frames into segmented regions. In some embodiments, the segmentation engine 320 operates on imaging data that is pre-processed by the image pre-processing engine 318. In operation, for a given image frame, the segmentation engine 320 executes one or more machine learning models with the image frame as input and generates a segmented representation corresponding to the image frame. The segmented representation (also referred to as a segmentation mask) identifies, for each pixel in the image frame, a segmented region with which the pixel is associated. For example, for an image captured by optical system 132, the segmented representation may identify segmented regions corresponding to individual cells, fragments, or131105427751\1\AMERICAScell aggregates within a material flow or a printed fiber. In various embodiments, one or more occlusion detection and correction operations may be performed on the segmented representation to account for cell aggregates that are occluded within the imaging data. The segmentation engine 320 is discussed in more detail in conjunction with at least Figure 4.

[0062] The frame stitching engine 322 stitches frames of a video captured by one or more of the optical systems included in the bioprinting system 100 to generate a stitched image of a bioprinted fiber. The video shows the flow of biomaterial through the bioprinting system 100. In various embodiments, the frame rate of an optical system capturing a video of the bioprinting process is greater than the speed at which biomaterial (e.g., cells, fragments, cell aggregates) flows through the printing system. Thus, consecutive frames captured in the video contain overlapping portions of the same scene of the biomaterial flowing through the bioprinting system. For the image of the bioprinting fiber to be accurate, the frames of the video are stitched in a manner that reduces or eliminates the amount of overlapping imagery across frames while maintaining the structural accuracy of biomaterial depicted in those frames. The occlusion detection engine 324 is discussed in more detail in conjunction with at least Figures 9-11.

[0063] The occlusion detection engine 324 detects and accounts for occlusions within the segmented representation of imaging data generated by the segmentation engine 320. An occlusion occurs when one or more objects in an image overlap with one or more other objects. In such a scenario, the segmented region corresponding to an occluded object that overlaps with another object may not properly define the boundaries of the occluded object. For example, the segmented representation for an image captured by optical system 132 may identify a segmented region corresponding to a cell aggregate that is occluded by another cell aggregate. The segmented region corresponding to the occluded cell aggregate may only cover a non-occluded portion of the occluded cell aggregate. The occlusion detection engine 324 detects occluded objects in the imaging data and then extrapolates boundaries around the occluded portions based on the boundary of the segmented region corresponding to the non-occluded portions. Such extrapolation allows for a more accurate segmented representation of objects, e.g., cell aggregates, in an image frame. The occlusion detection engine 324 is discussed in more detail in conjunction with at least Figure 4.141105427751\1\AMERICAS

[0064] The characterizing engine 326 operates on the segmented representations generated by the segmentation engine 320 to determine one or more characteristics associated with the biological material (e.g., cells, fragment, and / or cell aggregates) within a material flow or other characteristics of the bioprinted fibers or devices generated from the bioprinted fibers. These characteristics include, but are not limited to, a cell count (individual cells and / or cell aggregates), cell and / or cell aggregate morphology, cell aggregate area, volume, pIEQ (islet cell mass), diameter variations and spatial distribution of cell aggregates, and homogeneity of cell aggregates across a bioprinted fiber or devices generated from the bioprinted fibers. The characteristics determined by the characterizing engine 326 can be used for many different purposes, including, but not limited to, determining a therapeutic dose associated with the printed fiber and / or device, generating training data to improve the performance of the machine learning module 160 with further training and modifications, and making adjustments to other aspects of the bioprinting system 100 to improve the quality of bioprinted fibers and / or devices. For example, and without limitation, the flow of the material through the printhead may be adjusted, the motion profile of the material may be adjusted, or the composition of the biomaterial input into the bioprinting system 100 may be adjusted. The characterizing engine 326 is discussed in more detail in conjunction with at least Figures 4, 6, and 7.

[0065] FIG. 4 shows a detailed diagram of the segmentation engine 320 and the occlusion detection engine 324, according to various embodiments. As shown, the segmentation engine 320 includes a segmentation machine learning model 404 and segmented representations 406, and the occlusion detection engine 324 includes overlapping aggregate detector 408, boundary extrapolator 410, and modified segmented representations 412.

[0066] In operation, the segmentation machine learning model 404 included in the segmentation engine 320 operates on imaging data 402 to generate segmented representations 406. In various embodiments, imaging data 402 includes one or more image frames captured by the optical systems included in the bioprinting system 100. The image frames may include images of cells and / or cell aggregates during and / or after bioprinting. The image frames may be captured using one or more of microscopes, still cameras, and / or video cameras. In some embodiments, some or all of the imaging data 402 is pre-processed by image pre-processing engine 318 before transmission to the segmentation engine 320.151105427751\1\AMERICAS

[0067] The segmentation machine learning model 404 is a trained machine learning model that processes an input imaging data 402 and generates segmented representations of the input imaging data 402. The segmentation machine learning model 404 can be one or more of a convolutional neural network (CNN) (e.g., a U-Net), a region-based CNN (R-CNN), a deformable convolutional network based model, a Mask R-CNN, a you only look once (YOLO) based model, and a transformer-based instance segmentation model. In some embodiments, the segmentation machine learning model 404 is trained to perform segmentation tasks on any type of image. In other embodiments, the segmentation machine learning model 404 is fine-tuned for segmentation of images capturing biological materials, such as cells and cell aggregates.

[0068] For a given image frame, the segmentation machine learning model 404 generates a segmented representation that identifies a boundary around each region in the image frame that includes a detected object (“referred to as a segmented region”). For example, consider an image with one or more individual cells and one or more cell aggregates. A segmented representation of the image frame identifies a boundary around each of the individual cells and each of the cell aggregates. In some embodiments, depending on the performance of the segmentation machine learning model 404, not all of the individual cells and / or cell aggregates may be separately segmented into their respective segmented regions by the segmentation machine learning model 404.

[0069] In various embodiments, the segmented representation generated by the segmentation machine learning model is a segmentation mask. The segmentation mask corresponding to an image frame is an array that represents the image segmentation result by labeling each pixel (or voxel) of the input image according to the segmented region to which the pixel belongs. In a segmentation mask, the boundaries of a segmented region are defined by the area of the pixels that are labeled as belonging to the segmented region.

[0070] In certain situations, objects captured in an image frame may be overlapping, such that a given object may be partially or fully occluded by another object. For example, in bioprinting, where multiple layers of material flow having cells are stacked, images captured of the printed material may include cells or cell aggregates that are partially or fully occluded by cells or cell aggregates in higher layers of the stack. When image segmentation is performed on images with161105427751\1\AMERICASpartially or fully occluded objects, the segmented representation may not accurately capture the boundaries of an object that is occluded by another object. For example, the segmented region associated with a given object that is occluded by another object may only cover an area of the occluded object that is visible.

[0071] FIG. 5A illustrates an example of a segmented representation with segmented regions corresponding to occluded cell aggregates, according to various embodiments. As shown, imaging data 502 is an image frame captured by an optical system in bioprinting system 100. The imaging data 502 comprises an image of cell aggregates within a material flow during and / or after printing by the bioprinting system 100. The segmentation machine learning model 404 processes the imaging data 502 to generate the segmented representation 504, which includes, among other segmented regions, a segmented region corresponding to an individual cell aggregate 506, a segmented region corresponding to an occluded cell aggregate 508, and a segmented region corresponding to an occluded cell aggregate 510. As is visible in FIG. 5, the segmented region corresponding to an individual cell aggregate 506 covers most, if not all, of the corresponding cell aggregate. By contrast, the segmented regions covering occluded cell aggregate 508 and occluded cell aggregate 510 do not account for the occluded portions of the corresponding cell aggregates. Further, the boundaries surrounding the segmented regions corresponding to occluded cell aggregate 508 and occluded cell aggregate 510 do not coincide with the true boundaries of the respective cell aggregates.

[0072] Turning back to FIG. 4, the occlusion detection engine 324 detects and accounts for occlusions within the segmented representation 406. The overlapping aggregate detector 408 processes the segmented representation 406 to identify each segmented region in the segmented representation 406 that corresponds to an occluded cell aggregate. In operation, the overlapping aggregate detector 408 performs one or more edge detection techniques on the image frame associated with the segmented representation 406 to identify edges within the image frame. Examples of edge detection techniques include, but are not limited to, Canny edge detection, Lapacian of Gaussian, Difference of Gaussians. Once the edges in the image frame are identified, the overlapping aggregate detector 408, for each segmented region, selects points on the boundary surrounding that segmented region that overlap with points on the identified edges. The points on171105427751\1\AMERICASthe boundary that align with the identified edges correspond to non-occluded parts of the object associated with the segmented region.

[0073] The overlapping aggregate detector 408 next fits a pre-determined shape associated with cell aggregates, e.g., an ellipse, to the portion of the segmented region that is non-occluded. The fitting operation can be performed using methods such as OPENCV’s Fit Ellipse via Direct Least Squares (DLS), Approximate Mean Square (AMS), etc. For a given segmented region, if all or approximately all of the segmented region is covered by the fitted shape, then the segmented region corresponds to an aggregate that is non-occluded. If, however, for a given segmented region, an area larger than the segmented region is covered by the fitted shape, then the segmented region corresponds to an aggregate that is at least partially overlapping with another cell aggregate and is, thus, at least partially occluded by that other cell aggregate.

[0074] FIG. 5B illustrates an example of edge detection, according to various embodiments. The example in FIG. 5B is a follow-on from the example in FIG. 5A. As discussed above, imaging data 502 is an image frame captured by an optical system in bioprinting system 100. The imaging data 502 captures an image of cell aggregates within a material flow during and / or after printing by the bioprinting system 100. The segmentation machine learning model 404 processes the imaging data 502 to generate the segmented representation 504, which includes, among other segmented regions, a segmented region corresponding to an individual cell aggregate 506, a segmented region corresponding to an occluded cell aggregate 508, and a segmented region corresponding to occluded cell aggregate 510. Further, the occlusion detection engine 324 performs one or more edge detection operations on the imaging data 502 to detect edges within the image. Detected edges 512 visually illustrate the result of edge detection performed on imaging data 502.

[0075] For individual cell aggregate 506, the overlapping aggregate detector 408 selects points on the boundary surrounding the corresponding segmented region that overlap with points on the identified edges. Point 514 is an example of such an overlapping point. The points on the boundary that align with the identified edges correspond to non-occluded parts of the individual cell aggregate 506. In this case, the individual cell aggregate 506 is non-occluded so every point on the boundary surrounding the corresponding segmented region overlaps with points in the181105427751\1\AMERICASdetected edges 512. As a result, an ellipse fitted to the overlapping points would cover most if not all of the corresponding segmented region, indicating that the individual cell aggregate 506 is not occluded.

[0076] For occluded cell aggregate 508, the overlapping aggregate detector 408 selects points on the boundary surrounding the corresponding segmented region that overlap with points on the identified edges. Point 516 is an example of such an overlapping point. By contrast, point 518 does not overlap with any point on the identified edges. In this case, point 518 is in an occluded portion of occluded cell aggregate 508 and is, thus, not on a true boundary of occluded cell aggregate 508. As a result, an ellipse fitted to the overlapping points would cover an area larger than the segmented region corresponding to occluded cell aggregate 508, indicating that the occluded cell aggregate 508 is at least partially occluded.

[0077] Turning back to FIG. 4, the boundary extrapolator 410 extrapolates the boundary surrounding the non-occluded portions of occluded cell aggregates based on the shape fitted to the non-occluded portions. In particular, for a given cell aggregate identified as occluded, the boundary extrapolator 410 extrapolates the portion of the boundary around the segmented region corresponding to the non-occluded portion to match the shape fitted to the non-occluded portion. This extension of the boundary approximates the true boundary of the cell aggregate. In such a manner, the boundary extrapolator 410 modifies the segmented representation associated with the imaging data such that the segmented regions corresponding to occluded cell aggregates have an extended boundary that approximates the true boundary of those cell aggregates.

[0078] FIG. 5C illustrates an example of boundary extrapolation, according to various embodiments. The example in FIG. 5C is a follow-on from the examples in FIG. 5A and 5B. As shown, the modified segmented representation 520 includes segmented regions surrounding occluded cell aggregate 508 and occluded cell aggregate 510 where the boundaries of those segmented regions are extrapolations of the boundaries in segmented representation 504. In particular, the boundary extrapolation 522 illustrates the extrapolated boundary surrounding the segmented region that approximates to the true boundary of occluded cell aggregate 510. Similarly, the boundary extrapolation 524 illustrates the extrapolated boundary surrounding the segmented region that approximates to the true boundary of occluded cell aggregate 508.191105427751\1\AMERICAS

[0079] Turning back to FIG. 4, the characterizing engine 326 receives the segmented representation and / or the modified segmented representation corresponding to imaging data and performs one or more characterizing operations to generate characteristics 414. These characteristics include, but are not limited to, a cell count (individual cells and / or cell aggregates), cell and / or cell aggregate morphology, cell aggregate area, volume, pIEQ (islet cell mass), diameter variations and spatial distribution of cell aggregates, and homogeneity of cell aggregates across a bioprinted fiber or a device generated from one or more bioprinted fibers. The characteristics of a bioprinted fiber or a device generated from one or more bioprinted fibers, as determined by the characterizing engine 326, can be used for many different purposes, including, but not limited to, determining a therapeutic dose associated with the printed fiber and / or device, generating training data to improve the performance of the machine learning module 160 with further training and modifications, and making adjustments to other aspects to bioprinting system 100 to improve the quality of bioprinted fibers and / or devices.

[0080] FIG. 6 shows a detailed diagram of the characterizing engine 326, according to various embodiments. As shown, the characterizing engine 326 includes a granular cell quantification 604, a coarse cell quantification 606, and fidelity indicators 608.

[0081] The granular cell quantification 604 determines a granular (e.g., upper bound) estimate of a quantity of cells within a printed fiber based at least on the individual cell aggregates and deoccluded cell aggregates included in imaging data captured during and / or after printing of the printed fiber. As discussed above, when processing imaging data, the segmentation engine 320 generates a segmented representation of the imaging data that identifies a boundary around each region in the image frame that includes detected biological materials, such as a cell, a fragment or a cell aggregate. In some scenarios, a segmented region corresponds to a cell aggregate that is occluded by another cell aggregate and, thus, the boundary around the segmented region does not capture the true boundary of the cell aggregate. The occlusion detection engine 324 detects such occlusions in the segmented representation of the imaging data and, for occluded cell aggregates, extrapolates the boundary of the segmented region to approximate the true boundary of the cell aggregate. In such a manner, the occlusion detection engine 324 generates a modified segmented representation that identifies individual cell aggregates and de-occluded cell aggregates. The modified segmented representations 610 provided to the granular cell quantification 604 is an201105427751\1\AMERICASexample of such a representation generated by the occlusion detection engine 324 for imaging data corresponding to a printed fiber.

[0082] The granular cell quantification 604 determines the granular estimate of the quantity of cells in the printed fiber by computing an estimated cell quantity for each cell aggregate represented in the modified segmented representations 610. In operation, for each individual aggregate or de-occluded cell aggregate, the granular cell quantification 604 determines an ellipsoidal volume associated with the cell aggregate. In various embodiments, to compute the ellipsoidal volume, the granular cell quantification 604 first determines an area associated with the cell aggregate. From the area, the granular cell quantification 604 determines the circle diameter of the aggregate using the following equation:

[0083] Based on the diameter of the aggregate, the granular cell quantification 604 determines the ellipsoidal volume associated with the aggregate. The ellipsoidal volume can be computed using the following equation:

[0084] The ellipsoidal volume can then be used to estimate cell count for the particular cell aggregate represented in the fiber and / or device. In the exemplary embodiment shown in Figure 6, from the ellipsoidal volume, the granular cell quantification 604 determines a measure of pancreatic islet cell quantity associated with the aggregate, commonly referred to as the pIEQ. The pIEQ for a given aggregate is determined based on the following equation:211105427751\1\AMERICASwhere VIEQ I S a standard islet volume of 1.77xl06pm3, containing approximately 1,500-2,000 cells.

[0085] In the manner described above, the granular cell quantification 604 determines the measure of cell quantity, e.g. pIEQ, associated with each cell aggregate (individual or de-occluded) represented in the modified segmented representations 610. The sum or combined pIEQ of all the cell aggregates represented in the modified segmented representations 610 represents a measure of cell quantity in the printed fiber. This measure of cell quantity is a granular or upper bound estimate of the number of cells within the printed fiber.

[0086] The coarse cell quantification 606 determines a coarse (e.g., lower bound) estimate of a quantity of cells within a printed fiber based at least on the individual cell aggregates and merged cell aggregates included in imaging data capturing during and / or after printing of the printed fiber. As discussed above, when processing imaging data, the segmentation engine 320 generates a segmented representation of the imaging data that identifies a boundary around each region in the image frame that includes detected biological materials, such as a cell, a fragment or a cell aggregate. In some cases, if any two or more cell aggregates identified in the segmented representation overlap by more than a threshold percent (e.g., 5%), those cell aggregates are considered merged. The segmented regions around those merged cell aggregates are combined to define a larger segmented region and corresponding boundary surrounding all of the merged cell aggregates.

[0087] The coarse cell quantification 606 determines the measure of cell quantity, e.g. pIEQ, for each individual cell aggregate and merged cell aggregate represented in the segmented representations 612. In various embodiments, the pIEQ of an individual cell aggregate is determined in a similar manner as described above in conjunction with the granular cell quantification 604. For the merged cell aggregates, the coarse cell quantification determines the pIEQ based on an average pIEQ of individual aggregates in the imaging data. More specifically, for a given merged cell aggregate, the coarse cell quantification 606 first determines an average area of individual cell aggregates in the imaging data and then divides the size of the area of the merged cell aggregate with the size of the average area. The resulting value is an aggregate count indicating an approximate number of aggregates included in the merged area. The coarse cell221105427751\1\AMERICASquantification 606 also determines an average pIEQ of individual cell aggregates in the imaging data. The coarse cell quantification 606 then determines the pIEQ of the merged cell aggregate by multiplying the aggregate count with the average pIEQ.

[0088] The sum or combined pIEQ of the individual and merged cell aggregates represented in the segmented representations 612 represents a measure of cell quantity in the printed fiber. This measure of cell quantity is a coarse or lower bound estimate of the number of cells within the printed fiber. The coarse estimate of the number of cells within the printed fiber may be used for quality control, training of machine learning models, or benchmarking related to the printed fiber.

[0089] In various embodiments, the characterizing engine 326 computes other metrics associated with the printed fibers based on the modified segmented representations 610 or the segmented representations 612. These metrics could include fidelity indicators 608, such as cell or cell aggregate morphology, cell aggregate area, diameter variations and spatial distribution of cell aggregates, and homogeneity of cell aggregates across a bioprinted fiber or a device generated from one or more bioprinted fibers. The characteristics of a bioprinted fiber or a device generated from one or more bioprinted fibers, as determined by the characterizing engine 326, can be used for many different purposes, including, but not limited to, determining a therapeutic dose associated with the printed fiber and / or device, generating training data to improve the performance of the machine learning module 160 with further training and modifications, and making adjustments to other aspects to bioprinting system 100 to improve the quality of bioprinted fibers and / or devices.

[0090] FIG. 7 illustrates an example of cell quantification operations, according to various embodiments. Imaging data 702 includes five representative images of a printed fiber. The imaging data 702 is processed by the segmentation engine 320 to generate a segmented representation of the imaging data 702. The occlusion detection engine 324 generates a modified segmented representation that identifies individual cell aggregates and de-occluded cell aggregates 704. In some cases, if any two or more cell aggregates identified in the segmented representation overlap more than a threshold percent (e.g., 5%), those cell aggregates are considered merged. The segmented regions around those merged cell aggregates are combined to define a larger231105427751\1\AMERICASsegmented region and corresponding boundary surrounding all of the merged cell aggregates. These merged cells are shown in the individual and merged cell aggregates 706.

[0091] The characterizing engine 326 operates on the individual and de-occluded cell aggregates 704 associated with a portion of the fiber to generate a granular cell quantification for the portion of the fiber 708 using the techniques discussed above in conjunction with FIG. 6. Similarly, the characterizing engine 326 operates on the individual and merged cell aggregates 706 associated with the portion of the fiber to generate a coarse cell quantification for the portion of the fiber 710 using the techniques discussed above in conjunction with FIG. 6.

[0092] The characterizing engine 326 computes a granular linear density, within fiber linear density estimation 712, based on the granular cell quantification and the length of the portion of the fiber. The length of the portion of the fiber may be determined by calculating a length of a midline 718 determined for the portion. The granular linear density is multiplied by the actual length of the fiber to generate the granular cell quantification for the fiber 714. In various embodiments, the actual length of the fiber during the printing by measuring the fiber in a petri dish.

[0093] The characterizing engine 326 also computes a coarse linear density, within fiber linear density estimation 712, based on the coarse cell quantification and the length of the portion of the fiber. The coarse linear density is multiplied by the length of the fiber to generate the coarse cell quantification for the fiber 716.

[0094] In various embodiments, the machine learning module 160 segments, identifies, measures, quantifies and calculates cell aggregates in a bioprinted fiber and / or product and the area, diameter, and ellipsoidal volume of each aggregate. The ellipsoidal volume of each aggregate can be converted to an equivalent of a certain number of a standardized aggregate with a predetermined diameter. Each standardized aggregate with a predetermined diameter has or equals to an associated number of single cells, depending on the specific cell type. For example, for pancreatic islet aggregates, the ellipsoidal volume of each aggregate is converted to the measurements of an estimated number of aggregates with 150um diameter (e g. pIEQs for pancreatic cells). Based on the number of aggregates with 150um diameter, a total cell number for a variety of different tissue types, which varies according to how many cells are included in an aggregate with a given241105427751\1\AMERICASdiameter, such as a 150um diameter, can be calculated using the following equation: (Total number of 150um aggregates (pIEQs) in bioprinted product) X (number of cells in a 150um aggregate). In the case of pancreatic cells, for an aggregate with diameter of 150 pm, the volume is 1.77 E6 pm3, denoting 1,750 cells per pIEQ (this is the mid-point of the range shown in Buchwald P, Bernal A, Echeverri F, Tamayo-Garcia A, Linetsky E, Ricordi C. Fully Automated Islet Cell Counter (ICC) for the Assessment of Islet Mass, Purity, and Size Distribution by Digital Image Analysis. Cell Transplant. 2016 Oct;25(10): 1747-1761. doi: 10.3727 / 096368916X691655. PMID: 27196960). For other cell types, the calculations can be adjusted accordingly depending on the number of cells in a 150um aggregate by changing the denominator coefficients to generate the number of aggregates with 150um diameter (e.g. pIEQ), and number of cells. Similarly, the aggregates diameter and the number of cells per aggregate can be adjusted for different cell types.

[0095] FIG. 8A illustrates the results of the cell quantification operation in FIG. 7, according to various embodiments. The upper estimate number of cell aggregates corresponds to the granular cell quantification described above, and the lower estimate number of cell aggregates in FIG. 8A corresponds to the coarse cell quantification described above.

[0096] FIG. 8B illustrates the results of the cell quantification operation in FIG. 7, according to various embodiments. The upper estimate number of cell aggregates in FIG. 8B corresponds to the granular cell quantification described above, and the lower estimate number of cell aggregates in FIG. 8B corresponds to the coarse cell quantification described above.

[0097] FIG. 8C illustrates the results of the cell quantification operation in FIG. 7 relative to a DNA-based quantification, according to various embodiments. The Al upper estimate in FIG. 8C corresponds to the granular cell quantification described above, and the Al lower estimate in FIG. 8C corresponds to the coarse cell quantification described above. The Al upper estimate (the granular cell quantification) and the DNA quantification are within 5%. This benchmark illustrates the effectiveness and performance of the de-occlusion and cell quantification techniques described herein.

[0098] Bioprinting Video Frame Stitching:251105427751\1\AMERICAS

[0099] As discussed above, the bioprinting system 100 includes various optical systems for capturing videos of the bioprinting process. For example, optical system 122, which may comprise microscopes, still cameras, and / or video cameras, obtains a video at appropriate angles, of cells in the printhead, for example, in a nozzle of the printhead. In various embodiments, the frame rate of an optical system capturing a video of the bioprinting process is greater than the speed at which biomaterial (e.g., cells, fragments, cell aggregates) flows through the printing system. Thus, consecutive frames captured in the video contain repeated imagery of the same scene of the biomaterial flowing through the bioprinting system. For the image of the bioprinting fiber to be accurate and of high quality, the frame stitching engine 322 stitches the frames of the video in a manner that reduces or eliminates the amount of repeated imagery across frames while maintaining the structural accuracy of biomaterial depicted in those frames. Reducing or eliminating the amount of repeated imagery and maintaining the structural accuracy of biomaterial depicted in the frames is important for downstream operations performed on the stitched image, such as image segmentation and cell counting, to yield accurate results.

[0100] FIG. 9 illustrates a detailed diagram of the frame stitching engine 322, according to various embodiments. As shown, the frame stitching engine 322 includes a frame extractor 902, a template matcher 904, a segmentation-aware stitcher 906, and a stitched image of a bioprinted fiber 908.

[0101] The frame stitching engine 322 receives, as input, imaging data 910 and segmented representations 912 associated with the imaging data 910. The imaging data 910 includes a video of a bioprinting process of bioprinting a fiber, as captured by an optical system in the bioprinting system 100. In other embodiments, the imaging data 910 includes frames extracted from a video of a bioprinting process. The segmented representations 912 associated with the imaging data 910 include segmented representations of each frame in the video of the bioprinting process. As discussed above in conjunction with FIGs 3-4, the segmentation engine 320 includes a segmentation machine learning model that processes input imaging data, such as frames of a video, and generates segmented representations of the input imaging data. For a given image frame, the segmentation machine learning model generates a segmented representation that identifies a boundary around each region in the image frame that includes a detected object (“referred to as a segmented region”). For example, consider an image frame with one or more individual cells and one or more cell aggregates. A segmented representation of the image frame identifies a boundary261105427751\1\AMERICASaround each of the individual cells and each of the cell aggregates. In various embodiments, the segmented representation generated by the segmentation machine learning model is a segmentation mask. The segmentation mask corresponding to an image frame is an array that represents the image segmentation result by labeling each pixel (or voxel) of the input image according to the segmented region to which the pixel belongs. In a segmentation mask, the boundaries of a segmented region are defined by the area of the pixels that are labeled as belonging to the segmented region.

[0102] The frame extractor 902 processes an input video and extracts one or more image frames from the input video. In various embodiments, the image frames extracted from the input video match the image frames of the video for which the segmentation engine 320 generated segmented representations 912. In various embodiments, the frame stitching engine 322 receives image frames extracted from the input video either as a part of the imaging data 910 or from the segmentation engine 320. In such embodiments, the frame extractor 902 is an optional component.

[0103] The template matcher 904 and the segmentation-aware stitcher 906 operate in conjunction on pairs of consecutive image frames extracted from a video to stitch together an image of a bioprinted fiber 908 that minimizes the amount of repeated imagery and maintains structural accuracy of biomaterials captured in the image frames. For a given pair of consecutive image frames, such as a frame captured at t and a frame captured at t-N, the template matcher 904 removes a repeated portion from frame t that is also depicted in frame t-N. The segmentation-aware stitcher 906 then determines a seam for stitching the remaining portion of frame 1 to the most current version of the image of the bioprinted fiber 908.

[0104] In operation, the first frame of the video captured at Z=0 is added to an image buffer that stores the result of the stitching operations performed by the template matcher 904 and the segmentation-aware stitcher 906. For the next frame and each frame thereafter, the template matcher 904 determines a portion of the frame that repeats imagery captured in the prior frame. In particular, for a pair of consecutive frames, frame t and frame t-N, the template matcher 904 first determines an overlap width, w, that defines an estimated boundary of repeated imagery across the two frames. In various embodiments, the overlap width w may be pre-defined for all frames in the video. For example, w may be 50% of the width of the frames in the video. In various271105427751\1\AMERICASembodiments, w may be determined based on configurations of the bioprinting system 100, including configurations of the optical systems therein. In some embodiments, w may be experimentally determined.

[0105] Once the overlap width w is determined, the template matcher 904 determines a template portion of frame t-N that is of size w. The template portion of frame t-N starts at t-w until the end of the frame t-N. In other words, the template portion of frame t-N corresponds to the last w length of frame t-N The template matcher 904 then compares the template portion with frame t to identify the portion of the frame t that contains repeated or overlapping imagery with frame t-N. In some embodiments, the comparison operation is performed on the segmented representations of frame t and frame t-N. In various embodiments, the comparison operation can be performed using various computer vision techniques, such as normalized correlation coefficient and normalized squared difference. The template matcher 904 identifies the remaining portion of the frame t - the portion of frame t starting after the portion of frame t that matches up with the template portion - as the part of frame t that contains new imagery (referred to herein as “non-repeated portion of frame / ”).

[0106] The segmentation-aware stitcher 906 combines the non-repeated portion of frame t with the part of the image of the bioprinted fiber 908 already stored in the image buffer. In particular, the non-repeated portion of frame t is combined with all the imagery captured from the images of the video up to frame t. In order to minimize artifacts and maintain the structural aspects of biomaterial depicted in the image of the bioprinted fiber 908, the segmentation-aware stitcher 906 determines the optimal seam between the non-repeated portion of frame t and the end of the partial image of the bioprinted fiber 908 stored in the image buffer. In practice, the optimal seam is determined between the beginning edge of the non-repeated portion of frame t and the ending edge of frame t-N. The optimal seam identifies the best path for stitching together the beginning edge of the non-repeated portion of frame t and the ending edge of frame t-N in order to minimize the visibility of seams and avoid cutting through regions of the frames that include cells, fragments, and / or cell aggregates.

[0107] In various embodiments, the segmentation-aware stitcher 906 determines the optimal seam using various computer vision techniques, such as dynamic programming seam finder, graph cut, etc. In one example, the segmentation-aware stitcher 906 computes a cost map for the region281105427751\1\AMERICASaround the beginning edge of the non-repeated portion of frame t and the ending edge of frame t- N. The cost map indicates, for each pixel in the region, a cost for placing a seam at that pixel. In various embodiments, the cost for a given pixel is higher when the pixel covers locations of the inner areas of cells, fragments, and / or cell aggregates. In various embodiments, the segmentation- aware stitcher 906 computes the cost map based on the segmented representations of the frame t and the frame t-N. The segmentation-aware stitcher 906 then algorithmically determines the path with the lowest cumulative cost in the cost map as the optimal seam for stitching together the beginning edge of the non-repeated portion of frame t and the ending edge of frame t-N. Once the optimal seam between the non-repeated portion of frame t and the end of the partial image of the bioprinted fiber 908 is determined, the segmentation-aware stitcher combines the non-repeated portion of frame t with the partial image based on the optimal seam and stores the updated image of the bioprinted fiber 908 in the image buffer.

[0108] In various embodiment, the stitched image of the bioprinted fiber 908 can be used for downstream operations, such as determining characteristics of the bioprinted fiber and / or of the bioprinting system. For example, the image of the bioprinted fiber 908 may be input as imaging data 402 into the segmentation engine 320, the results of which are processed by occlusion detection engine 324 and characterizing engine 326 to generate characteristics 414. These characteristics include, but are not limited to, a cell count (individual cells and / or cell aggregates), cell and / or cell aggregate morphology, cell aggregate area, volume, pIEQ (islet cell mass), diameter variations and spatial distribution of cell aggregates, and homogeneity of cell aggregates across a bioprinted fiber or a device generated from one or more bioprinted fibers. The characteristics of a bioprinted fiber or a device generated from one or more bioprinted fibers, as determined by the characterizing engine 326 can be used for many different purposes, including, but not limited to, determining a therapeutic dose associated with the printed fiber, generating training data to improve the performance of the machine learning module 160 with further training and modifications, and making adjustments to other aspects to bioprinting system 100 to improve the quality of bioprinted fibers.

[0109] In various embodiments, the techniques implemented by the frame stitching engine 322 can be for fiber speed estimation. For example, the template matching operations performed by the template matcher 904 can provide an estimate for the print speed of the moving fiber.291105427751\1\AMERICAS

[0110] FIG. 10A illustrates an example of frame stitching, according to various embodiments. As shown, the input video 1002 is a video a bioprinting process. The frame stitching engine 322 processes the input video 1002 to extract frames 1004. Furthermore, the segmentation engine 320, using one or more image segmentation machine learning models, generates segmented representations 1006 corresponding to the extracted frames 1004. The frame stitching engine 322 performs template matching and segmentation-aware stitching, as described above in conjunction with FIG. 9, to generate the stitched image fiber 1008. The frames in the stitched image fiber 1008 are combined in a manner that reduces or eliminates repeated imagery while also maintaining structural integrity of objects, such as cell aggregates, depicted in the frames.[0U1] FIG. 10B is a detailed illustration of the stitched image fiber 1008, according to various embodiments. As shown, the stitched image fiber 1008 comprises a series of video frame portions combined at seams identified by the frame stitching engine 322. Seam 1010 is an example of such a seam. Seam 1010 is generated by the frame stitching engine 322 in a manner that maintains cell aggregate structural integrity 1012 by selecting a path for the seam that avoids cutting through objects, such as cell aggregates, located at or around the seam.

[0112] FIG. 11 is a flow diagram of method steps for performing stitching of frames of a video capturing a bioprinting process, according to various embodiments. Although the method steps are described in conjunction with the systems of Figures 1-4 and 9, persons skilled in the art will understand that any system configured to perform the method steps in any order falls within the scope of the present disclosure.

[0113] As shown, in operation 1102 of method 1100, the machine learning module 160 receives video imaging data of a plurality of cells within a material flow. The video imaging data is captured during passage of the plurality of cells through the bioprinting system 100. In operation 1104, the machine learning module 160 extracts frames of the video imaging data using one or more frame extraction techniques. In operation 1106, the machine learning module 160 generates, using one or more machine learning based image segmentation operations, segmented representations of the frames that, for each frame, identifies a boundary around each region in the image frame that includes detected biological materials, such as a cell, a fragment or a cell aggregate. In operation 1108, the machine learning module 160, for each pair of consecutive301105427751\1\AMERICASframes, identifies a portion of the later frame that is to be removed because the portion overlaps with the previous frame and further identifies an optimal seam for combining the remaining portion of the frame with the previous frame. The optimal seam identifies the best path for stitching together the beginning edge of the non-overlapping portion of later frame and the ending edge of the previous frame in order to minimize the visibility of seams and avoid cutting through regions of the frames that include cells, fragments, and / or cell aggregates. In operation 1110, the machine learning module 160 stitches the frames at the identified seams to generate a stitched image of the entire length of the bioprinted fiber.

[0114] In various embodiment, the stitched image of the bioprinted fiber can be used for downstream operations, such as determining characteristics of the bioprinted fiber and / or of the bioprinting system. These characteristics include, but are not limited to, a cell count, cell aggregate morphology, cell aggregate area, volume, pIEQ (islet cell mass), diameter variations and spatial distribution of cell aggregates, and homogeneity of cell aggregates across a bioprinted fiber. The characteristics of a bioprinted fiber can be used for many different purposes, including, but not limited to, determining a therapeutic dose associated with the printed fiber, generating training data to improve the performance of the machine learning module 160 with further training and modifications, and making adjustments to other aspects to bioprinting system 100 to improve the quality of bioprinted fibers.

[0115] Example: Comparing nozzle video and post print full device (BTT) images for dose estimation using a sample print with beads.

[0116] FIG. 12 shows a partial screenshot of a stitched image of a bioprinted fiber from the nozzle video, while FIG. 13 shows an image of the resulting post-print full device, according to various embodiments. The following example discusses the use of both for dose estimation.

[0117] The ground truth may be acquired by visually counting the number of beads passing through the nozzle: Ground Truth for Total Bead Count = 2294. Further, each bead has radius of 35 microns denoting pIEQ of 0.102: Ground Truth for Total pIEQ = 233.1

[0118] Table 1 illustrates the results of dose estimation operations performed according to the disclosed embodiments.311105427751\1\AMERICASTABLE 1

[0119] Nozzle video stitching method has 94.5% agreement with ground truth pIEQ, and post print model has 94.2% agreement. Nozzle video stitching method has >99% agreement with ground truth bead count, and post print model has 93% agreement. A factor affecting post print bead count are the beads hidden in dark regions in edges of device where connected to frame. Stitched nozzle video is very accurate for count, but pIEQ calculation can be affected by camera optical distortion, as that directly changes the beads area, varies pixel scaling across the frame, and subsequently affects pIEQ calculation.

[0120] Overall, nozzle video and post-print BTT images both have >90% agreement with ground truth. Nozzle video has less occlusion compared to post print images of multi-layer devices. Postprint images have less optical distortion, and also can provide complementary information about the structure of BTT and cell distribution after settlement of the fiber post print. The two methods can provide complementary information and one or the other could be more accurate dependent on print settings. For example, for high density multi-layer printed devices, nozzle video has less occlusion and can better provide dose estimates. On the other hand, post-print image provides information such as device structural integrity, cell distribution across the device after print settlement, and also has less camera optical distortion compared to nozzle. As such these alternative approaches provide complementary information, and one or both may be more suited to specific print settings.321105427751\1\AMERICAS

[0121] Results: FIG. 14 illustrates the results of dose estimation of bioprinted fibres containing aggregated pancreatic islet cells, according to various embodiments. A total of 8 (n = 4 low dose, target 1484 pIEQ; n = 4 high dose, target 2448 pIEQ) bioprinted fibres containing aggregated pancreatic islet cells were produced and then analyzed using an established DNA assay and compared with microscopic image analysis using machine learning embodiments disclosed herein. Performing the DNA assay on the first cohort of bioprinted fibres (low dose, 1484 pIEQ target) resulted in an average dosage of 2096.43 pIEQ per sample. Dose estimation of the same samples based on ML tool analysis revealed an estimated average pIEQ dosage of 1701.35 per sample- a difference of 20.81%. In the second cohort (high dose, target 2448 pIEQ), the DNA assay revealed an average dosage estimation of 3753.90 pIEQ, whereas the ML tool estimated an average dosage of 3029.14 pIEQ- a difference of 21.37%.

[0122] FIG. 15 illustrates the results of cell counting for three BTT samples containing aggregated Retinal Pigment Epithelium (RPE-EPO) cells, according to various embodiments. The three BTT samples containing aggregated Retinal Pigment Epithelium (RPE-EPO) cells were bioprinted and subsequently imaged microscopically. For each sample, a total aggregate count was produced through the use of machine learning embodiments disclosed herein, which analyzed each image automatically and labeled each aggregate identified, resulting in a count of 2024, 2349, and 2005 for samples A3, A4, and A6 respectively. On the same set of images, a trained operator manually labelled each discernible aggregate to produce a second set of images and corresponding aggregate counts, resulting in manual counts of 2329, 2682, and 2383 for samples A3, A4, and A6 respectively. Comparison of the automated ML tool’s output against the manual process revealed a percentage difference of 14.01%, 23.24%, and 17.23% for each of samples A3, A4, and A6 respectively.

[0123] Material Flows:

[0124] Aspects of the bioprinting system 100 include material flows that can be used for printing structures for advantageous use as biomaterials. “Biomaterial” as used herein refers to a natural or synthetic substance that is useful for constructing or replacing tissue, e.g. human tissue with or without living cells. In the field of bioprinting, the term “biomaterial” is often synonymous with the term “bioink.”331105427751\1\AMERICAS

[0125] The material flow will generally comprise at least one cross-linkable material, e.g., hydrogels including but not limited to, alginate, chitosan, PEGDA, PEGTA, Hyaluronic acid (HA), HAMA, collagen, CollMA, gelatin, gelMA, agarose, gellan, fibrin (fibrinogen), PVA, and the like, or any combination thereof, as well as non-hydrogels including but not limited to, PCL, PLGA, PLA, and the like, or any combination thereof. In preferred embodiments the material flow comprises at least one hydrogel. Non-limiting examples of hydrogels include alginate, agarose, collagen, fibrinogen, gelatin, chitosan, hyaluronic acid-based gels, or any combination thereof. A variety of synthetic hydrogels are known and can be used in embodiments of the systems and methods provided herein. For example, in some embodiments, one or more hydrogels form at least part of the structural basis for three dimensional structures that are printed. In some embodiments, a hydrogel has the capacity to support growth and / or proliferation of one or more cell types, which may be dispersed within the hydrogel or added to the hydrogel after it has been printed in a three dimensional configuration.

[0126] In embodiments, a hydrogel is cross-linkable by a chemical cross-linking agent. For example, a hydrogel comprising alginate may be cross-linkable in the presence of a divalent cation such as calcium chloride (CaC12), a hydrogel containing chitosan may be cross-linked using a polyvalent anion such as sodium tripolyphosphate (STP), a hydrogel comprising fibrinogen may be cross-linkable in the presence of an enzyme such as thrombin, and a hydrogel comprising collagen, gelatin, agarose or chitosan may be cross-linkable in the presence of heat or a basic solution.

[0127] In some embodiments, a hydrogel comprises alginate. Alginate forms solidified colloidal gels (high water imagery gels, or hydrogels) when contacted with divalent cations. Any suitable divalent cation can be used to form a solidified hydrogel with an input material that comprises alginate. In the alginate ion affinity series Cd2+>Ba2+>Cu2+>Ca2+>Ni2+>Co2+>Mn2+, Ca2+ is the best characterized and most used to form alginate gels (Ouwerx, C. et al., Polymer Gels and Networks, 1998, 6(5):393-408). Studies indicate that Ca-alginate gels form via a cooperative binding of Ca2+ ions by poly G blocks on adjacent polymer chains, the so-called “egg-box” model (ISP Alginates, Section 3 : Algin-Manufacture and Structure, in Alginates: Products for Scientific Water Control, 2000, International Specialty Products: San Diego, pp. 4-7). G-rich alginates tend to form thermally stable, strong yet brittle Ca-gels, while M-rich alginates tend to form less341105427751\1\AMERICASthermally stable, weaker but more elastic gels. In some embodiments, a hydrogel comprises a depolymerized alginate.

[0128] In some embodiments, a hydrogel is cross-linkable using a free-radical polymerization reaction to generate covalent bonds between molecules. Free radicals can be generated by exposing a photoinitiator to light (often ultraviolet), or by exposing the hydrogel precursor to a chemical source of free radicals such as ammonium peroxodi sulfate (APS) or potassium peroxodi sulfate (KPS) in combination with N,N,N,N-Tetramethylethylenediamine (TEMED) as the initiator and catalyst respectively. Non-limiting examples of photo cross-linkable hydrogels include: methacrylated hydrogels, such as hyaluronic acid methacrylate (HAMA), gelatin methacrylate (GEL-MA) or polyethylene (glycol) acrylate-based (PEG-Acylate) hydrogels, which are used in cell biology due to their inertness to cells. Polyethylene glycol diacrylate (PEG-DA) is commonly used as scaffold in tissue engineering, since polymerization occurs rapidly at room temperature and requires low energy input, has high water imagery, is elastic, and can be customized to include a variety of biological molecules.

[0129] In embodiments, the material flow comprises a non-biodegradable polymer. In examples the input material may be a synthetic polymer, for example polyvinyl acetate (PVA). In embodiments, the material flow may comprise hyaluronic acid (HA).

[0130] In embodiments, the material flow comprises microparticles, “Microparticles” as used herein refers to immiscible particles in the range of about 0. lum to about lOOum that are typically composed of a polymer, a metal, or other inorganic material. They can be symmetrical (e.g. spherical, cubic, etc.) although this is not a requirement. Microparticles having an aspect ratio of 2: 1 or greater may be considered a microrod or microfiber.

[0131] Additional Components:

[0132] Material flows in accordance with embodiments of the invention can comprise any of a wide variety of natural or synthetic polymers that support the viability of living cells, including, e g., alginate, laminin, fibrin, hyaluronic acid, poly(ethylene) glycol based gels, gelatin, chitosan, agarose, or combinations thereof. In embodiments, the subject compositions are physiologically compatible, i.e., conducive to cell growth, differentiation and communication. In certain351105427751\1\AMERICASembodiments, an input material comprises one or more physiological matrix materials, or a combination thereof. By “physiological matrix material” is meant a biological material found in a native mammalian tissue. Non-limiting examples of such physiological matrix materials include: fibronectin, thrombospondin, glycosaminoglycans (GAG) (e.g., hyaluronic acid, chondroitin-6- sulfate, dermatan sulfate, chondroitin-4-sulfate, or keratin sulfate), deoxyribonucleic acid (DNA), adhesion glycoproteins, and collagen (e.g., collagen I, collagen II, collagen III, collagen IV, collagen V, collagen VI, or collagen XVIII).

[0133] Collagen gives most tissues tensile strength, and multiple collagen fibrils approximately 100 nm in diameter combine to generate strong coiled-coil fibers of approximately 10 pm in diameter. Biomechanical function of certain tissue constructs is conferred via collagen fiber alignment in an oriented manner. In some embodiments, an input material comprises collagen fibrils. An input material comprising collagen fibrils can be used to create a fiber structure that is formed into a tissue construct. By modulating the diameter of the fiber structure, the orientation of the collagen fibrils can be controlled to direct polymerization of the collagen fibrils in a desired manner.

[0134] In embodiments, the plurality of cells is selected from the group comprising or consisting of a single-cell suspension, cell aggregates, cell spheroids, cell organoids, or combinations thereof. Flow materials in accordance with embodiments of the invention can incorporate any mammalian cell type, including but not limited to stem cells (e.g., embryonic stem cells, adult stem cells, induced pluripotent stem cells), germ cells, endoderm cells (e.g., lung, liver, pancreas, gastrointestinal tract, or urogenital tract cells), mesoderm cells (e.g., kidney, bone, muscle, endothelial, or heart cells), ectoderm cells (skin, nervous system, pituitary, or eye cells), stem cell- derived cells, or any combination thereof.

[0135] For example, a flow material can comprise cells from endocrine and exocrine glands including pancreas (alpha, beta, delta, epsilon, gamma), liver (hepatocyte, Kuppfer, stellate, sinusoidal endothelial cells, cholangiocytes), thyroid (Follicular cells), pineal gland (pinealocytes), pituitary gland (somatotropes, Lactotropes, gonadotropes, corticotropes, and thyrotropes), thymus (thymocytes, thymic epithelial cells, thymic stromal cells), adrenal gland (cortical cells, chromaffin cells), ovary (granulosa cells), testis (Leydig cells), gastrointestinal tract361105427751\1\AMERICAS(enteroendocrine cells - intestinal, gastric, pancreatic), fibroblasts, chondrocytes, meniscus fibrochondrocytes, bone marrow stromal (stem) cells, embryonic stem cells, mesenchymal stem cells, induced pluripotent stem cells, differentiated stem cells, tissue-derived cells, smooth muscle cells, skeletal muscle cells, cardiac muscle cells, epithelial cells, endothelial cells, myoblasts, chondroblasts, osteoblasts, osteoclasts, and any combinations thereof.

[0136] Cells can be obtained from donors from the same species as the recipient (allogenic), from a different species to the recipient (xenogeneic) or from recipients (autologous). Specifically, in embodiments, cells can be obtained from a suitable donor, such as a human or animal, or from the subject into which the cells are to be implanted. Mammalian species include, but are not limited to, humans, monkeys, dogs, cows, horses, pigs, sheep, goats, cats, mice, rabbits, and rats. In one embodiment, the cells are human cells. In other embodiments, the cells can be xenogeneic, e.g., derived from animals such as dogs, cats, horses, monkeys, or any other mammal.

[0137] In some embodiments, the at least one biological material comprises a plurality of cells expressing / secreting one or more endogenous biologically active agent(s), e.g., insulin, glucagon, ghrelin, pancreatic polypeptide, Factor VII, Factor VIII, Factor IX, alpha- 1 -antitrypsin, an angiogenic factor, a growth factor, a hormone, an antibody, an enzyme, a protein, an exosome, and the like. Discussed herein, endogenous biologically active agents comprise those agents that the cell naturally produces in a biological context (e.g., insulin release in response to elevated glucose concentrations). An endogenous biologically active agent can constitute a therapeutic agent in the context of the present disclosure.

[0138] In some embodiments, a flow material can comprise genetically engineered cells that secrete specific factors. It is within the scope of this disclosure that a plurality of cells as discussed above can comprise, in embodiments, engineered cells (e.g., genetically engineered cells) that secrete specific factors. Cells can also be from established cell culture lines, or can be cells that have undergone genetic engineering and / or manipulation to achieve a desired genotype or phenotype. In some embodiments, pieces of tissue can also be used, which may provide a number of different cell types within the same structure.

[0139] Genetic engineering techniques applicable to the present disclosure can include but are not limited to recombinant DNA (rDNA) technology (Stryjewska et al., Pharmacol ogi al Reports.371105427751\1\AMERICAS2013; 65: 1075), cell-engineering based on use of targeted nucleases (e.g., meganuclease, zinc finger nucleases (ZFN), transcription activator-like effector nucleases (TALEN), clustered regularly interspaced short palindromic repeat-associated nuclease Cas9 (CRISPR-Cas9), etc. (Lim et al., Nature Communications. 2020; 11 : 4043; Stoddard BL, Structure. 2011; 19(1): 7-15; Gaj et al., Trends Biotechnol. 2013; 31(7): 397-405; Hsu et al., Cell. 2014; 157(6): 1262; Miller et al., Nat Biotechnol. 2010; 29(2): 143-148), cell-engineering based on use of site-specific recombination using recombinase systems (e.g., Cre-Lox) (Osborn et al., Mol Ther. 2013; 21(6): 1151-1159; Hockemeyer et al., Nat Biotechnol. 2009; 27(9): 851-857; Uhde-Stone et al., RNA. 2014; 20(6): 948-955; Ho et al., Nucleic Acids Res. 2015; 43(3): el7; Sengupta et al., Journal of Biological Engineering. 2017; 11(45): 1-9), and the like. In some embodiments, some combination of the above-mentioned techniques for cell-engineering may be used.

[0140] Encompassed by the present disclosure are engineered cells capable of producing one or more therapeutic agents, including but not limited to proteins, peptides, nucleic acids (e.g., DNA, RNA, mRNA, siRNA, miRNA, nucleic acid analogs), peptide nucleic acids, aptamers, antibodies or fragments or portions thereof, antigens or epitopes, hormones, hormone antagonists, growth factors or recombinant growth factors and fragments and variants thereof, cytokines, enzymes, antibiotics or antimicrobial compounds, anti-inflammation agent, antifungals, antivirals, toxins, prodrugs, small molecules, drugs (e.g., drugs, dyes, amino acids, vitamins, antioxidants) or any combination thereof.

[0141] In embodiments, the printhead may comprise one or more transparent channels, and an optical system may monitor material flow through at least one of the one or more transparent channels, preferably wherein the printhead comprises a transparent nozzle or dispensing channel. In embodiments, the printhead may be an extrusion printhead or, a co-extrusion printhead, including but not limited to multi-material extrusion, multi-axial extrusion, or melt-extrusion, an inkjet printhead, an electrowriting printhead, or a microfluidic printhead, including but not limited to on-chip crosslinking, microfluidic stereolithography or microfluidic spinning.

[0142] Generally, the printhead will comprise one or more inputs fluidically connected to one or more reservoirs, one or more dispensing channels for depositing the material flow onto the print surface, and preferably one or more transparent channels or regions. The cross-linkable material381105427751\1\AMERICASwithin the material flow may be cross-linked within the printhead, while dispensing from the printhead, and / or after deposition on the print surface, as is known in the art. The print surface may further comprise, e.g. a cross-linker bath.

[0143] In embodiments, the optical system may comprise one or more microscopes, one or more still cameras, and / or one or more video cameras. There may be an optical system at each stage of the bioprinting system, from the reservoir to the printhead to the print surface.

[0144] Any and all combinations of any of the claim elements recited in any of the claims and / or any elements described in this application, in any fashion, fall within the contemplated scope of the present invention and protection.

[0145] The descriptions of the various embodiments have been presented for purposes of illustration but are not intended to be exhaustive or limited to the embodiments disclosed. Many modifications and variations will be apparent to those of ordinary skill in the art without departing from the scope and spirit of the described embodiments.

[0146] Aspects of the present embodiments may be embodied as a system, method or computer program product. Accordingly, aspects of the present disclosure may take the form of an entirely hardware embodiment, an entirely software embodiment (including firmware, resident software, micro-code, etc.) or an embodiment combining software and hardware aspects that may all generally be referred to herein as a “module,” a “system,” or a “computer.” In addition, any hardware and / or software technique, process, function, component, engine, module, or system described in the present disclosure may be implemented as a circuit or set of circuits. Furthermore, aspects of the present disclosure may take the form of a computer program product embodied in one or more computer readable medium(s) having computer readable program code embodied thereon.

[0147] Any combination of one or more computer readable medium(s) may be utilized. The computer readable medium may be a computer readable signal medium or a computer readable storage medium. A computer readable storage medium may be, for example, but not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any suitable combination of the foregoing. More specific examples (a non-exhaustive391105427751\1\AMERICASlist) of the computer readable storage medium would include the following: an electrical connection having one or more wires, a portable computer diskette, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or Flash memory), an optical fiber, a portable compact disc read-only memory (CD- ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing. In the context of this document, a computer readable storage medium may be any tangible medium that can contain or store a program for use by or in connection with an instruction execution system, apparatus, or device.

[0148] Aspects of the present disclosure are described above with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems) and computer program products according to embodiments of the disclosure. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions may be provided to a processor of a general-purpose computer, special purpose computer, or other programmable data processing apparatus to produce a machine. The instructions, when executed via the processor of the computer or other programmable data processing apparatus, enable the implementation of the functions / acts specified in the flowchart and / or block diagram block or blocks. Such processors may be, without limitation, general purpose processors, special-purpose processors, application-specific processors, or field- programmable gate arrays.

[0149] The flowchart and block diagrams in the figures illustrate the architecture, functionality, and operation of possible implementations of systems, methods and computer program products according to various embodiments of the present disclosure. In this regard, each block in the flowchart or block diagrams may represent a module, segment, or portion of code, which comprises one or more executable instructions for implementing the specified logical function(s). It should also be noted that, in some alternative implementations, the functions noted in the block may occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently, or the blocks may sometimes be executed in the reverse order, depending upon the functionality involved. It will also be noted that each block of the block diagrams and / or flowchart illustration, and combinations of blocks in the block401105427751\1\AMERICASdiagrams and / or flowchart illustration, can be implemented by special purpose hardware-based systems that perform the specified functions or acts, or combinations of special purpose hardware and computer instructions.

[0150] All patent and non-patent references cited in the present specification are hereby incorporated by reference in their entirety and for all purposes.

[0151] Aspects of the present invention are set out in the following clauses or any combination thereof:

[0152] CLAUSE 1 :A method comprising: receiving imaging data of a plurality of cells within a material flow while passing through and / or exiting from a bioprinting system, wherein the passage of the plurality of cells through the bioprinting system creates a printed fiber; identifying a first frame in the imaging data captured at time t-n and a second frame in the imaging data captured at time / ; generating, via a machine learning algorithm executing on the imaging data, a plurality of segmented regions in the imaging data, wherein each segmented region in the plurality of segmented regions defines a boundary around a different set of cell aggregates or cells in the plurality of cells; determining, for combining with the first frame and based on the imaging data and / or the plurality of segmented regions, a portion of the second frame that is nonoverlapping with the first frame; determining, based on the plurality of segmented regions, one or more cells or cell aggregates in the plurality of cells that are located along an edge of the first frame or an edge of the portion of the second frame; identifying a seam where the first frame and the portion of the second frame are to be combined based at least on locations of the one or more cells or cell aggregates; combining the first frame and the portion of the second frame along the seam to generate at least a portion of a stitched image of the printed fiber; and411105427751\1\AMERICASdetermining one or more parameters associated with the plurality of cells, the material flow, or the bioprinting system based on the stitched image.

[0153] CLAUSE 2: The method of claim 1, further comprising identifying a template portion of the first frame, wherein determining the portion of the second frame comprises matching the second frame to the template portion of the first frame.

[0154] CLAUSE 3: The method of claim 2, wherein the template portion is identified based on a pre-determined overlap width.

[0155] CLAUSE 4: The method of claim 2, wherein the template portion is identified based on a configuration of the bioprinting system.

[0156] CLAUSE 5: The method of claim 2, further comprising generating a first segmentation mask of the first frame and a second segmentation mask of the second frame, wherein matching the second frame to the template portion of the first frame comprises comparing the second segmentation mask to a portion of the first segmentation mask corresponding to the template portion.

[0157] CLAUSE 6: The method of claim 2, wherein matching the second frame to the template portion of the first frame comprises performing one or more of normalized correlation coefficient matching or normalized square difference between pixels in the template portion image and / or segmentation mask and pixels in the second frame image and / or segmentation mask.

[0158] CLAUSE 7: The method of claim 1, wherein identifying the seam where the first frame and the portion of the second frame are to be combined comprises determining the locations of the one or more cells or cell aggregates based on segmented regions in the plurality of segmented regions corresponding to the first frame and the portion of the second frame.

[0159] CLAUSE 8: The method of claim 1, wherein identifying the seam where the first frame and the portion of the second frame are to be combined comprises determining an optimal path along an edge of the first frame and an edge of the portion of the second frame that avoids passing through the one or more cells or cell aggregates.421105427751\1\AMERICAS

[0160] CLAUSE 9: The method of claim 1, wherein the bioprinting system comprises a printhead with one or more transparent channels, and the method further comprising collecting, with an optical system included in the bioprinting system, the imaging data through at least one of the one or more transparent channels.

[0161] CLAUSE 10: The method of claim 9, wherein the printhead comprises a transparent nozzle or dispensing channel.

[0162] CLAUSE 11 : The method of claim 1, further comprising: collecting the imaging data with an optical system included in the bioprinting system, wherein the optical system includes one or more of a microscope, a still camera, or a video camera; and positioning the optical system at different positions in the bioprinting system to collect the imaging data during passage of the plurality of cells through the bioprinting system.

[0163] CLAUSE 12: The method of claim 11, wherein the positioning comprises positioning one or more imaging devices at each of an input reservoir, a printhead, and a print surface included in the bioprinting system.

[0164] CLAUSE 13: The method of claim 1, wherein the machine learning algorithm comprises a neural network selected from a group consisting of a convolutional neural network (CNN), a region-based CNN (R-CNN), a Mask R-CNN, a you only look once (YOLO) based model, deformable convolutional network based model, and a transformer-based instance segmentation model, to perform image segmentation on the imaging data.

[0165] CLAUSE 14: The method of claim 1, wherein the plurality of segmented regions are generated via amodal panoptic segmentation, semantic segmentation, instance segmentation, occlusion-aware segmentation, image inpainting, or any combination thereof.

[0166] CLAUSE 15: The method of claim 1, wherein determining one or more parameters comprises adjusting one or more configurations of the bioprinting system.431105427751\1\AMERICAS

[0167] CLAUSE 16: The method of claim 15, wherein the one or more configurations include a flow of cross-linkable material through a printhead, a motion profile of cross-linkable material, or compositions of biomaterial input in the bioprinting system.

[0168] CLAUSE 17: The method of claim 1, wherein determining the one or more parameters comprises calculating a therapeutic dose.

[0169] CLAUSE 18: The method of claim 1, further comprising determine at least one of aggregate shape, diameter variations, spatial distribution of aggregates, or a homogeneity of aggregates within the material flow based on the stitched image.

[0170] CLAUSE 19: The method of claim 1, further comprising determining a printing speed associated with the bioprinting system based on at least the portion of the second frame that is nonoverlapping with the first frame.

[0171] CLAUSE 20:A bioprinting system comprising: a bioprinter, optionally comprising one or more reservoirs, a printhead, and a print surface on which the bioprinting is dispensed; an optical system to collect imaging data of a plurality of cells within a material flow while passing through and / or exiting from the bioprinter, wherein the passage of the plurality of cells through the bioprinter creates a printed fiber; and a computing system that performs the steps of: identifying a first frame in the imaging data captured at time t-n and a second frame in the imaging data captured at time / ; generating, via a machine learning algorithm executing on the imaging data, a plurality of segmented regions in the imaging data, wherein each segmented region in the plurality of segmented regions defines a boundary around a different set of cell aggregates or cells in the plurality of cells; determining, for combining with the first frame and based on the plurality of segmented regions, a portion of the second frame that is non-overlapping with the first frame;441105427751\1\AMERICASdetermining, based on the plurality of segmented regions, one or more cells or cell aggregates in the plurality of cells that are located along an edge of the first frame or an edge of the portion of the second frame; identifying a seam where the first frame and the portion of the second frame are to be combined based at least on locations of the one or more cells or cell aggregates; combining the first frame and the portion of the second frame along the seam to generate at least a portion of a stitched image of the printed fiber; and determining one or more parameters associated with the plurality of cells, the material flow, or the bioprinting system based on the stitched image.

[0172] CLAUSE 21: The bioprinting system of claim 20, further comprising identifying a template portion of the first frame, wherein determining the portion of the second frame comprises matching the second frame to the template portion of the first frame.

[0173] CLAUSE 22: The bioprinting system of claim 21, wherein the template portion is identified based on a pre-determined overlap width.

[0174] CLAUSE 23: The bioprinting system of claim 21, wherein the template portion is identified based on a configuration of the bioprinting system.

[0175] CLAUSE 24: The bioprinting system of claim 21, further comprising generating a first segmentation mask of the first frame and a second segmentation mask of the second frame, wherein matching the second frame to the template portion of the first frame comprises comparing the second segmentation mask to a portion of the first segmentation mask corresponding to the template portion.

[0176] CLAUSE 25: The bioprinting system of claim 21, wherein matching the second frame to the template portion of the first frame comprises performing one or more of normalized correlation coefficient matching or normalized square difference between pixels in the template portion image and / or segmentation mask and pixels in the second frame image and / or segmentation mask.

[0177] CLAUSE 26: The bioprinting system of claim 20, wherein identifying the seam where the first frame and the portion of the second frame are to be combined comprises determining the451105427751\1\AMERICASlocations of the one or more cells or cell aggregates based on segmented regions in the plurality of segmented regions corresponding to the first frame and the portion of the second frame.

[0178] CLAUSE 27: The bioprinting system of claim 20, wherein identifying the seam where the first frame and the portion of the second frame are to be combined comprises determining an optimal path along an edge of the first frame and an edge of the portion of the second frame that avoids passing through the one or more cells or cell aggregates.

[0179] CLAUSE 28: The bioprinting system of claim 20, wherein the bioprinting system comprises a printhead with one or more transparent channels, and the method further comprising collecting, with an optical system included in the bioprinting system, the imaging data through at least one of the one or more transparent channels.

[0180] CLAUSE 29: The bioprinting system of claim 28, wherein the printhead comprises a transparent nozzle or dispensing channel.

[0181] CLAUSE 30: The bioprinting system of claim 20, further comprising: collecting the imaging data with an optical system included in the bioprinting system, wherein the optical system includes one or more of a microscope, a still camera, or a video camera; and positioning the optical system at different positions in the bioprinting system to collect the imaging data during passage of the plurality of cells through the bioprinting system.

[0182] CLAUSE 31 : The bioprinting system of claim 30, wherein the positioning comprises positioning one or more imaging devices at each of an input reservoir, a printhead, and a print surface included in the bioprinting system.

[0183] CLAUSE 32: The bioprinting system of claim 20, wherein the machine learning algorithm comprises a neural network selected from a group consisting of a convolutional neural network (CNN), a region-based CNN (R-CNN), a Mask R-CNN, a you only look once (YOLO) based model, deformable convolutional network based model, and a transformer-based instance segmentation model, to perform image segmentation on the imaging data.461105427751\1\AMERICAS

[0184] CLAUSE 33: The bioprinting system of claim 20, wherein the plurality of segmented regions are generated via amodal panoptic segmentation, semantic segmentation, instance segmentation, occlusion-aware segmentation, image inpainting, or any combination thereof.

[0185] CLAUSE 34: The bioprinting system of claim 20, wherein determining one or more parameters comprises adjusting one or more configurations of the bioprinting system.

[0186] CLAUSE 35: The bioprinting system of claim 34, wherein the one or more configurations include a flow of cross-linkable material through a printhead, a motion profile of cross-linkable material, or compositions of biomaterial input in the bioprinting system.

[0187] CLAUSE 36: The bioprinting system of claim 20, wherein determining the one or more parameters comprises calculating a therapeutic dose.

[0188] CLAUSE 37: The bioprinting system of claim 20, further comprising determining at least one of aggregate shape, diameter variations, spatial distribution of aggregates, or a homogeneity of aggregates within the material flow based on the stitched image.

[0189] CLAUSE 38:One or more computer-readable media storing instructions that, when executed by one or more computing devices, cause the one or more computing devices to perform the steps of: receiving imaging data of a plurality of cells within a material flow while passing through and / or exiting from a bioprinting system, wherein the passage of the plurality of cells through the bioprinting system creates a printed fiber; identifying a first frame in the imaging data captured at time l-n and a second frame in the imaging data captured at time / ; generating, via a machine learning algorithm executing on the imaging data, a plurality of segmented regions in the imaging data, wherein each segmented region in the plurality of segmented regions defines a boundary around a different set of cell aggregates or cells in the plurality of cells; determining, for combining with the first frame and based on the plurality of segmented regions, a portion of the second frame that is non-overlapping with the first frame;471105427751\1\AMERICASdetermining, based on the plurality of segmented regions, one or more cells or cell aggregates in the plurality of cells that are located along an edge of the first frame or an edge of the portion of the second frame; identifying a seam where the first frame and the portion of the second frame are to be combined based at least on locations of the one or more cells or cell aggregates; combining the first frame and the portion of the second frame along the seam to generate at least a portion of a stitched image of the printed fiber; and determining one or more parameters associated with the plurality of cells, the material flow, or the bioprinting system based on the stitched image.

[0190] CLAUSE 39: The one or more non-computer readable media of claim 38, wherein identifying the seam where the first frame and the portion of the second frame are to be combined comprises determining an optimal path along an edge of the first frame and an edge of the portion of the second frame that avoids passing through the one or more cells or cell aggregates.481105427751\1\AMERICAS

Claims

CLAIMS1. A method comprising: receiving imaging data of a plurality of cells within a material flow while passing through and / or exiting from a bioprinting system, wherein the passage of the plurality of cells through the bioprinting system creates a printed fiber; identifying a first frame in the imaging data captured at time t-n and a second frame in the imaging data captured at time / ; generating, via a machine learning algorithm executing on the imaging data, a plurality of segmented regions in the imaging data, wherein each segmented region in the plurality of segmented regions defines a boundary around a different set of cell aggregates or cells in the plurality of cells; determining, for combining with the first frame and based on the imaging data and / or the plurality of segmented regions, a portion of the second frame that is nonoverlapping with the first frame; determining, based on the plurality of segmented regions, one or more cells or cell aggregates in the plurality of cells that are located along an edge of the first frame or an edge of the portion of the second frame; identifying a seam where the first frame and the portion of the second frame are to be combined based at least on locations of the one or more cells or cell aggregates; combining the first frame and the portion of the second frame along the seam to generate at least a portion of a stitched image of the printed fiber; and determining one or more parameters associated with the plurality of cells, the material flow, or the bioprinting system based on the stitched image.

2. The method of claim 1, further comprising identifying a template portion of the first frame, wherein determining the portion of the second frame comprises matching the second frame to the template portion of the first frame.

3. The method of claim 2, wherein the template portion is identified based on a predetermined overlap width.491105427751\1\AMERICAS4. The method of claim 2, wherein the template portion is identified based on a configuration of the bioprinting system.

5. The method of claim 2, further comprising generating a first segmentation mask of the first frame and a second segmentation mask of the second frame, wherein matching the second frame to the template portion of the first frame comprises comparing the second segmentation mask to a portion of the first segmentation mask corresponding to the template portion.

6. The method of claim 2, wherein matching the second frame to the template portion of the first frame comprises performing one or more of normalized correlation coefficient matching or normalized square difference between pixels in the template portion image and / or segmentation mask and pixels in the second frame image and / or segmentation mask.

7. The method of claim 1, wherein identifying the seam where the first frame and the portion of the second frame are to be combined comprises determining the locations of the one or more cells or cell aggregates based on segmented regions in the plurality of segmented regions corresponding to the first frame and the portion of the second frame.

8. The method of claim 1, wherein identifying the seam where the first frame and the portion of the second frame are to be combined comprises determining an optimal path along an edge of the first frame and an edge of the portion of the second frame that avoids passing through the one or more cells or cell aggregates.

9. The method of claim 1, wherein the bioprinting system comprises a printhead with one or more transparent channels, and the method further comprising collecting, with an optical system included in the bioprinting system, the imaging data through at least one of the one or more transparent channels.

10. The method of claim 9, wherein the printhead comprises a transparent nozzle or dispensing channel.

11. The method of claim 1, further comprising:501105427751\1\AMERICAScollecting the imaging data with an optical system included in the bioprinting system, wherein the optical system includes one or more of a microscope, a still camera, or a video camera; and positioning the optical system at different positions in the bioprinting system to collect the imaging data during passage of the plurality of cells through the bioprinting system.

12. The method of claim 11, wherein the positioning comprises positioning one or more imaging devices at each of an input reservoir, a printhead, and a print surface included in the bioprinting system.

13. The method of claim 1 , wherein the machine learning algorithm comprises a neural network selected from a group consisting of a convolutional neural network (CNN), a region-based CNN (R-CNN), a Mask R-CNN, a you only look once (YOLO) based model, deformable convolutional network based model, and a transformer-based instance segmentation model, to perform image segmentation on the imaging data.

14. The method of claim 1, wherein the plurality of segmented regions are generated via amodal panoptic segmentation, semantic segmentation, instance segmentation, occlusion-aware segmentation, image inpainting, or any combination thereof.

15. The method of claim 1, wherein determining one or more parameters comprises adjusting one or more configurations of the bioprinting system.

16. The method of claim 15, wherein the one or more configurations include a flow of crosslinkable material through a printhead, a motion profile of cross-linkable material, or compositions of biomaterial input in the bioprinting system.

17. The method of claim 1, wherein determining the one or more parameters comprises calculating a therapeutic dose.

18. The method of claim 1, further comprising determine at least one of aggregate shape, diameter variations, spatial distribution of aggregates, or a homogeneity of aggregates within the material flow based on the stitched image.

19. The method of claim 1, further comprising determining a printing speed associated with the bioprinting system based on at least the portion of the second frame that is non-overlapping with the first frame.511105427751\1\AMERICAS20. A bioprinting system comprising: a bioprinter, optionally comprising one or more reservoirs, a printhead, and a print surface on which the bioprinting is dispensed; an optical system to collect imaging data of a plurality of cells within a material flow while passing through and / or exiting from the bioprinter, wherein the passage of the plurality of cells through the bioprinter creates a printed fiber; and a computing system that performs the steps of: identifying a first frame in the imaging data captured at time l-n and a second frame in the imaging data captured at time / ; generating, via a machine learning algorithm executing on the imaging data, a plurality of segmented regions in the imaging data, wherein each segmented region in the plurality of segmented regions defines a boundary around a different set of cell aggregates or cells in the plurality of cells; determining, for combining with the first frame and based on the plurality of segmented regions, a portion of the second frame that is non-overlapping with the first frame; determining, based on the plurality of segmented regions, one or more cells or cell aggregates in the plurality of cells that are located along an edge of the first frame or an edge of the portion of the second frame; identifying a seam where the first frame and the portion of the second frame are to be combined based at least on locations of the one or more cells or cell aggregates; combining the first frame and the portion of the second frame along the seam to generate at least a portion of a stitched image of the printed fiber; and determining one or more parameters associated with the plurality of cells, the material flow, or the bioprinting system based on the stitched image.

21. The bioprinting system of claim 20, further comprising identifying a template portion of the first frame, wherein determining the portion of the second frame comprises matching the second frame to the template portion of the first frame.

22. The bioprinting system of claim 21, wherein the template portion is identified based on a pre-determined overlap width.521105427751\1\AMERICAS23. The bioprinting system of claim 21, wherein the template portion is identified based on a configuration of the bioprinting system.

24. The bioprinting system of claim 21, further comprising generating a first segmentation mask of the first frame and a second segmentation mask of the second frame, wherein matching the second frame to the template portion of the first frame comprises comparing the second segmentation mask to a portion of the first segmentation mask corresponding to the template portion.

25. The bioprinting system of claim 21, wherein matching the second frame to the template portion of the first frame comprises performing one or more of normalized correlation coefficient matching or normalized square difference between pixels in the template portion image and / or segmentation mask and pixels in the second frame image and / or segmentation mask.

26. The bioprinting system of claim 20, wherein identifying the seam where the first frame and the portion of the second frame are to be combined comprises determining the locations of the one or more cells or cell aggregates based on segmented regions in the plurality of segmented regions corresponding to the first frame and the portion of the second frame.

27. The bioprinting system of claim 20, wherein identifying the seam where the first frame and the portion of the second frame are to be combined comprises determining an optimal path along an edge of the first frame and an edge of the portion of the second frame that avoids passing through the one or more cells or cell aggregates.

28. The bioprinting system of claim 20, wherein the bioprinting system comprises a printhead with one or more transparent channels, and the method further comprising collecting, with an optical system included in the bioprinting system, the imaging data through at least one of the one or more transparent channels.

29. The bioprinting system of claim 28, wherein the printhead comprises a transparent nozzle or dispensing channel.

30. The bioprinting system of claim 20, further comprising:531105427751\1\AMERICAScollecting the imaging data with an optical system included in the bioprinting system, wherein the optical system includes one or more of a microscope, a still camera, or a video camera; and positioning the optical system at different positions in the bioprinting system to collect the imaging data during passage of the plurality of cells through the bioprinting system.

31. The bioprinting system of claim 30, wherein the positioning comprises positioning one or more imaging devices at each of an input reservoir, a printhead, and a print surface included in the bioprinting system.

32. The bioprinting system of claim 20, wherein the machine learning algorithm comprises a neural network selected from a group consisting of a convolutional neural network (CNN), a region-based CNN (R-CNN), a Mask R-CNN, a you only look once (YOLO) based model, deformable convolutional network based model, and a transformer-based instance segmentation model, to perform image segmentation on the imaging data.

33. The bioprinting system of claim 20, wherein the plurality of segmented regions are generated via amodal panoptic segmentation, semantic segmentation, instance segmentation, occlusion-aware segmentation, image inpainting, or any combination thereof.

34. The bioprinting system of claim 20, wherein determining one or more parameters comprises adjusting one or more configurations of the bioprinting system.

35. The bioprinting system of claim 34. wherein the one or more configurations include a flow of cross-linkable material through a printhead, a motion profile of cross-linkable material, or compositions of biomaterial input in the bioprinting system.

36. The bioprinting system of claim 20, wherein determining the one or more parameters comprises calculating a therapeutic dose.

37. The bioprinting system of claim 20, further comprising determining at least one of aggregate shape, diameter variations, spatial distribution of aggregates, or a homogeneity of aggregates within the material flow based on the stitched image.

38. One or more computer-readable media storing instructions that, when executed by one or more computing devices, cause the one or more computing devices to perform the steps of541105427751\1\AMERICASreceiving imaging data of a plurality of cells within a material flow while passing through and / or exiting from a bioprinting system, wherein the passage of the plurality of cells through the bioprinting system creates a printed fiber; identifying a first frame in the imaging data captured at time t-n and a second frame in the imaging data captured at time / ; generating, via a machine learning algorithm executing on the imaging data, a plurality of segmented regions in the imaging data, wherein each segmented region in the plurality of segmented regions defines a boundary around a different set of cell aggregates or cells in the plurality of cells; determining, for combining with the first frame and based on the plurality of segmented regions, a portion of the second frame that is non-overlapping with the first frame; determining, based on the plurality of segmented regions, one or more cells or cell aggregates in the plurality of cells that are located along an edge of the first frame or an edge of the portion of the second frame; identifying a seam where the first frame and the portion of the second frame are to be combined based at least on locations of the one or more cells or cell aggregates; combining the first frame and the portion of the second frame along the seam to generate at least a portion of a stitched image of the printed fiber; and determining one or more parameters associated with the plurality of cells, the material flow, or the bioprinting system based on the stitched image.

39. The one or more non-computer readable media of claim 38, wherein identifying the seam where the first frame and the portion of the second frame are to be combined comprises determining an optimal path along an edge of the first frame and an edge of the portion of the second frame that avoids passing through the one or more cells or cell aggregates.551105427751\1\AMERICAS

Images