Bioprinting methods and systems

Abstract

In the bioprinting system and method, the bioprinter may include one or more reservoirs, a print head, and a print surface from which the bioprinting material is ejected. An optical system may collect imaging data of a plurality of cells within a crosslinkable material before, during, and / or after the plurality of cells pass through the bioprinting system. A computing system may process the imaging data to determine the amount, distribution, concentration, and / or morphology of the plurality of cells. A machine learning system may compare characteristics of the plurality of cells before, during, and / or after the plurality of cells pass through the bioprinting system to user-established cell morphologies that correspond to physical characteristics of the cells and / or cell aggregates within predetermined tolerances. Additionally or alternatively, the machine learning system may compare characteristics of the plurality of cells at one stage of the bioprinting system to characteristics of the plurality of cells at another stage of the bioprinting system.

Description

[Technical Field]

[0001] CROSS-REFERENCE TO RELATED APPLICATIONS This application claims the benefit of U.S. Provisional Patent Application No. 63 / 449,291, entitled "BIOPRINTING METHODS AND SYSTEMS," filed March 1, 2023, the entire contents of which are incorporated herein by reference.

[0002]

[0003] Aspects of the present invention relate to quality control methods and apparatus for monitoring three-dimensional (3D) bioprinting systems. More particular aspects relate to methods and apparatus for monitoring different stages of a 3D bioprinting system. Even more particular aspects relate to computer vision and machine learning systems that provide real-time visual images of multiple cells at different locations within a bioprinting platform to assess various properties of the multiple cells, including, but not limited to, quantity, distribution, concentration, quality, and / or morphology of the multiple cells. [Background technology]

[0003] The 3D bioprinting process involves multiple stages, including a pre-printing stage, a printing stage, and a post-printing stage. It would be desirable to provide a system capable of monitoring different stages of the 3D bioprinting process to assess how operation of the various stages may affect print quality. Such a system would enable improvements in operation of each of the various stages, thereby facilitating the production of 3D bioprinted tissues suitable for therapeutic applications. Summary of the Invention

[0004] The present invention addresses the aforementioned problems in the art by integrating computer vision and deep learning into a three-dimensional (3D) bioprinting system, thereby enabling direct monitoring of the manipulation and flow of multiple different materials at different stages within the bioprinting system. In embodiments, the materials may include, for example, cell-containing hydrogels and other crosslinkable materials. In embodiments, the different stages of the bioprinting system may include one or more reservoirs, print heads, and print surfaces.

[0005] Embodiments of the present invention enable non-contact sensing of properties of multiple cells as they flow through a bioprinting system before, during, and / or after printing. Embodiments also enable automated analysis, which enables derivation of quantitative measures of properties of multiple cells. These measures may serve as quality control and / or quality assurance parameters. In some embodiments, such quantitative measures include real-time, high-level quantification of biological material (e.g., cells) before, during, and / or after printing to qualitatively assess the quality of the cells with respect to the consistency of the biological material. In some embodiments, such quantitative measures include real-time, high-level quantification of other objects in the material flow.

[0006] In embodiments, the material flow comprises at least one crosslinkable material, and preferably at least one hydrogel, and optionally further comprises a plurality of cells in at least one biological material, such as a biocompatible material. In embodiments, the plurality of cells comprises or is selected from the group consisting of a single cell suspension, a cell aggregate, a cell cluster, a cell aggregate particle, a cell spheroid, a cell organoid, or a combination thereof. In embodiments, the material flow comprises microparticles.

[0007] In embodiments, the plurality of cells comprises cells from endocrine and exocrine glands selected from the group consisting of pancreas, liver, thyroid, parathyroid, pineal gland, pituitary gland, thymus, adrenal gland, ovary, testis, enteroendocrine cells, stem cells, stem cell-derived cells, or cells engineered to secrete a bioactive substance of interest. In embodiments, the plurality of cells releases cell-derived extracellular vesicles in the form of exosomes containing a therapeutic protein or nucleic acid. In embodiments, the biocompatible material is selected from alginate, collagen, decellularized extracellular matrix, hyaluronic acid, PEG, fibrin, gelatin, GEL-MA, silk, chitosan, cellulose, PCL, PLA, POEGMA, and combinations thereof.

[0008] In embodiments, the printhead can include one or more transparent channels, and an optical system can monitor material flow through at least one of the one or more transparent channels, preferably the printhead includes a transparent nozzle or ejection channel. In embodiments, the printhead can be an extrusion printhead, including but not limited to multi-material extrusion, multi-screw extrusion, or melt extrusion, or a co-extrusion printhead, an inkjet printhead, an electrowriting printhead, or a microfluidic printhead, including but not limited to on-chip crosslinking, microfluidic stereolithography, or microfluidic spinning.

[0009] Generally, the print head includes one or more inputs fluidly connected to one or more reservoirs, one or more ejection channels for depositing the material flow onto the print surface, and preferably one or more transparent channels or regions. Crosslinkable materials in the material flow may be crosslinked within the print head during ejection from the print head and / or after deposition onto the print surface, as known in the art. The print surface may further include, for example, a crosslinking bath.

[0010] In embodiments, the optical system may include one or more microscopes, one or more still cameras, and / or one or more video cameras. There may be an optical system at each stage of the bioprinting system, from the reservoir to the print head to the print surface.

[0011] In an exemplary embodiment, the print head may include multiple transparent channels, and the optical system may include an equal number of microscopes, still cameras, and / or video cameras.

[0012] In embodiments, the machine learning system identifies one or more deviations in the flow of the plurality of cells from user-established material flow parameters at each of the plurality of stages as the plurality of cells pass through the bioprinting system. In embodiments, in response to the identified one or more deviations, the machine learning system identifies whether the material flow parameters need to be adjusted. In embodiments, the machine learning system can adjust the material flow parameters in response to cumulative deviations exceeding a predetermined amount. In embodiments, the machine learning system adjusts the material flow parameters to maintain suitable physical properties of the plurality of cells within a predetermined tolerance.

[0013] In embodiments, the machine learning system performs panoptic and / or semantic and / or instance segmentation of streaming images of the flow of a plurality of cells through the bioprinting system. In embodiments, the panoptic and / or semantic and / or instance segmentation enables detection of the location of one or more objects within the material flow. In embodiments, the panoptic and / or semantic and / or instance segmentation enables visual estimation of the shape and / or size of one or more objects within the material flow. In embodiments, the panoptic and / or semantic and / or instance segmentation enables visual estimation of the approximate amount and / or distribution of biological material (e.g., a plurality of cells) within the material flow.

[0014] In embodiments, the device includes a three-dimensional (3D) bioprinting print head, in embodiments, the 3D bioprinting print head includes a plurality of channels for selectively providing respective ones of a plurality of materials to a material flow.

[0015] In embodiments, the at least one crosslinkable material comprises a hydrogel. In embodiments, the material flow further comprises at least one biological material, preferably the at least one biological material comprises a plurality of cells. In embodiments, the plurality of cells comprises or is selected from the group consisting of a single cell suspension, cell aggregates, cell clusters, cell aggregate particles, cell spheroids, cell organoids, or a combination thereof. In embodiments, the material flow further comprises a microparticle. In embodiments, the material flow further comprises a dye, pigment, or colloid. In embodiments, the presence of cells in the material flow serves as a contrast agent to facilitate measurement of physical properties of the bioprinted fibers.

[0016] In another embodiment, the computing system uses the results of the comparison to control material flow by adjusting the pressure of the material flow in the bioprinting system, hi an embodiment, the system further includes a pressure controller responsive to the results of the comparison to control the material flow and pressure through the print head during printing of the printed fibers.

[0017] In embodiments, the machine learning system may include a neural network selected from the group consisting of a convolutional neural network (CNN), a fully convolutional neural network (FCN), a region-based CNN (R-CNN), a Mask R-CNN, a you-only-look-once (YOLO)-based model, and a transformer-based instance segmentation model. Additionally, in embodiments, flow estimation networks may be used, including, but not limited to, the FlowNet family of flow estimation networks (e.g., FlowNet, FlowNet 2), recursive all-field transform (RAFT), and self-taught multi-frame unsupervised RAFT with full image warping (SMURF). Furthermore, in embodiments, a technology known as SegFlow may be used for integrated prediction of cell segmentation and flow estimation.

[0018] In embodiments, determining further includes using a machine learning system to perform panoptic and / or semantic and / or instance segmentation of the streaming image of the material flow. In embodiments, the panoptic and / or semantic and / or instance segmentation enables identification of cells, fragments, and aggregates within the material flow. In embodiments, the panoptic and / or semantic and / or instance segmentation enables visual estimation of the shape and / or size of one or more objects within the material flow. In embodiments, the panoptic and / or semantic and / or instance segmentation enables visual estimation of the approximate amount and / or distribution of biological material (e.g., multiple cells) within the material flow.

[0019] In embodiments, acquiring includes acquiring streaming images through one or more transparent channels in a three-dimensional (3D) bioprinting print head within the device, the 3D bioprinting print head producing the bioprinted fibers. In embodiments, monitoring includes monitoring a plurality of channels within the 3D bioprinting print head, the plurality of channels selectively providing respective ones of the plurality of materials to the material flow.

[0020] The accompanying drawings, which are incorporated in and form a part of this specification, illustrate embodiments of the present disclosure and, together with the description, further serve to explain the principles of the disclosure and to enable those skilled in the art to make and use the disclosure. [Brief explanation of the drawings]

[0021] [Figure 1] FIG. 1 illustrates a high-level diagram of a bioprinting system according to an embodiment. [Figure 2A]1 is a high-level flowchart illustrating different stages before, during, and after the passage of a plurality of cells through different stages of a bioprinting system according to an embodiment. [Figure 2B] 1 is a high-level flowchart illustrating different stages before, during, and after the passage of a plurality of cells through different stages of a bioprinting system according to an embodiment. [Figure 3A] 1 illustrates the characterization of different geometric and characteristic features of cells, fragments, and aggregates, according to one embodiment. [Figure 3B] 1 illustrates the characterization of different geometric and characteristic features of cells, fragments, and aggregates, according to one embodiment. [Figure 3C] 1 illustrates the characterization of different geometric and characteristic features of cells, fragments, and aggregates, according to one embodiment. [Figure 3D] 1 illustrates the characterization of different geometric and characteristic features of cells, fragments, and aggregates, according to one embodiment. [Figure 3E] 1 illustrates the characterization of different geometric and characteristic features of cells, fragments, and aggregates, according to one embodiment. [Figure 3F] 1 illustrates the characterization of different geometric and characteristic features of cells, fragments, and aggregates, according to one embodiment. [Figure 4] 1 is a flow chart illustrating a method for differentiating between cells, fragments, and aggregates, according to one embodiment. [Figure 5] 1 illustrates multiple microwells for characterization of multiple cells before printing, according to one embodiment. [Figure 6A] 1 illustrates the application of cell, fragment, and aggregate characterization according to one embodiment. [Figure 6B] 1 illustrates the application of cell, fragment, and aggregate characterization according to one embodiment. [Figure 6C] 1 illustrates the application of cell, fragment, and aggregate characterization according to one embodiment. [Figure 6D]1 illustrates the application of cell, fragment, and aggregate characterization according to one embodiment. [Figure 7A] 1 illustrates the detection of aggregates, according to one embodiment. [Figure 7B] 10 shows the detection of both cells and aggregates according to another embodiment. [Figure 8] 1A and 1B show characterization of cells, fragments, and aggregates before printing, according to one embodiment. [Figure 9] 1A and 1B show characterization of cells, fragments, and aggregates during printing, according to one embodiment. [Figure 10A] 10A-10C illustrate characterization of cells, fragments, and aggregates after printing, according to an embodiment. [Figure 10B] 10A-10C illustrate characterization of cells, fragments, and aggregates after printing, according to an embodiment. [Figure 10C] 10A-10C illustrate characterization of cells, fragments, and aggregates after printing, according to an embodiment. [Figure 10D] 10A-10C illustrate characterization of cells, fragments, and aggregates after printing, according to an embodiment. [Figure 10E] 10A-10C illustrate characterization of cells, fragments, and aggregates after printing, according to an embodiment. [Figure 10F] 10A-10C illustrate characterization of cells, fragments, and aggregates after printing, according to an embodiment. [Figure 10G] 10A-10C illustrate characterization of cells, fragments, and aggregates after printing, according to an embodiment. [Figure 10H] 10A-10C illustrate characterization of cells, fragments, and aggregates after printing, according to an embodiment. [Figure 11] 1A-C show characterization of cells, fragments and aggregates according to one embodiment. [Figure 12A] 10 illustrates characterization of cells, fragments, and aggregates after printing, according to one embodiment. [Figure 12B] 10 illustrates characterization of cells, fragments, and aggregates after printing, according to one embodiment. [Figure 13]A and B show the differentiation of cells, fragments and aggregates according to one embodiment. [Figure 14A] The detection of cells, fragments and aggregates according to one embodiment is described. [Figure 14B] The detection of cells, fragments and aggregates according to one embodiment is described. [Figure 15A] The detection of cells, fragments and aggregates according to one embodiment is described. [Figure 15B] The detection of cells, fragments and aggregates according to one embodiment is described. [Figure 16A] The detection of cells, fragments and aggregates according to one embodiment is described. [Figure 16B] The detection of cells, fragments and aggregates according to one embodiment is described. [Figure 17A] The detection of cells, fragments and aggregates according to one embodiment is described. [Figure 17B] The detection of cells, fragments and aggregates according to one embodiment is described. [Figure 18A] The detection of cells, fragments and aggregates according to one embodiment is described. [Figure 18B] The detection of cells, fragments and aggregates according to one embodiment is described. [Figure 19A] 13 illustrates the identification of cells, fragments, and aggregates after printing, according to one embodiment. [Figure 19B] 13 illustrates the identification of cells, fragments, and aggregates after printing, according to one embodiment. [Figure 20A] 13 illustrates the identification of cells, fragments, and aggregates after printing, according to one embodiment. [Figure 20B] 13 illustrates the identification of cells, fragments, and aggregates after printing, according to one embodiment. [Figure 21A] This is an input image. [Figure 21B] 1 shows detected cells according to one embodiment. [Figure 21C] A bar graph of the distribution of cell characteristics is shown. [Figure 21D] This is an input image. [Figure 21E] 1 shows detected cells according to one embodiment. [Figure 21F] A bar graph of the distribution of cell characteristics is shown. [Figure 21G] This is an input image. [Figure 21H] 1 shows detected cells according to one embodiment. [Figure 21I] A bar graph of the distribution of cell characteristics is shown. [Figure 21J] This is an input image. [Figure 21K] 1 shows detected cells according to one embodiment. [Figure 21L] A bar graph of the distribution of cell characteristics is shown. [Figure 22A] 10 illustrates the identification of cells and aggregates before printing, according to one embodiment. [Figure 22B] 10 illustrates the identification of cells and aggregates before printing, according to one embodiment. [Figure 22C] 10 illustrates the identification of cells and aggregates before printing, according to one embodiment. [Figure 22D] 10 illustrates the identification of cells and aggregates before printing, according to one embodiment. [Figure 22E] 10 illustrates the identification of cells and aggregates before printing, according to one embodiment. [Figure 22F] 10 illustrates the identification of cells and aggregates before printing, according to one embodiment. [Figure 23A] 10 illustrates the identification of cells and aggregates during printing, according to one embodiment. [Figure 23B] 10 illustrates the identification of cells and aggregates during printing, according to one embodiment. [Figure 23C] 10 illustrates the identification of cells and aggregates during printing, according to one embodiment. [Figure 23D] 10 illustrates the identification of cells and aggregates during printing, according to one embodiment. [Figure 23E] 10 illustrates the identification of cells and aggregates during printing, according to one embodiment. [Figure 23F]10 illustrates the identification of cells and aggregates during printing, according to one embodiment. [Figure 24A] 10 illustrates the identification of cells and aggregates after printing, according to one embodiment. [Figure 24B] 10 illustrates the identification of cells and aggregates after printing, according to one embodiment. [Figure 24C] 10 illustrates the identification of cells and aggregates after printing, according to one embodiment. [Figure 24D] 10 illustrates the identification of cells and aggregates after printing, according to one embodiment. [Figure 24E] 10 illustrates the identification of cells and aggregates after printing, according to one embodiment. [Figure 24F] 10 illustrates the identification of cells and aggregates after printing, according to one embodiment. [Figure 24G] 10 illustrates the identification of cells and aggregates after printing, according to one embodiment. [Figure 24H] 10 illustrates the identification of cells and aggregates after printing, according to one embodiment. [Figure 24I] 10 illustrates the identification of cells and aggregates after printing, according to one embodiment. [Figure 24J] 10 illustrates the identification of cells and aggregates after printing, according to one embodiment. [Figure 24K] 10 illustrates the identification of cells and aggregates after printing, according to one embodiment. [Figure 24L] 10 illustrates the identification of cells and aggregates after printing, according to one embodiment. [Figure 24M] 10 illustrates the identification of cells and aggregates after printing, according to one embodiment. [Figure 24N] 10 illustrates the identification of cells and aggregates after printing, according to one embodiment. [Figure 24O] 10 illustrates the identification of cells and aggregates after printing, according to one embodiment. [Figure 24P] 10 illustrates the identification of cells and aggregates after printing, according to one embodiment. [Figure 25A] 13 illustrates the identification of cells and aggregates depicting beads in printed fibers, according to one embodiment. [Figure 25B]13 illustrates the identification of cells and aggregates depicting beads in printed fibers, according to one embodiment. [Figure 25C] 13 illustrates the identification of cells and aggregates depicting beads in printed fibers, according to one embodiment. [Figure 25D] 13 illustrates the identification of cells and aggregates depicting beads in printed fibers, according to one embodiment. [Figure 25E] 13 illustrates the identification of cells and aggregates depicting beads in printed fibers, according to one embodiment. [Figure 25F] 13 illustrates the identification of cells and aggregates depicting beads in printed fibers, according to one embodiment. [Figure 26] 1 shows the bead quantification model integrated into the print analysis software. DETAILED DESCRIPTION OF THE INVENTION

[0022] Certain exemplary aspects of the systems, apparatus, and methods according to this invention are described herein in connection with the following description and the accompanying drawings. These aspects are indicative, however, of but a few of the various ways in which the principles of the invention may be employed, and the present invention is intended to include all such aspects and their equivalents. Other advantages and novel features of the present invention will become apparent from the following detailed description when considered in conjunction with the drawings.

[0023] In the following detailed description, numerous specific details are set forth to provide a thorough understanding of the present invention. In other instances, well-known structures, interfaces, and processes have not been described in detail so as not to unnecessarily obscure the present invention. However, it will be apparent to those skilled in the art that those specific details disclosed herein are not required to practice the present invention, except as described in the claims, and do not represent limitations on the scope of the present invention. It is intended that nothing in this specification be construed as affecting any disclaimer of any portion of the overall scope of the present invention. Although specific embodiments of the present disclosure are described, these embodiments are similarly not intended to limit the overall scope of the present invention.

[0024] 1 shows a high-level diagram of a bioprinting system 100 according to one embodiment. Bioprinting system 100 includes multiple stages, labeled for ease of reference as reservoir 110 (which may be one or more reservoirs), print head 120, and print surface 130 (which may be any surface onto which material from print head 120 may be deposited). In embodiments, the printhead may be an extrusion or co-extrusion printhead, including, but not limited to, multi-material extrusion (e.g., Liu et al. Adv Mater. 2017 Jan;29(3): 10.1002), multi-screw extrusion, or melt extrusion (e.g., Loewner et al., Frontiers in Bioengineering and Biotechnology 2022 DOI 10.3389 / fbioe.2022.896719), inkjet printhead (e.g., Lal Roy, Lab on a Chip 2021 DOI 10.1039 / d1lc00524c), electrowriting printhead (e.g., Kade and Dalton, Advanced Healthcare Materials 2020 DOI 10.1002 / adhm.202001232), or on-chip crosslinking, microfluidic stereolithography (e.g., Miri et al., Adv Mater. 2018 Jul;30(27): e1800242) or microfluidic printing (e.g., Lee et al. Nature Materials 2011 10:877-883). In a preferred embodiment, the bioprinting system uses on-chip crosslinking as disclosed in U.S. Pat. No. 11,046,930, PCT / CA2014 / 050556, PCT / CA2018 / 050315, PCT / CA2018 / 050315, PCT / CA2018 / 050315, and PCT / CA2022 / 051855, the disclosures of which are expressly incorporated herein by reference.

[0025] Each stage has a respective optical system associated with it, which provides image data for the cells and cell aggregates at each stage. For example, reservoir 110 may have optical system 112 associated with it, which may include a microscope, a still camera, and / or a video camera to capture images of the cells in the reservoir at appropriate angles to characterize the cells in the reservoir before printing. The cells may be individual cells, what are called fragments, or cell aggregates. Similarly, print head 120 may have optical system 122 associated with it, which may also include a microscope, a still camera, and / or a video camera to capture images of the cells in the print head, for example, in the nozzle of the print head as described in the above-referenced U.S. provisional and PCT applications, at appropriate angles to characterize the cells during printing. The cells may also be individual cells, what are called fragments, or cell aggregates. Similarly, the printing surface 130 may also have an optical system 132 associated with it, which may also include a microscope, still camera, and / or video camera to capture images of the cells on the printing surface at appropriate angles to characterize the images of the cells in the reservoir after printing. The cells may also be individual cells, what are called fragments, or they may be cell aggregates.

[0026] In embodiments, bioprinting system 100 may be under the control of computing system 150, which may include one or more central processing units (CPUs), volatile memory, non-volatile memory, and non-transitory non-volatile storage. Computing system 150 may connect to one or more of optical systems 112, 122, and 132 directly or through various connections over a local area network (LAN) or via the cloud.

[0027] Computing system 150 may include machine learning system 160. In embodiments, machine learning system 160 may use one or more graphical processing units (GPUs) with multiple cores to facilitate the calculations necessary for the machine learning system to perform rapid computational analysis of data streams or captured images and training of models that provide feedback for controlling material flow within the bioprinting system. In embodiments, the GPUs of machine learning system 160 may be part of computing system 150. In embodiments, machine learning system 160 may use memory and storage included with computing system 150. In other embodiments, machine learning system 160 may have its own volatile memory, non-volatile memory, and / or non-transitory non-volatile storage. In embodiments, model training may include testing and validation to facilitate optimization and inference by the trained model.

[0028] In an embodiment, machine learning system 160 may include a neural network selected from the group consisting of a convolutional neural network (CNN), a region-based CNN (R-CNN), a Mask R-CNN, a you only look once (YOLO)-based model, and a transformer-based instance segmentation model to perform image segmentation on image data received from optical systems 112, 122, and 132. In an embodiment, machine learning system 160 may also interact with computing system 150 to perform several user-interactive and system-interactive functions. For user interaction, computing system 150 may provide appropriate graphic displays and graphical user interfaces for interacting with various other components, such as the optical systems, machine learning system 160, and the remainder of bioprinting system 100.

[0029] In a cloud environment, the machine learning system 160 and its user interface may be deployed as a cloud application, e.g., as a login-accessible website through which users can view and interact with the print video online and view the results of the machine learning analysis. In such an embodiment, the computing system 150 and the machine learning system are hosted in the cloud.

[0030] Depending on the embodiment, computing system 150 may control optical systems 112, 122, and 132 via one or more graphical user interfaces on one or more displays. Depending on the embodiment, optical systems 112, 122, and 132 may have their own controllers so that they may be appropriately positioned relative to reservoir 110, print head 120 and its associated feed and nozzle(s), and print surface 130, respectively, in response to commands from computing system 150.

[0031] Part of the control performed by computing system 150 includes overseeing access to control systems within the bioprinting system to regulate material flow by controlling the pressure of various print head channels (e.g., including microfluidic print head channels) described below or by controlling the displacement of material through the channels. In embodiments, the control may include toggling on or off or otherwise regulating the opening and closing of pneumatic valves on the print head. In embodiments, part of the control performed by computing system 150 includes panoptic segmentation and / or semantic segmentation and / or instance segmentation. In embodiments, panoptic segmentation and / or semantic segmentation and / or instance segmentation enable a visual estimation of the approximate amount and / or distribution of biological material (e.g., multiple cells) within the material flow.

[0032] The computing system 150 may also enable reading of each feed from one or all of the optical systems, enable recording and / or loading of one or both of the feeds, and adjust display parameters such as contrast, color, brightness, and sharpness, among others.

[0033] Depending on the embodiment, machine learning system 160 can characterize particles in bioprinting system 100 in a prescribed manner. Alternatively or additionally, machine learning system 160 can serve to provide focus to images from optical systems 112, 122, and 132 to aid in particle characterization. Alternatively or additionally, machine learning system 160 can provide output to print surface 130 that can be fed back in real time to the 3D bioprinting system to adjust pressure and / or displacement and subsequent flow of material from reservoir 110 to print head 120, enabling consistent production of high-quality tissue and minimizing loss of expensive biomaterial and cellular inputs.

[0034] 2A shows a very high-level diagram of the flow involved in the operation of bioprinting system 100 in combination with computing system 150 and machine learning system 160, according to one embodiment. At 210, quantification and characterization of cells can occur prior to printing, for example, in reservoir 110 of bioprinting system 100. FIG. 5, described below, shows cells in a plate of microwells, each containing an aggregate. The process described below for FIG. 4 may be used to quantify the aggregates in each microwell.

[0035] At 212, cells may optionally be preprinted on the plate, for example, to promote aggregation. Cells may also be quantified and characterized at this stage. Step 212 is not required and may be omitted. At 214, cells are placed in the print head and printed. Again, the cells in the print head may be characterized using the optical system, computing system, and machine learning system of FIG. 1 . In one embodiment, a comparison may be made between the cells in the print head and the cells in the microwells. At 216, cells may be deposited onto a printed device, such as print surface 130 of FIG. 1 . At 218, the device structure is characterized. At 220, cell analysis may be performed on the resulting tissue before implantation. In some embodiments, the same machine learning system 160 of FIG. 1 or different machine learning systems may be applied to each stage 210-220, and results from a given stage may be compared to results from other stages. In one embodiment, results from a given stage may be compared to a reference to enable determining whether and / or how additional training of the machine learning system should be performed.

[0036] 2B is a high-level flowchart for the implementation and use of computer vision and machine learning tools to provide automatic feedback control. At 230, imaging devices in optical systems 112, 122, and 132 are controlled to record video and / or still images of material passing through each stage of bioprinting system 100. These images are acquired through various types of operation of the bioprinting system. To generate a training set of data for the machine learning model, for example, the system can be operated with known materials and parameters to obtain known results. Other data sets may also be generated for use in testing and evaluation of the trained system. In one embodiment, an existing data set may be used for training.

[0037] At 235, the collected data may be separated into the aforementioned training set and test / evaluation set. At 240, the training set is focused and features of interest are labeled to facilitate focusing on them during training. At 245, the appropriately labeled training set is applied to a machine learning model of the type described above to train and evolve the model to generate correct results from the training data. Some machine learning models may use forward propagation of results. Other models may use back propagation.

[0038] At 250, the training results may be analyzed to identify various features of interest, such as the quantity, distribution, concentration, quality, and / or morphology of a plurality of cells passing through a particular stage of the bioprinting system 100. In one embodiment, morphology may include at least one characteristic selected from the group consisting of cell number, area, perimeter, ellipse volume, circularity, smoothness, compactness, circle diameter, mean diameter, minimum diameter, maximum diameter, average intensity, and cell density in fibers. The results may then be classified according to features of interest, level of accuracy, etc. At 255, a test / validation set may be used to evaluate the performance of the trained network. If the model is satisfactory at 260, the full trained model may be deployed at 270. If the model is not satisfactory, additional training samples may be added and / or the model may be otherwise adjusted at 265. Control then returns to 240 for further training. In one embodiment, additions to the training set may be informed by the nature of the results obtained in previous training sets.

[0039] 3A-3F aid in determining a method for characterizing a plurality of cells passing through bioprinting system 100, according to one embodiment. In one embodiment, the morphology of a plurality of cells may be characterized. FIG. 3A shows an aggregate 300 including one or more cells to be characterized. Aggregate 300 may be approximated as a polygon. A polygon has a center of gravity and has an area. FIG. 3B shows a circle 310 having approximately the same area and center of gravity as the polygon. FIG. 3B shows areas 311, 313, 315, and 317 where aggregate 300 is outside the circumference of circle 310. FIG. 3B also shows areas 312, 314, 316, and 318 where circle 310 is outside aggregate 300. A circle has an area A=πr 2 By solving the equation for r, we can determine the diameter of the circle.

[0040] Figure 3C shows the major diameter D L and minor diameter D S The ellipsoid volume 320 is depicted as having the same volume as a sphere.

number

[0041] With the above in mind, a value for the circularity of agglomerate 300 can be obtained. Circularity is the ratio of the area of ​​circle 310 to the area of ​​agglomerate 300, as shown in Figure 3D, which indicates the same areas 311, 313, 315, 317 where agglomerate 300 is outside the circumference of circle 310, and the same areas 312, 314, 316, 318 where circle 310 is outside agglomerate 300. The more circular agglomerate 300 is, the closer the circularity is to 1. The less circular the shape is (e.g., more elliptical), the closer the circularity is to 0.

[0042] FIG. 3E is a diagram useful for characterizing the surface complexity of agglomerate 300. The more convex an agglomerate 300 is, the smoother it is. FIG. 3E shows agglomerate 300 with a convex hull 330 surrounding it. Convex hull 330 is the smallest convex polygon that contains agglomerate 300. The smoothness of agglomerate 300 (or the compactness of the convex hull) is then the ratio of the area of ​​agglomerate 300 to the area of ​​convex hull 330. For a perfectly convex polygon of agglomerate 300, the smoothness is 1. The greater the number of concavities in agglomerate 300 and the larger the concavities, the closer the smoothness approaches 0.

[0043] Figure 3F shows an alternative approach for determining the area and / or diameter of agglomerate 300. In Figure 3F, a circle 350 surrounds agglomerate 300. In one embodiment, circle 350 has the same center of gravity as agglomerate 350. Several diameters 360, 365, and 370 of agglomerate 300 are shown, with diameter 360 being the longest and diameter 365 being the shortest. Diameter 370 represents the average of diameters 360 and 365.

[0044] The following table summarizes the metrics that Figures 3A-3F illustrate. [Table 1]

[0045] FIG. 4 is a flowchart for characterizing a group of cells among a plurality of cells passing through bioprinting system 100. At 410, a determination is made whether the diameter of the group of cells is greater than 40 μm. If so, then at 412, the group of cells is determined to be an aggregate. If not, then at 420, a determination is made whether the diameter of the group of cells is greater than 25 μm but less than 40 μm. If so, then at 422, a determination is made whether the ratio of the major axis of the group of cells to the minor axis of the group of cells is greater than or equal to 1.3. If so, then at 424, the group of cells is determined to be a fragment. If not, then at 430, a determination is made whether the aforementioned ratio of the major axis to the minor axis is less than or equal to 1.3. If so, then at 432, a determination is made whether the diameter of the group of cells is less than 30 μm. If so, then at 434, the group of cells is determined to be a single cell. If not, then at 440, a determination is made whether the diameter of the group of cells is greater than 30 μm but less than 40 μm. If so, then at 442 a determination is made whether the circularity of the group of cells is greater than or equal to 0.70. If so, then at 444 the group of cells is determined to be a fragment. If not, then at 450 a determination is made whether the circularity of the group of cells is greater than 0.70. If so, then at 452 the group of cells is determined to be an aggregate. Finally, at the start of FIG. 4 , if it was determined at 420 that the diameter of the group of cells is less than or equal to 25 μm and less than 40 μm, then at 460 a determination is made whether the diameter of the group of cells is less than 25 μm. If so, then at 462 the group of cells is determined to be a single cell.

[0046] Figure 5 shows multiple microwells, each containing multiple cells therein, with numbers indicating the smoothness and circularity of the cells within each microwell.

[0047] Figure 6A shows a preprint cell. Figure 6B is a slightly enlarged view of a portion of Figure 6C, and Figures 6B and 6C include color coding to characterize the multiple cells shown. Figure 6D is a set of bar graphs corresponding to the color coding of Figures 6C and 6D. The first bar graph shows the area of ​​various aggregates clustered around a relatively low number. The second bar graph, consistent with the first bar graph, shows the radius of various aggregates clustered around a relatively low number. The third and fourth bar graphs show that various aggregates tend to have high circularity and smoothness.

[0048] Figure 7A shows an example of cell detection according to one embodiment. Figure 7B shows examples of both cell and aggregate detection according to another embodiment in the preprint stage. Notably, the comparison shows that the latter embodiment identifies significantly more cells than the former embodiment.

[0049] Figure 8A shows multiple cells color-coded with different colors to represent structures of different sizes, as determined by a machine learning system according to one embodiment. Figure 8B includes a table showing what the different colors mean and classifying the various structures in Figure 8A as cells, fragments, or cell aggregates.

[0050] Figures 9A and 9B show the quantification of cells in the print head during printing. Similar to Figures 6B and 6C, Figure 9B shows the classification of multiple cells within the print head.

[0051] Figure 10A shows cells after printing, and Figure 10B shows quantification of cells according to one embodiment. Similarly, Figure 10C shows cells after printing, and Figure 10D shows quantification of cells after printing.

[0052] Figures 10E-10H show detection of aggregates and cells in post-printed images. According to one embodiment, the cells and aggregates are located in the shell region of the printing device. Figure 10E shows the cells after printing, and Figure 10F shows quantification of the cells according to one embodiment. Similarly, Figure 10G shows the cells after printing, and Figure 10H shows quantification of the cells according to one embodiment.

[0053] Figure 11A shows postprinted cells, and Figure 11B shows quantification of those postprinted cells. Similar to Figure 6D, Figure 11C shows the generally small area and small radius of the sampled cells, as well as their high circularity and smoothness.

[0054] 12A and 12B, along with several subsequent figures, illustrate different tasks that a machine learning system can be trained to perform, according to one embodiment. FIG. 12A illustrates cell detection after printing, providing a boundary around each identified cell. FIG. 12B identifies which cells are in focus and provides a boundary around each of them. From these figures, one skilled in the art will understand that a machine learning system, according to an embodiment, can distinguish between in-focus and out-of-focus images from an optical system and work only with cells that are in focus within an in-focus image.

[0055] Figures 13A and 13B are similar to Figures 12A and 12B in that Figure 13A provides a boundary around the detected cells after printing, and Figure 13B shows a determination as to which of the detected cells are actually in focus. Figure 13B, like Figures 12A and 12B, has fewer boundaries relative to Figure 13A because Figure 13B only places a boundary around the cells that are in focus.

[0056] FIG. 14A shows a photograph of postprinted cells. FIG. 14B shows a more clearly focused postprinted cell. A machine learning system, according to one embodiment, can be trained to identify cells, fragments, and aggregates, and can further identify which of the detected cells, fragments, and aggregates are in focus and highlight them accordingly, in this case, by contrast. Similar to FIGS. 14A and 14B, FIG. 15A shows a photograph of postprinted cells, and FIG. 15B shows a more clearly focused postprinted cell.

[0057] 16A and 16B illustrate yet another task that a machine learning system according to an embodiment can perform. FIG. 16A shows a photograph of multiple cells, fragments, and aggregates within a fiber. The cells, fragments, and aggregates are in varying degrees of focus. FIG. 16B shows a photograph of the same multiple cells, fragments, and aggregates, but with more of them in focus. A machine learning system according to one embodiment can not only identify the cells, fragments, and aggregates, but also sharpen their focus.

[0058] FIG. 17A shows postprint cells before detection by the machine learning system. FIG. 17B shows cells detected both in focus and out of focus. These figures therefore demonstrate that machine learning systems according to embodiments can detect cells and appropriately highlight them, whether in focus or out of focus. This detection facilitates quantification and quality confirmation. FIGS. 18A and 18B, 19A and 19B, and 20A and 20B have similar effects to FIGS. 17A and 17B, respectively. FIGS. 20A and 20B show cells within fibers according to one embodiment.

[0059] Figures 21A, 21D, 21G, and 21J are example input Z-stack images acquired at different times. These images show in-focus cell and aggregate detection in different Z-stack layers of a printed fiber at different times. In one embodiment, the model detects in-focus cells and aggregates. As the user changes the focus of the microscope to image different layers, the in-focus cells and aggregates in different Z-stack layers can be quantified. Figures 21B, 21E, 21H, and 21K show cell detection in the images of Figures 21A, 21D, 21G, and 21J, respectively. Figures 21C, 21F, 21I, and 21L show bar graphs illustrating the distribution of a wider range of properties (area, radius, circularity, and smoothness) than those shown in Figures 6A-6D and 11A-11C. These figures therefore demonstrate that a machine learning system according to one embodiment can be trained to detect and quantify cells, fragments, and aggregates even when they are irregular in shape.

[0060] Material flow: Aspects of the present invention include material flows that can be used to print structures for advantageous use as biomaterials. As used herein, "biomaterial" refers to natural or synthetic substances, with or without living cells, that are useful for constructing or replacing tissue, e.g., human tissue. In the field of bioprinting, the term "biomaterial" is often synonymous with the term "biolink."

[0061] The material flow generally includes at least one crosslinkable material, such as a hydrogel, including but not limited to alginate, chitosan, PEGDA, PEGTA, hyaluronic acid (HA), HAMA, collagen, CollMA, gelatin, gelMA, agarose, gellan, fibrin (fibrinogen), PVA, or any combination thereof, as well as a non-hydrogel, including but not limited to PCL, PLGA, PLA, or any combination thereof. In preferred embodiments, the material flow includes at least one hydrogel. Non-limiting examples of hydrogels include alginate, agarose, collagen, fibrinogen, gelatin, chitosan, hyaluronic acid-based gels, or any combination thereof. Various synthetic hydrogels are known and can be used in embodiments of the systems and methods provided herein. For example, in some embodiments, one or more hydrogels form at least a portion of the structural basis of the printed three-dimensional structure. In some embodiments, the hydrogel has the ability to support the growth and / or proliferation of one or more cell types, which may be dispersed within the hydrogel or added to the hydrogel after it has been printed in a three-dimensional configuration.

[0062] In some embodiments, the hydrogels are crosslinkable by chemical crosslinkers. For example, hydrogels comprising alginate may be crosslinkable in the presence of divalent cations such as calcium chloride (CaCl), hydrogels comprising chitosan may be crosslinked using polyvalent anions such as sodium tripolyphosphate (STP), hydrogels comprising fibrinogen may be crosslinkable in the presence of enzymes such as thrombin, and hydrogels comprising collagen, gelatin, agarose, or chitosan may be crosslinkable in the presence of heat or a basic solution.

[0063] In some embodiments, the hydrogel comprises alginate. Alginate forms a solidified colloidal gel (high-water content gel or hydrogel) when contacted with divalent cations. Any suitable divalent cation can be used to form a solidified hydrogel with an input material containing alginate. In the alginate ion affinity series Cd2+ > Ba2+ > Cu2+ > Ca2+ > Ni2+ > Co2+ > Mn2+, Ca2+ has the optimal characteristics and is most commonly used to form alginate gels (Ouwerx, C. et al., Polymer Gels and Networks, 1998, 6(5):393-408). Studies have shown that calcium alginate gels, the so-called "egg box" model, form through cooperative binding of Ca ions by poly-G blocks on adjacent polymer chains (ISP Alginates, Section 3: Algin - Manufacture and Structure, in Alginates: Products for Scientific Water Control, 2000, International Specialty Products: San Diego, pp. 4-7). G-rich alginates tend to form thermally stable, strong but brittle Ca gels, while M-rich alginates tend to form weaker but more elastic gels with less thermal stability. In some embodiments, the hydrogel comprises depolymerized alginate.

[0064] In some embodiments, hydrogels can be crosslinked using free-radical polymerization reactions, which create covalent bonds between molecules. Free radicals can be generated by exposing a photoinitiator to light (often ultraviolet light) or by exposing hydrogel precursors to a chemical source of free radicals, such as ammonium persulfate (APS) or potassium peroxodisulfate (KPS) in combination with N,N,N,N-tetramethylethylenediamine (TEMED) as the initiator and catalyst, respectively. Non-limiting examples of photocrosslinkable hydrogels include methacrylated hydrogels, such as hyaluronic acid methacryloyl (HAMA), gelatin methacrylate (GEL-MA), or polyethylene (glycol) acrylate-based (PEG-acrylate) hydrogels, which are used in cell biology because they are inert to cells. Polyethylene glycol diacrylate (PEG-DA) is commonly used as a scaffold in tissue engineering because polymerization occurs rapidly at room temperature, requires little energy input, has a high water content, is elastic, and can be customized to contain various biomolecules.

[0065] In embodiments, the material flow includes a non-biodegradable polymer. In examples, the input material may be a synthetic polymer, such as polyvinyl acetate (PVA). In embodiments, the material flow may include hyaluronic acid (HA).

[0066] In embodiments, the material flow includes microparticles, and as used herein, "microparticles" refers to immiscible particles typically ranging from about 0.1 μm to about 100 μm, composed of polymers, metals, or other inorganic materials. They can be symmetrical (e.g., spherical, cubic, etc.), but this is not required. Microparticles with an aspect ratio of 2:1 or greater may be considered microrods or microfibers.

[0067] Additional ingredients: Material flows according to embodiments of the present invention may include any of a wide variety of natural or synthetic polymers that support the viability of living cells, including, for example, alginate, laminin, fibrin, hyaluronic acid, poly(ethylene)glycol-based gels, gelatin, chitosan, agarose, or combinations thereof. In embodiments, the subject compositions are physiologically compatible, i.e., they promote cell growth, cell differentiation, and intercellular communication. In certain embodiments, the input material includes one or more physiological matrix materials, or combinations thereof. "Physiological matrix material" refers to biological materials found in natural mammalian tissue. Non-limiting examples of such physiological matrix materials include fibronectin, thrombospondin, glycosaminoglycans (GAGs) (e.g., hyaluronic acid, chondroitin-6-sulfate, dermatan sulfate, chondroitin-4-sulfate, or keratin sulfate), deoxyribonucleic acid (DNA), adhesive glycoproteins, and collagen (e.g., collagen I, collagen II, collagen III, collagen IV, collagen V, collagen VI, or collagen XVIII).

[0068] Collagen provides tensile strength to most tissues, with multiple collagen fibrils, approximately 100 nm in diameter, joining together to create strong, multi-coil fibers approximately 10 μm in diameter. The biomechanical function of a particular tissue construct is imparted through the alignment of collagen fibrils in an oriented manner. In some embodiments, the input material comprises collagen fibrils. The fibrous structure that forms into a tissue construct can be created using an input material comprising collagen fibrils. By modifying the diameter of the fiber cells, collagen fibrils can be controlled to direct collagen fibril polymerization in a desired manner.

[0069] In embodiments, the plurality of cells comprises or is selected from the group consisting of single cell suspension, cell aggregate, cell spheroid, cell organoid, or a combination thereof.The flow material according to embodiments of the present invention can include any mammalian cell type, including but not limited to stem cells (e.g., embryonic stem cells, adult stem cells, induced pluripotent stem cells), embryonic cells, endodermal cells (e.g., lung cells, liver cells, pancreatic cells, gastrointestinal cells, or urogenital tract cells), mesodermal cells (e.g., kidney cells, bone cells, muscle cells, endothelial cells, or cardiac cells), ectodermal cells (e.g., skin cells, nervous system cells, pituitary cells, or eye cells), stem cell-derived cells, or any combination thereof.

[0070] For example, flow material may be derived from cells of the pancreas (α, β, δ, ε, γ), liver (hepatocytes, Kupfer cells, stellate cells, sinusoidal endothelial cells, cholangiocytes), thyroid gland (follicular cells), pineal gland (pineal cells), pituitary gland (growth hormone cells, lactotrophs, gonadotrophs, corticotrophs, and thyrotrophs), thymus gland (thymocytes, thymic epithelial cells, thymic stromal cells), adrenal gland (cortical cells, chromaffin cells), These include cells from endocrine and exocrine glands, including ovaries (granulosa cells), testes (Leydig cells), gastrointestinal tract (enteroendocrine cells - intestine, stomach, pancreas), fibroblasts, chondrocytes, meniscal fibrochondrocytes, bone marrow stromal (stem) cells, embryonic stem cells, mesenchymal stem cells, induced pluripotent stem cells, differentiated stem cells, tissue-derived cells, smooth muscle cells, skeletal muscle cells, cardiac muscle cells, epithelial cells, endothelial cells, myoblasts, chondroblasts, osteoblasts, osteoclasts, and any combination thereof.

[0071] The cells can be obtained from a donor of the same species as the recipient (allogeneic), from a different species than the recipient (xenogeneic), or from the recipient (autologous). Specifically, in embodiments, the cells can be obtained from a suitable donor, such as a human or animal, or from the subject into which the cells are to be transplanted. Mammalian species include, but are not limited to, humans, monkeys, dogs, cows, horses, pigs, sheep, goats, cats, mice, rabbits, and rats. In one embodiment, the cells are human cells. In other embodiments, the cells can be xenogeneic, derived from an animal such as a dog, cat, horse, monkey, or any other mammal.

[0072] In some embodiments, at least one biological material comprises a plurality of cells that express / secrete one or more endogenous bioactive substance(s), such as insulin, glucagon, ghrelin, pancreatic polypeptide, Factor VII, Factor VIII, Factor IX, alpha-1-antitrypsin, angiogenic factors, growth factors, hormones, antibodies, enzymes, proteins, exosomes, etc. Endogenous bioactive agents as described herein include agents that cells naturally produce in a biological context (e.g., insulin release in response to elevated glucose concentrations). An endogenous bioactive agent can constitute a therapeutic agent in the context of the present disclosure.

[0073] In some embodiments, the flow material may include genetically modified cells that secrete a specific factor. It is within the scope of this disclosure that, in embodiments, the plurality of cells described above may include modified cells (e.g., genetically modified cells) that secrete a specific factor. The cells may also be from an established cell culture line, or may be cells that have been genetically engineered and / or manipulated to obtain a desired genotype or phenotype. In some embodiments, tissue fragments may also be used, which may provide several different cell types within the same structure.

[0074] Genetic modification techniques applicable to the present disclosure include recombinant DNA (rDNA) technology (Stryjewska et al., Pharmacologial Reports. 2013;65:1075), cell modification based on the use of targeted nucleases (e.g., meganucleases, zinc finger nucleases (ZFNs), transcription activator-like effector nucleases (TALENs), CRISPR-associated nuclease Cas9 (CRISPR-Cas9)), and the like (Lim et al., Nature Communications. 2020;11:4043; Stoddard BL, Structure. 2011;19(1):7-15; Gaj et al., Trends Biotechnol. 2013;31(7):397-405; Hsu et al., Cell. 2014;157(6):1262; Miller et al., Nat Biotechnol. 2010;29(2):143-148), cell modification based on the use of site-specific recombination using recombinase systems (e.g., Cre-Lox) (Osborn et al., Mol Ther. 2013;21(6):1151-1159; Hockemeyer et al., Nat Biotechnol. 2009;27(9):851-857; Uhde-Stone et al., RNA. 2014;20(6):948-955; Ho et al., Nucleic Acids Res. 2015;43(3):e17; Sengupta et al., Journal of Biological Engineering. 2017;11(45):1-9), and the like. In some embodiments, a combination of some of the above techniques may be used for cell modification.

[0075] The present disclosure encompasses recombinant cells capable of producing one or more therapeutic agents, including, but not limited to, a protein, a peptide, a nucleic acid (e.g., DNA, RNA, mRNA, siRNA, miRNA, nucleic acid analogs), a peptide nucleic acid, an aptamer, an antibody or fragment or portion thereof, an antigen or epitope, a hormone, a hormone antagonist, a growth factor or recombinant growth factor and fragments and variants thereof, a cytokine, an enzyme, an antibiotic or antimicrobial compound, an anti-inflammatory agent, an antifungal agent, an antiviral agent, a toxin, a prodrug, a small molecule, a drug (e.g., a drug, dye, amino acid, vitamin, antioxidant), or any combination thereof. [Example]

[0076] We collected an accumulated dataset of 660 images, including annotations of cells, aggregates, and beads, collected from the preprint, print, and postprint stages of the bioprinting pipeline. The model architecture chosen was an adapted Torchvision implementation of the Mask R-CNN instance segmentation model with a ResNet-50-FPN backbone pre-trained on the Common Images in Context (COCO) dataset. Images were fed into the model by tiling through a moving window that slid over each image. Various on-the-fly data augmentation methods were utilized to improve the model's generalizability, including random rotation, scale, shift, color jitter (changing brightness, contrast, saturation, and hue), addition of Gaussian noise, random perspective transformation, random CLAHE and gamma transformation, and random image sharpening and blurring. For model optimization, we used stochastic gradient descent with a step-decay policy. The method's hyperparameters were tuned and fine-tuned for each downstream task. Figures 22A-22F, 23A-23F, 24A-24P, 25A-25F, and 26 show the results of sample segmentation and quantification at various printing stages. The following is a description of the results.

[0077] ● Figures 22A to 22F (preprint cells): These figures show examples of printed cell quantification illustrating aggregates, cells, and fragments. Cell quality features are calculated and stored based on annotations. Figures 22B and 22E are bar graphs corresponding to Figures 22A and 22D, respectively, showing the distribution of values ​​for area, diameter, perimeter, circularity, smoothness, and compactness, among other features.

[0078] ● Figures 23A to 23F (printed cells): These figures show an example of cell quantification using a camera trained on the print head nozzle to delineate fibers, aggregates, and cells during printing. Cell quality features are calculated and stored based on the annotations. Figures 23B and 25E show the corresponding bar graphs in Figures 23A and 23D, respectively, showing the distribution of values ​​for area, diameter, perimeter, circularity, smoothness, and compactness, among other properties.

[0079] ● Figures 24A to 24P (postprint cells): Postprint measurements provide the same aggregate and cell detection and morphology as preprint and print analyses. The postprint example also considers linear density measurements, defined as the expected number of aggregates per unit fiber length. To avoid additional complications in areas where fibers overlap or contact the image edge, annotations of "segment" and "midline" are provided, allowing for targeted aggregation density measurements. Figures 24B and 24C, 24F and 24G, 24J and 24K, and 24N and 24O show bar graphs corresponding to Figures 24A, 24D, 24I, and 24M, respectively, showing the distribution of values ​​for area, diameter, perimeter, circularity, smoothness, and compactness, as well as core diameter, among other properties.

[0080] ● Figures 25A to 25F (printing beads): These figures show an example of bead-in-print quantification depicting beads in the printed fiber. Figures 25B and 25E show corresponding bar graphs to Figures 25A and 25D, respectively, showing the distribution of values ​​for area, diameter, perimeter, circularity, smoothness, and compactness, among other properties.

[0081] ● Figure 26 (Software Integration): This figure shows the bead quantification model integrated into the print analysis software of an embodiment. The software depicts the quantification and relative distribution of beads throughout the length of the print as shown on various plots. The fiber diameter throughout the length of the print is also shown. Several video feeds of the printing process from the nozzle and print surface perspectives can be seen, as well as a representation of the toolpath that defined the print pattern.

[0082] All patent and non-patent references cited herein are incorporated by reference in their entirety for all purposes.

[0083] Aspects of the present invention are described in the following clauses. Clause 1. A bioprinting system comprising: Optionally, a bioprinter including one or more reservoirs, a print head, and a print surface onto which said bioprinting is ejected; an optical system that collects imaging data of the plurality of cells in a material flow comprising a crosslinkable material before, during, and / or after the plurality of cells pass through the bioprinting system; a computing system that processes the imaging data to determine the amount, distribution, concentration, and / or morphology of the plurality of cells; Including, the computing system is a machine learning system, comparing the quantity, distribution, concentration, and / or morphology of the plurality of cells before, during, and / or after passage of the plurality of cells through the bioprinting system to user-established cell morphologies that correspond to physical properties of cells and / or cell aggregates within predetermined tolerances; and / or comparing the quantity, distribution, concentration, and / or morphology of the plurality of cells either before, during, and / or after passage of the plurality of cells through the bioprinting system to the quantity, distribution, concentration, and / or morphology of the plurality of cells at another time before, during, and / or after passage of the plurality of cells through the bioprinting system; recording the quantity, distribution, concentration, and / or morphology of the plurality of cells before, during, and / or after passage of the plurality of cells through the bioprinting system, and the results of the comparison; the machine learning system, The bioprinting system.

[0084] Clause 2. The system of clause 1, wherein the cross-linkable material is cross-linked to form printed microcapsules or fibers, preferably the cross-linkable material is cross-linked to form printed fibers.

[0085] Clause 3. The system of clause 1, wherein the machine learning system performs image segmentation on the imaging data to determine the cell morphology.

[0086] Clause 4. The system of clause 3, wherein the machine learning system includes a neural network selected from the group consisting of a convolutional neural network (CNN), a fully convolutional network (FCN), a region-based CNN (R-CNN), a Mask R-CNN, a you only look once (YOLO) based model, a transformer-based instance segmentation model, a flow estimation network selected from the group consisting of a flow estimation network from the FlowNet family, a recursive all-pairs-field transform (RAFT), and a self-taught multi-frame unsupervised RAFT with whole image warping (SMURF) to perform the image segmentation on the imaging data.

[0087] Clause 5. The system of clause 3 or clause 4, wherein the image segmentation comprises panoptic segmentation and / or semantic segmentation and / or instance segmentation.

[0088] Clause 6. The system of any preceding clause, wherein in response to the comparison, the computing system adjusts one or more parameters of the bioprinting system to alter the flow of the plurality of cells through the print head.

[0089] Clause 7. The system of any preceding clause, wherein the image data includes multiple image frames of a portion of the fiber to be printed, and the computing system assembles the multiple image frames to provide an image of the entire length of the fiber to be printed.

[0090] Clause 8. The system of clause 6, wherein the computing system uses the image of the length of the printed fiber to obtain a measurement of the number of cells within the printed fiber.

[0091] Clause 9. The system of Clause 7, wherein the computing system calculates a therapeutic dose using the measurement of the number of cells within the printed fiber and / or a comparison of the quantity, distribution, concentration, and / or morphology of the cells before, during, and / or after passage of the plurality of cells through the bioprinting system.

[0092] Clause 10. A system described in any of the preceding clauses, wherein the print head has one or more transparent channels and the optical system collects imaging data of the plurality of cells passing through at least one of the one or more transparent channels, and preferably the print head includes a transparent nozzle or ejection channel.

[0093] Clause 11. The system of any of the preceding clauses, wherein the optical system includes one or more imaging devices selected from the group consisting of a microscope, a still camera, and a video camera, and the optical system is positioned at different locations within the bioprinting system to collect the imaging data before, during, and / or after the plurality of cells pass through the bioprinting system.

[0094] Clause 12. The system of clause 10, wherein the optical system includes one or more imaging devices disposed on each of the input reservoir, the print head, and the print surface.

[0095] Clause 13. A system according to any preceding clause, wherein the machine learning system distinguishes between in-focus and out-of-focus imaging data, and performs the comparison and recording on the in-focus imaging data.

[0096] Clause 14. A system described in any of the preceding clauses, wherein the machine learning system distinguishes between focused and out-of-focus imaging data, improves focusing on the out-of-focus imaging data to provide further focused imaging data, and performs comparison and recording on both the focused imaging data and the further focused imaging data.

[0097] Clause 15. A system described in any preceding clause, wherein the imaging data includes a series of Z-stack images of the printed fiber, and the machine learning model detects in-focus cells and aggregates in different Z-stack layers of the printed fiber, and wherein changing the focus of the optical system to image the different Z-stack layers enables quantification of the in-focus cells and aggregates in the different Z-stack layers.

[0098] Clause 16. The system of clause 1, wherein the printhead comprises a printhead selected from the group consisting of an extrusion printhead, a co-extrusion printhead, an inkjet printhead, an electrowriting printhead, and a microfluidic printhead.

[0099] Clause 17. The system of any preceding clause, further comprising determining a quality of the plurality of cells in response to determining the amount, distribution, concentration, and / or morphology.

[0100] Clause 18. The system of any preceding clause, wherein the morphology includes at least one characteristic selected from the group consisting of cell number, area, perimeter, ellipse volume, circularity, smoothness, compactness, circle diameter, average diameter, minimum diameter, maximum diameter, average strength, and cell density in the fiber.

[0101] Clause 19. A bioprinting method comprising: collecting imaging data of a plurality of cells within a material flow comprising a crosslinkable material before, during, and / or after the plurality of cells pass through a bioprinting system, the bioprinting system optionally comprising a bioprinter including one or more input reservoirs, a print head, and a print surface onto which the bioprinting material is dispensed; processing the imaging data using a computer to determine the amount, distribution, concentration and / or morphology of the plurality of cells; using a machine learning system to compare the quantity, distribution, concentration, and / or morphology of the plurality of cells before, during, and / or after passage of the plurality of cells through the bioprinting system with user-established cell morphologies that correspond to physical properties of cells and / or cell aggregates within predetermined tolerances; and / or using the machine learning system to compare the quantity, distribution, concentration, and / or morphology of the plurality of cells either before, during, and / or after crosslinking in the bioprinting system with the quantity, distribution, concentration, quality, and / or morphology of the plurality of cells at another time before, during, and / or after passage of the plurality of cells through the bioprinting system; and using the machine learning system to record the quantity, distribution, concentration, and / or morphology of the plurality of cells before, during, and / or after passage of the plurality of cells through the bioprinting system, and the results of the comparison; said processing comprising: The bioprinting method comprising:

[0102] Clause 20. The method of clause 19, wherein the crosslinkable material is crosslinked to form printed microcapsules or fibers, preferably the crosslinkable material is crosslinked to form printed fibers.

[0103] Clause 21. The method of clause 19, further comprising using the machine learning system to perform image segmentation on the imaging data to determine the cell morphology.

[0104] Clause 22. The method of clause 19, wherein the machine learning system includes a neural network selected from the group consisting of a convolutional neural network (CNN), a region-based CNN (R-CNN), a Mask R-CNN, a you only look once (YOLO)-based model, a transformer-based instance segmentation model, a flow estimation network selected from the group consisting of a flow estimation network from the FlowNet family, a recursive all-pairs-field transform (RAFT), and a self-taught multi-frame unsupervised RAFT with whole image warping (SMURF) to perform the image segmentation on the imaging data.

[0105] Clause 23. The method of clause 21 or clause 22, wherein the image segmentation comprises panoptic segmentation and / or semantic segmentation and / or instance segmentation.

[0106] Clause 24. The method of any of clauses 19-23, further comprising using the computer to vary the flow of the crosslinkable material through the print head in response to one or both of the comparing and adjusting one or more parameters of the bioprinting system.

[0107] Clause 25. The method of any of clauses 19-24, wherein the image data includes a plurality of image frames of a portion of the fiber to be printed, the method further comprising using the computer to assembling the plurality of image frames to provide an image of the entire length of the fiber to be printed.

[0108] Clause 26. The method of clause 25, further comprising using the computer to obtain a measurement of the number of cells within the printed fiber using the image of the length of the printed fiber.

[0109] Clause 27. The method of clause 26, further comprising calculating a therapeutic dose in response to said obtaining and / or comparing said quantity, distribution, concentration, quality, and / or morphology of said cells before, during, and / or after crosslinking in said bioprinting system.

[0110] Clause 28. The method of clause 19, wherein the print head has one or more transparent channels, and the method further comprises using an optical system to collect imaging data of the plurality of cells passing through at least one of the one or more transparent channels, and preferably the print head includes a transparent nozzle or ejection channel.

[0111] Clause 29. The method of Clause 19, wherein collecting image data includes providing one or more imaging devices selected from the group consisting of a microscope, a still camera, and a video camera in an optical system, and positioning the optical system at different locations within the bioprinting system to collect the imaging data before, during, and / or after the plurality of cells pass through the bioprinting system.

[0112] Clause 30. The method of clause 29, wherein said disposing includes disposing one or more imaging devices on each of said input reservoir, said print head, and said print surface.

[0113] Clause 31. A method according to any one of clauses 19 to 30, wherein the machine learning system distinguishes between in-focus and out-of-focus imaging data, and the comparison and recording is performed on the in-focus imaging data.

[0114] Clause 32. A method according to any one of clauses 19 to 31, wherein the machine learning system distinguishes between focused and out-of-focus imaging data, improves focusing on the out-of-focus imaging data to provide further focused imaging data, and performs comparison and recording on both the focused imaging data and the further focused imaging data.

[0115] Clause 33. A method according to any of clauses 19 to 32, wherein the imaging data comprises a series of Z-stack images of the printed fiber, the machine learning model detects in-focus cells and aggregates in different Z-stack layers of the printed fiber, and changing the focus of the optical system to image the different Z-stack layers enables quantification of the in-focus cells and aggregates in the different Z-stack layers.

[0116] Clause 34. The method of any of clauses 19-33, wherein the printhead comprises a printhead selected from the group consisting of an extrusion printhead, a co-extrusion printhead, an inkjet printhead, and a microfluidic printhead.

[0117] Clause 35. The method of any of clauses 19 to 34, further comprising determining the quality of said plurality of cells in response to determining the amount, distribution, concentration, and / or morphology.

[0118] Clause 36. The method of any of clauses 19 to 35, wherein the morphology comprises at least one characteristic selected from the group consisting of cell number, area, perimeter, ellipse volume, circularity, smoothness, compactness, circle diameter, mean diameter, minimum diameter, maximum diameter, average strength, and cell density in the fiber.

Claims

1. 1. A bioprinting system, comprising: Optionally, a bioprinter including one or more reservoirs, a print head, and a print surface onto which said bioprinting is ejected; an optical system that collects imaging data of the plurality of cells in a material flow comprising a crosslinkable material before, during, and / or after the plurality of cells pass through the bioprinting system; a computing system that processes the imaging data to determine the amount, distribution, concentration, and / or morphology of the plurality of cells; Including, the computing system is a machine learning system, comparing the quantity, distribution, concentration, and / or morphology of the plurality of cells before, during, and / or after passage of the plurality of cells through the bioprinting system to user-established cell morphologies that correspond to physical properties of cells and / or cell aggregates within predetermined tolerances; and / or comparing the quantity, distribution, concentration, and / or morphology of the plurality of cells either before, during, and / or after passage of the plurality of cells through the bioprinting system to the quantity, distribution, concentration, and / or morphology of the plurality of cells at another time before, during, and / or after passage of the plurality of cells through the bioprinting system; recording the quantity, distribution, concentration, and / or morphology of the plurality of cells before, during, and / or after passage of the plurality of cells through the bioprinting system, and the results of the comparison; the machine learning system, The bioprinting system.

2. The system of claim 1 , wherein the cross-linkable material is cross-linked to form printed microcapsules or fibers, preferably the cross-linkable material is cross-linked to form printed fibers.

3. The system of claim 1 , wherein the machine learning system performs image segmentation on the imaging data to determine the cell morphology.

4. 4. The system of claim 3, wherein the machine learning system includes a neural network selected from the group consisting of a convolutional neural network (CNN), a fully convolutional network (FCN), a region-based CNN (R-CNN), a mask R-CNN, a you only look once (YOLO)-based model, a transformer-based instance segmentation model, a flow estimation network selected from the group consisting of a flow estimation network from the FlowNet family, a recursive all-pair-field transform (RAFT), and a self-taught multi-frame unsupervised RAFT with whole image warping (SMURF) to perform the image segmentation on the imaging data.

5. The system of claim 3 or claim 4, wherein the image segmentation comprises panoptic segmentation and / or semantic segmentation and / or instance segmentation.

6. 6. The system of claim 1, wherein in response to the comparison, the computing system adjusts one or more parameters of the bioprinting system to alter a flow of the plurality of cells through the print head.

7. 7. The system of claim 1, wherein the image data includes multiple image frames of a portion of the fiber to be printed, and the computing system assembles the multiple image frames to provide an image of the entire length of the fiber to be printed.

8. 7. The system of claim 6, wherein the computing system uses the image of the length of the printed fiber to obtain a measurement of the number of cells within the printed fiber.

9. 10. The system of claim 7, wherein the computing system calculates a therapeutic dose using the measurement of the number of cells within the printed fiber and / or a comparison of the quantity, distribution, concentration, and / or morphology of the cells before, during, and / or after the plurality of cells pass through the bioprinting system.

10. 10. The system of claim 1, wherein the print head has one or more transparent channels, and the optical system collects imaging data of the plurality of cells passing through at least one of the one or more transparent channels, and preferably the print head includes a transparent nozzle or ejection channel.

11. 11. The system of any of claims 1-10, wherein the optical system includes one or more imaging devices selected from the group consisting of a microscope, a still camera, and a video camera, and the optical system is positioned at different locations within the bioprinting system to collect the imaging data before, during, and / or after the plurality of cells pass through the bioprinting system.

12. The system of claim 10 , wherein the optical system includes one or more imaging devices disposed on each of the input reservoir, the print head, and the print surface.

13. 13. The system of claim 1, wherein the machine learning system distinguishes between in-focus and out-of-focus imaging data, and wherein the comparing and recording is performed on the in-focus imaging data.

14. 14. The system of claim 1, wherein the machine learning system distinguishes between focused and out-of-focus imaging data, improves focusing on the out-of-focus imaging data to provide further focused imaging data, and performs comparison and recording on both the focused imaging data and the further focused imaging data.

15. 15. The system of any one of claims 1 to 14, wherein the imaging data comprises a series of Z-stack images of the printed fiber, and the machine learning model detects in-focus cells and aggregates in different Z-stack layers of the printed fiber, and wherein changing the focus of the optical system to image the different Z-stack layers enables quantification of the in-focus cells and aggregates in the different Z-stack layers.

16. 10. The system of claim 1, wherein the printhead comprises a printhead selected from the group consisting of an extrusion printhead, a co-extrusion printhead, an inkjet printhead, an electrowriting printhead, and a microfluidic printhead.

17. The system of any preceding claim, further comprising determining a quality of the plurality of cells in response to determining the amount, distribution, concentration, and / or morphology.

18. The system of any one of claims 1 to 17, wherein the morphology includes at least one characteristic selected from the group consisting of cell number, area, perimeter, ellipse volume, circularity, smoothness, compactness, circle diameter, average diameter, minimum diameter, maximum diameter, average strength, and cell density in the fiber.

19. 1. A bioprinting method comprising: collecting imaging data of a plurality of cells within a material flow comprising a crosslinkable material before, during, and / or after the plurality of cells pass through a bioprinting system, the bioprinting system optionally comprising a bioprinter including one or more input reservoirs, a print head, and a print surface onto which the bioprinting material is dispensed; processing the imaging data using a computer to determine the amount, distribution, concentration and / or morphology of the plurality of cells; using a machine learning system to compare the quantity, distribution, concentration, and / or morphology of the plurality of cells before, during, and / or after passage of the plurality of cells through the bioprinting system with user-established cell morphologies that correspond to physical properties of cells and / or cell aggregates within predetermined tolerances; and / or using the machine learning system to compare the quantity, distribution, concentration, and / or morphology of the plurality of cells either before, during, and / or after crosslinking in the bioprinting system with the quantity, distribution, concentration, quality, and / or morphology of the plurality of cells at another time before, during, and / or after passage of the plurality of cells through the bioprinting system; and using the machine learning system to record the quantity, distribution, concentration, and / or morphology of the plurality of cells before, during, and / or after passage of the plurality of cells through the bioprinting system, and the results of the comparison; said processing comprising: The bioprinting method comprising:

20. 20. The method of claim 19, wherein the crosslinkable material is crosslinked to form printed microcapsules or fibers, preferably the crosslinkable material is crosslinked to form printed fibers.

21. 20. The method of claim 19, further comprising performing image segmentation on the imaging data using the machine learning system to determine the cell morphology.

22. 20. The method of claim 19, wherein the machine learning system includes a neural network selected from the group consisting of a convolutional neural network (CNN), a region-based CNN (R-CNN), a mask R-CNN, a you only look once (YOLO)-based model, a transformer-based instance segmentation model, a flow estimation network selected from the group consisting of a flow estimation network from the FlowNet family, a recursive all-pair-field transform (RAFT), and a self-taught multi-frame unsupervised RAFT with whole image warping (SMURF) to perform the image segmentation on the imaging data.

23. The method of claim 21 or claim 22, wherein the image segmentation comprises panoptic segmentation and / or semantic segmentation and / or instance segmentation.

24. 24. The method of any of claims 19-23, further comprising: using the computer to vary the flow of crosslinkable material through the print head in response to the comparing, adjusting, or both, one or more parameters of the bioprinting system.

25. 25. The method of any of claims 19-24, wherein the image data includes a plurality of image frames of a portion of a fiber to be printed, the method further comprising assembling, with the computer, the plurality of image frames to provide an image of the entire length of the fiber to be printed.

26. 26. The method of claim 25, further comprising using the computer to obtain a measurement of the number of cells within the printed fiber using the image of the length of the printed fiber.

27. 27. The method of claim 26, further comprising calculating a therapeutic dose in response to said obtaining and / or comparing the amount, distribution, concentration, quality, and / or morphology of the cells before, during, and / or after crosslinking in the bioprinting system.

28. 20. The method of claim 19, wherein the print head has one or more transparent channels, the method further comprising collecting imaging data of the plurality of cells passing through at least one of the one or more transparent channels with an optical system, preferably wherein the print head includes a transparent nozzle or ejection channel.

29. 20. The method of claim 19, wherein collecting image data comprises providing one or more imaging devices selected from the group consisting of a microscope, a still camera, and a video camera in an optical system, and positioning the optical system at different locations within the bioprinting system to collect the imaging data before, during, and / or after passage of the plurality of cells through the bioprinting system.

30. 30. The method of claim 29, wherein said disposing comprises disposing one or more imaging devices on each of the input reservoir, the print head, and the print surface.

31. The method of any of claims 19 to 30, wherein the machine learning system distinguishes between in-focus and out-of-focus imaging data, and the comparing and recording is performed on the in-focus imaging data.

32. 32. The method of any one of claims 19 to 31, wherein the machine learning system distinguishes between in-focus and out-of-focus imaging data, improves focusing on the out-of-focus imaging data to provide further in-focus imaging data, and performs comparison and recording on both the in-focus imaging data and the further in-focus imaging data.

33. 33. The method of any of claims 19 to 32, wherein the imaging data comprises a series of Z-stack images of the printed fiber, and the machine learning model detects in-focus cells and aggregates in different Z-stack layers of the printed fiber, and wherein changing the focus of the optical system to image the different Z-stack layers enables quantification of the in-focus cells and aggregates in the different Z-stack layers.

34. The method of any of claims 19 to 33, wherein the printhead comprises a printhead selected from the group consisting of an extrusion printhead, a co-extrusion printhead, an inkjet printhead, and a microfluidic printhead.

35. The method of any of claims 19 to 34, further comprising determining a quality of the plurality of cells in response to determining the amount, distribution, concentration, and / or morphology.

36. 36. The method of any one of claims 19 to 35, wherein the morphology comprises at least one characteristic selected from the group consisting of cell number, area, perimeter, ellipse volume, circularity, smoothness, compactness, circle diameter, average diameter, minimum diameter, maximum diameter, average strength, and cell density in the fiber.

Images