Event-triggered growth of organoid structures

Abstract

Various techniques described herein facilitate an automated growth of organoid structures. For this purpose, the organoid structures are monitored in an automated growing device. In the process, one or more monitoring modalities of the automated growing device can be used. In particular, images of the organoid structures could be captured regularly or according to a specified rhythm for example. These images could be evaluated using suitable image evaluation algorithms in order to check whether the organoid structures have intended properties. The growth of the organoid structures can then be controlled on the basis of the monitoring process.

Description

[0001] DESCRIPTION

[0002] Event-triggered growth of organoid structures

[0003] TECHNICAL AREA

[0004] The following describes techniques for growing organoid structures in an automated incubation device. In particular, techniques are described that involve monitoring the growth of organoid structures using one or more monitoring modalities.

[0005] BACKGROUND

[0006] Organoid production involves isolating stem cells from various sources, such as embryonic stem cells, induced pluripotent stem cells, or tumor samples. These cells can then be differentiated under appropriate conditions to develop into different cell types typically found in a particular organ. The differentiated cells are cultured into three-dimensional structures that resemble the cellular organization and function of the organ in question. In organoids, the 3D structure can arise through self-assembly, growth, and development. Differentiation occurs concurrently. Structure and function are interdependent. The organoid cultures are then cultivated and matured under optimal conditions to promote their growth and development. An organoid culture can consist of one or more organoids.

[0007] Spheroids and tumoroids are further types of three-dimensional cell cultures. In spheroid cultures, a mixture of different cell types is brought together, i.e., aggregated, which can then partially (especially in the case of tumors) develop and grow further. Organoid, spheroid, and tumoroid cultures have applications in various fields. They are frequently used for disease modeling to study the origin and progression of diseases such as cancer, genetic disorders, and infectious diseases. Furthermore, they serve as a platform for drug development by enabling testing in a human tissue environment, leading to more accurate predictions of drug efficacy and toxicity. Organoid cultures can also be used therapeutically.Organoid cultures can contribute to personalized medicine by being produced from a specific patient's cells to develop personalized treatment approaches. Finally, they contribute to basic research by serving as model systems for investigating organ development, cell-cell interactions, and the underlying mechanisms of various diseases.

[0008] The production of organoids can require processes with long cultivation times and numerous steps. Typical cultivation times can range from several days to several months. The necessary steps can be defined in a process protocol, which includes, for example, information on the timing of media changes, media compositions, media temperatures, and environmental conditions such as light and atmospheric conditions. Further possible process steps include, for example, the addition of extracellular matrix, determining cell or...Object density, setting cultivation volumes, adapting / selecting a type of culture vessel, and mechanical steps such as cutting or otherwise dividing or sorting the organoids, separating objects from each other and / or from the bottom of the culture vessel (biochemically or mechanically), and mechanical or biochemical singulation into cell clumps or individual cells.

[0009] However, it has been observed that the results of organoid cultivation can vary from organoid to organoid when such strictly defined process protocols are used. This means that the cultivation results can vary considerably. This limits the scalability of organoid cultivation. For example, EP 4 361 944 A1 discloses a technique for producing a multicellular structure. This involves providing a three-dimensional cell culture comprising a cell population in or on a three-dimensional scaffold; cultivating the three-dimensional cell culture under conditions suitable for subjecting the cell population to a maturation process; and monitoring the cell population during the maturation process.The procedure further includes the execution of one or more control actions based on the predicted metrics indicating the progress or outcome of the maturation process, the control action being optionally selected from: adding a compound or composition to the cell culture, changing the liquid medium in the cell culture, modifying a planned time and / or concentration of the addition of a compound or composition to the cell culture.

[0010] While such techniques allow for some improvement in the yield of organoid manufacturing processes, it has been found that the yield still remains at a relatively limited level. Furthermore, there is a wide variety of organoid types, and the techniques disclosed in the aforementioned publication cannot simply be applied to processes for manufacturing different organoid types.

[0011] SUMMARY

[0012] There is a need for improved techniques for the automated or at least semi-automated cultivation of organoid structures. In particular, there is a need for techniques that can reliably and reproducibly produce organoid structures of various types, even on a larger scale.

[0013] This task is solved by the features of the independent patent claims. The features of the dependent patent claims define embodiments.

[0014] The following describes techniques that enable the semi-automated or fully automated growth of finished organoids or precursor structures in an automated growth device. Precursor structures are, for example, three-dimensional (3D) cell structures suitable for further processing to form a finished organoid with functional differentiation. Organoids and their precursor structures are referred to here as organoid structures.

[0015] The growth of organoid structures in the automated growth device can be monitored using the techniques described herein. One or more monitoring modalities of the automated growth device can be used for this purpose. In particular, for example, images of the organoid structures could be captured regularly or according to a predetermined schedule. These images could then be analyzed using suitable image processing algorithms to verify whether the organoid structures exhibit the intended properties.

[0016] The cultivation of the organoid structures can then be controlled based on monitoring. This means that the automated cultivation device can be controlled based on the monitoring of the organoid structure cultivation. For this purpose, the automated cultivation device is controlled based on both the process protocol and the monitoring data. This allows for improved cultivation results, particularly higher yields and lower variability in the process results.

[0017] For example, a control loop can be implemented which adjusts the rearing process depending on information about the organoid structures obtained through the monitoring modalities.

[0018] The rearing device can be controlled by means of a process protocol that specifies the process. At an abstract level, the process protocol can contain corresponding control instructions for the various modules, which can then be translated by driver software into concrete control signals for the different modules. A process protocol can therefore include control instructions for one or more system components of an automated rearing device. Thus, certain process protocols can be executed fully automatically or semi-automatically.

[0019] To achieve the best possible yield when cultivating different organoid types, the process protocol can be structured in an event-triggered manner. This means that certain process steps included in a process protocol are terminated and / or triggered when, based on monitoring of the cultivation process using one or more monitoring modalities, the occurrence of a specific event related to the organoid structures is detected. When the organoid structures meet certain criteria, an event occurs, which then terminates a process step and / or triggers the subsequent process step.

[0020] A process step can be a single action, such as adding a reagent, or a complex process, such as maintaining organoid structures in free-floating culture and performing a 50% media change every two days until the organoids meet a specific requirement. Such complex processes can also be referred to as process phases.

[0021] Such an event-triggered process log must be distinguished from a process log that merely contains timestamps indicating the end of a specific process step or the start of the next. By triggering the end or start of a process step through the occurrence of an event, it can be ensured that the organoid structure affected by the end or start of the process step has reached the required level of maturity.

[0022] In particular, the properties of the organoid structures can be monitored and evaluated within the context of an event. This means that, for example, values ​​for one or more properties of an organoid structure can be determined, and then it can be checked whether these values ​​meet certain specifications defined in the process protocol (e.g., thresholds or ranges). If this is the case, the corresponding process step can be terminated and / or a corresponding process step can be started.

[0023] Based on the event-triggered structured process protocol, the automated rearing device can also be controlled in an event-triggered manner.

[0024] In various examples, the (i) event-triggered process log is supplemented by (ii) instance-specific monitoring and, at least where possible, (iii) instance-specific control of the automated rearing device.

[0025] In the variants described herein, individual organoid structures can be monitored independently. This means that monitoring can be performed on an instance-specific basis. In particular, it is possible to determine values ​​for one or more properties of individual organoid structures for each of several organoid structures being raised in parallel in the automated rearing device. It is then also possible to monitor the progress of the process for each organoid structure being raised within an automated rearing device individually and to implement the control of the rearing device taking into account the information for individual organoid structures.

[0026] It would then be possible to implement the control of the propagation device on an instance-specific basis for each individual organoid structure; that is, the propagation process could be individually adapted for each of several organoid structures. A single process protocol could therefore be executed individually for each of several organoid structures.

[0027] Using such techniques, it is possible to ensure that individual organoid structures exhibit properties at the end of the growth process that closely match desired, nominal characteristics. The variation in property values ​​between instances during production is reduced. The growth process can be scalable, allowing for the cultivation of a larger number of organoid structures.

[0028] BRIEF DESCRIPTION OF THE FIGURES

[0029] FIG. 1 is a flowchart of an exemplary procedure for growing organoid structures.

[0030] FIG. 2 is a flowchart of an exemplary procedure for growing organoid structures.

[0031] FIG. 3 schematically illustrates an automated growth device according to various examples. FIG. 4 schematically illustrates a process for growing organoid structures according to various examples.

[0032] DETAILED DESCRIPTION

[0033] Some examples in this disclosure generally provide for a variety of circuits or other electrical devices. All references to the circuits and other electrical devices and the functionality they provide are not intended to be limited to only what is shown and described herein. Even if certain designations may be assigned to the various circuits or other electrical devices, these are not intended to limit the scope of function of the circuits and other electrical devices.

[0034] It is understood that the following description of embodiments is not to be understood in a limiting sense. The scope of the invention is not to be limited by the embodiments described below or by the drawings, which serve only for illustration.

[0035] The drawings are to be regarded as schematic representations, and the elements depicted in the drawings are not necessarily shown to scale. Rather, the various elements are represented in such a way that their function and general purpose are recognizable to a person skilled in the art. Any connection or coupling between functional blocks, devices, components, or other physical or functional units shown in or described herein may also be realized by an indirect connection or coupling. Coupling between components may also be established via a wireless connection. Functional blocks may be implemented in hardware, firmware, software, or a combination thereof.

[0036] While many cell cultures, especially two-dimensional cell cultures, which are grown adherently in plates with nutrient solution, for example, show essentially homogeneous growth under the same growth conditions, it has been found that the growth of organoids or their precursors is less deterministic and variations can occur even within strict protocol guidelines.

[0037] Organoids are three-dimensional cell structures grown from stem cells that can mimic specific tissues or organ parts, such as embryonic stem cells, induced pluripotent stem cells, adult stem cells, or cells from tumor tissue. These cells can be of human or animal origin, for example. They are characterized by a complex, self-organized architecture that reflects important functional and structural properties of the target tissue. Unlike traditional cell cultures, which are often set up as two-dimensional monolayers of cells (also called cell lawns) and exhibit limited cell differentiation and interaction, organoids offer an improved model for studying organ development, disease mechanisms, and drug testing because they more closely mimic the microenvironment and cellular interactions of a real organ.

[0038] The following describes techniques related to the generation and production of organoids. The cultivation of an organoid generally involves several growth phases. The division into different phases is not strictly defined, and individual phases (regardless of the specific definition used) can overlap or be difficult to distinguish relative to one another. This is also due to the partially instance-specific variation in the growth of organoid structures. Nevertheless, a possible sequence of phases in organoid production is described below for better illustration. However, this sequence is not limiting for the reasons mentioned above. First, the stem cells are isolated and cultured: The production of an organoid begins with the isolation of pluripotent or tissue-specific stem cells or cells from tumor tissue.Cryopreserved stem cells can also be used. The following explains a possible sequence of phases using the example of organoids based on pluripotent stem cells.

[0039] Pluripotent stem cells are cultivated under controlled conditions to ensure their proliferation and survival. These stem cells are not yet functionally differentiated. Therefore, depending on the specific process employed, a single stem cell can give rise to various types of organoids that mimic different organs. The 3D structure of these organoids can emerge through self-assembly, growth, and development. Differentiation can occur concurrently. During this process, the stem cells are exposed to specific growth factors and signaling molecules that direct differentiation into the desired cell types. These biochemical signals mimic those that occur in the cells' natural environment during organ development.Three-dimensional aggregation and organization then occur: The differentiated cells are induced to arrange themselves (self-organize) within a three-dimensional matrix, often using hydrogels as a scaffold. This matrix supports the spatial arrangement of the cells and promotes the cellular interactions typical of the organ structure. Finally, the organoid matures and becomes functionalized: In this phase, the cells within the aggregates develop complex organoid structures that mimic specific functions of the target organ. Maturation can take several weeks and requires a continuous supply of nutrients and growth factors, which may be tailored to the specific growth stage. During this time, specialized microenvironments also form, which are crucial for the organoid's final function. Additional steps are possible.For example, organoids can be sorted or cut manually or automatically. Specific organoid types may require additional stimuli; for instance, respiratory or lung organoids may require stimuli via a liquid-air interface, or muscle organoids may require mechanical stimuli.

[0040] Using the techniques described herein, process protocols can be implemented that enable the cultivation of an organoid in individual growth phases or from the first phase to the last. All such processes ultimately serve the purpose of producing finished organoids, even if a process only covers one, for example, an early growth phase of the organoids. Organoid structures, in this context, refer not only to finished organoids (i.e., with developed functionality that mimics an organ) but also to 3D cell structures in an earlier growth phase, i.e., precursors of the finished organoids.

[0041] The following describes techniques that enable the efficient and reliable cultivation of organoid structures. In particular, techniques are described that allow for the scalability of organoid structure cultivation to many instances (parallelization). Using these techniques, organoid structures with reproducible properties can be cultivated and produced. Specifically, these techniques may result in minimal variation in the properties of different organoid structures produced using these methods.

[0042] In the variants described herein, organoid structures are cultivated in an automated cultivation device. This automated cultivation device is typically configured to enable the simultaneous cultivation of a large number of organoid structures, either fully or semi-automatically. The automated cultivation device also provides one or more monitoring modalities. These modalities allow for the acquisition of information on the progress of the organoid structure cultivation. The corresponding monitoring modules can incorporate various sensors or data acquisition devices.

[0043] Various monitoring methods are conceivable. These methods can include, for example, a camera, a temperature sensor, a pressure sensor, a pH sensor, and the like. Imaging studies can be performed. Examples include light microscopy (e.g., using optical or digital phase contrast) as well as interferometric imaging. These monitoring methods can capture properties of the organoid structures, their population, and / or properties of the culture medium in a well (hereinafter also referred to as a vat) of the growth device. Examples include the temperature and pH of the culture medium, the ambient pressure in the vat area, and images of organoid structures within the vat.

[0044] The cultivation of organoid structures is based on a process protocol. This protocol describes the details of the organoid structure cultivation process. A process protocol can include control instructions for one or more system components of an automated cultivation device. For example, an automated cultivation device might have one or more so-called multiwell plates or culture plates, each with multiple cavities, also referred to as (process) wells or wells, in which organoid structures can be cultivated under specific, adjustable process conditions (temperature, humidity, aeration, etc.). An automated cultivation device might, for instance, include a dispenser module that automatically fills a well with liquids.The trays could have a controllable drain, allowing for the automated removal of any liquid contained within. The automated growing device could, for example, include a pipetting module (such as a pipetting robot) to aspirate a medium (e.g., a liquid) and / or transport objects, such as liquids, cells, cell clumps, cysts, organoid structures, etc.It would be conceivable for the automated growing device to include a robotic mechanism for moving individual plates with one or more trays from a tray or plate hotel (where the trays can be stored under conditions suitable for growth and development for the duration required to cultivate the organoid structures) to individual modules, such as the dispenser module or a drain module where the liquid in a tray can be drained by opening the tray's drain. An automated growing device could also include a manipulator. Using such a manipulator, individual organoid structures from a group—for example, an aggregate or simply a common quantity in a specific tray—can be grasped, manipulated, and separated, for example, by physical grasping, suction, adhesion, or cutting.For example, organoid structures can be transferred between different trays or plates. The automated growth device can also provide one or more modules for process monitoring. For example, such modules could include various sensors or data acquisition devices. For example, temperature or humidity sensors could be provided at each location of a plate hotel. For example, a module could be included that enables optical, particularly microscopic, imaging of a tray or part of a tray. For example, the imaging modality could also be varied in such a microscopy module. For example, phase contrast could be switched on or off. For example, fluorescence imaging could be switched on or off. Light sheet imaging could be selectively activated.Coherence tomography imaging can be used. A non-magnifying optical image can be used, for example, for sufficiently large organoid structures. Typically, this may require the relevant plate to be retrieved from a plate hotel by a robotic module and placed on a suitable stage for image acquisition. After image acquisition, the robotic module can then retrieve the plate and return it to its correct position in the plate hotel. In other examples, imaging modalities such as wide-field images or optical contrast images (e.g.,Digital phase contrast (DPC), quantum phase contrast (QPC), digital imaging contrast (DIC), transport of intensity (TIE), phase contrast, fluorescence imaging, confocal imaging (lightsheet microscopy, lattice lightsheet microscopy), optical coherence tomography (OCT), or dynamic OCT imaging are available. The acquired images can also be processed and enhanced using image processing techniques or machine-learned algorithms, for example, regarding noise level, contrast level, defocus, etc.

[0045] The cultivation of organoids requires clean, ideally sterile, conditions. Individual elements and / or the entire cultivation apparatus can be equipped accordingly, e.g., with suitable enclosures, filtration of the ambient air, and / or the technical capability to sterilize surfaces or individual units, e.g., by UV, steam, or manual cleaning.

[0046] Using such an automated cultivation device, certain process protocols can be carried out fully or semi-automatically. In semi-automated execution of process protocols, individual process steps may be performed by an operator. For example, a robot module can remove a specific plate from the plate storage area and place it in a sample transfer chamber, allowing the operator to manipulate the corresponding organoid structures in the plate or tray. Such manually performed process steps can be displayed in the process protocol. For instance, the process protocol could include control instructions for a human-machine interface.It would then be conceivable that, before or upon reaching the relevant section in the process, control instructions could be issued to the human-machine interface, prompting the user to perform the corresponding manipulation step. For example, a message could be displayed via a graphical user interface or sent to the user's portable device. This would enable continuous human-machine interaction within the process protocol.

[0047] The techniques described herein can, in particular, employ a process protocol that specifies, for at least one process step, a set of values ​​for one or more properties of one or more organoid structures, triggering or ending the respective process step. In other words, the respective process step begins or ends when the one or more values ​​of the one or more properties of the organoid structures meet the set of values. For example, the appearance, size, shape, etc., of the organoid structures could be considered properties. The additional or new development of a specific property, such as a new cell type, lumen, or other specific structure, can also be considered a property.

[0048] It is also possible to consider objects other than the organoids in the well. For example, the quantity or position of matrigel or other extracellular matrices can be taken into account, such as how they relate to the organoids.

[0049] For example, information on the distribution of Matrigel and organoids can be analyzed in real time or at short intervals, e.g., every 15 seconds or a few minutes, to initiate the termination or commencement of a process step. For instance, in a process step involving the addition of the enzyme Dispase, the temporal evolution of the distance between individual organoid cysts and bulging Matrigel can be determined. This can be achieved, for example, by first locating these objects, e.g., segmenting them, and then determining the shortest distances in pairs. Based on these distances, for example, a statistic of all distances, e.g.,...If a given threshold value is undershot by the 20th percentile value of the intervals, the phase can then be terminated – for example, if this value is undershot in three consecutive measurements. The distribution information can be determined in all wells, or in only some / individual wells and then extrapolated to the entire culture plates, e.g., for three consecutively measured wells.

[0050] The statistical distribution of objects within the wells can also be used as a decision criterion. For example, it is possible to classify organoids into different categories (e.g., using artificial intelligence (AI) analysis or instance segmentation) and to analyze the number or ratio of organoids within the categories.

[0051] The number or density of objects of the same category in a well can also be a criterion. If there are too few or too many objects in a well, manipulation of the same wells can be triggered to achieve an optimal object density.

[0052] The density of cell material in Matrigel or other extracellular matrices can also be a criterion for initiating a process step.

[0053] This corresponds to a trigger event that starts or ends the respective process step. Such an event-triggered process protocol can lead to improved cultivation results. For example, the process protocol can define specific process landmarks that describe the occurrence of a particular event in the cultivation of organoid structures. The precise time of occurrence of this event is not specified in the process protocol but can be identified through process monitoring, for example, using an image-based monitoring modality. Optionally, the protocol can include time limits for a trigger event, such as an acceptable timeframe within which the event should occur. For example, a process step is started or finished when a requirement is met, but no sooner than X days or hours later.Individual process steps can be completed within 10 to 15 minutes and therefore require high temporal resolution, such as one measurement point per minute. Other process segments can last 7-10 days, where one analysis point per day is sufficient.

[0054] This technique, in contrast to time-based or point-in-time-based process protocols, offers the advantage that the growing conditions can be adjusted with particular flexibility and tailored to the specific process. In particular, it can ensure that the organoid structures exhibit specific properties in one or more growth phases, resulting in a particularly good growth outcome at the end of the process.

[0055] In particular, an event-triggered process log can be combined with instance-specific monitoring of the organoid structures.

[0056] In the various examples disclosed herein, the automated growing device can, in particular, include means for instance-specific monitoring of the growth of organoid structures. This means, for example, that specific values ​​for one or more properties of individual organoid structures can be determined. This can enable monitoring of the development of the organoid structures in the various growth phases. It can be verified whether the actual state of an organoid structure corresponds to the target state.

[0057] A specific monitoring modality, suitable for instance-specific monitoring, captures images (e.g., microscope images) of wells in the automated growth system. One or more values ​​for properties of individual organoid structures or their populations can then be determined from these images. These properties include, in particular, geometric parameters of the organoid structures, such as the appearance of the organoid structures or parts thereof in an image; morphology; size; shape; color; the number of organoid structures in a well (total or relative, with a specific appearance); the distance to neighboring organoid structures, i.e., the inter-organoid distance, etc. Properties can also relate to substructures of the organoid structure, such as cell-free lumens, specific layering of cell strata, or the appearance of organ-specific cell types.Other objects (non-organoids) can also be relevant, such as dead cell material or extracellular matrices like Matrigel. Their properties can be captured from images, for example, from various microscopic modalities, such as fluorescence microscopy. If a microscopic image shows a multitude of organoid structures, individual organoid structures (e.g., within an aggregate) can first be localized. In such a case, individual instances of the organoid structures can be located, and then each instance can be evaluated accordingly. Determining instance-specific values ​​for one or more properties of organoid structures is particularly possible when several organoid structures are located in a common container, either loosely or aggregated together.In such a scenario, it would be conceivable to locate individual organoid structures in a corresponding image (e.g., a microscope image) and then determine corresponding values ​​for each organoid structure. For example, a list of values ​​could be created, with a corresponding entry for each organoid structure.

[0058] Therefore, if instance-specific values ​​for one or more properties of organoid structures within a composite are determined, it would be possible to further determine the distribution of such values. Based on such a distribution, the progress of the growth process for different organoid structures within the composite can be compared, and outlier organoid structures can be identified. It can also be verified whether a certain number of organoid structures in the composite exhibit specific property values.

[0059] Techniques for how to perform instance-specific monitoring of organoid structures have been described above.

[0060] Based on such instance-specific monitoring, it can then be verified whether the one or more values ​​of the one or more properties of organoid structures correspond to a specification according to the process protocol, so that a corresponding process step of the process protocol is terminated or triggered. For example, it would be conceivable that the process protocol contains a sequence of process steps. Each process step can describe a specific process setting, such as whether a particular nutrient solution should be added, whether the pH value should be changed, etc. It is possible that the process protocol specifies the sequence of such process steps, but not the concrete timing. Rather, it would be conceivable that individual process steps defined in the process protocol are triggered by events.In other words, controlling the automated cultivation device can involve selectively triggering and / or terminating one or more process steps if monitoring the organoid cultivation reveals that the values ​​of one or more properties of individual organoids meet certain specifications. For example, the values ​​of specific properties of individual organoids could be compared to predefined thresholds. Depending on the result of such a threshold comparison, a specific process step could then be terminated, or it would also be conceivable that a different process step could be initiated.

[0061] By combining instance-specific monitoring of the organoid structure growth with an event-triggered process protocol, individualized growth can be enabled for the various organoid structures grown in parallel within the automated growth device. The execution of the process protocol can be adapted for each organoid structure according to its individual requirements. This allows for a particularly high yield of organoid structure growth and excellent reproducibility of the growth results. The variation in the property values ​​of the generated organoid structures is relatively low due to the instance-specific monitoring and process adaptation.

[0062] Such instance-specific process adaptation is particularly feasible when multiple organoid structures are grown in separate environments within the automated grower. For example, it is conceivable that several organoid structures are grown in different wells. In this case, it would be possible, for instance, to pipette the appropriate nutrient solution individually for each organoid structure in the different wells. Process adaptation is also possible at the level of smaller groups of organoid structures. Each well, for example, can represent a subpopulation that can be optimized independently. Depending on the point in the protocol, these subpopulations can contain, for example, 5-50 individual objects.

[0063] However, there are technical and / or biological limitations to the possibility of instance-specific process adjustments. For example, multiple wells may be arranged in a multi-well plate (MWP). This MWP is then located in a specific space within a plate hotel, and the temperature or other environmental conditions can only be adjusted collectively for the different wells of the MWP, or even collectively for all the different MWPs in the plate hotel. Nevertheless, in such a scenario, where different organoid structures are located in different wells of an MWP, it is possible, for example, to modify the nutrient solution separately for the different organoid structures in the different wells of the MWP. It is also possible that at a given point during the growth process, more than one organoid structure may be present in a single well.It is therefore conceivable that a group of organoid structures could be exposed to the same growth conditions in a shared container. The nutrient solution could then not be individually manipulated for different organoid structures within the group. Such organoid structures could, for example, be loosely arranged within the same container. This means that different organoid structures could move independently within the liquid in the container. However, in certain processes, organoid structures might form a common aggregate, meaning they are connected to each other and cannot move independently within the container. In both cases (loose organoid structures in the shared container and organoid aggregates), the corresponding organoid structures are thus grown together, at least temporarily.

[0064] When adapting the process to individual organoid structures on a case-by-case basis, factors that necessitate at least partially coupled growth (such as a shared tank in a microwavable space, multiple organoid structures in an aggregate, etc.) can be taken into account. For example, it would be conceivable to determine a distribution of the values ​​of one or more properties for the organoid structures within a group. For instance, a distribution of the size of the organoid structures could be determined. Based on this distribution, a subset of all organoid structures included in the distribution could then be determined. For example, this subset could include all organoid structures whose size falls within the half-width of the distribution.Alternatively, only those organoid structures that are not located in the "heavy tails" of the distribution, i.e., at the extremes of the distribution, are considered. Then, the decision of whether to trigger or terminate a process step can be made in isolation based on the values ​​within the subset. For example, the size values ​​of those organoid structures in the subset could be compared to a target value. In other words, the distribution of values ​​can be analyzed, and it can be determined that, for example, for more than 50% of the organoid structures in the group, the values ​​of one or more properties correspond to the respective specification for terminating or starting a process step. It could also be determined whether certain "outlier" organoid structures are present, i.e., those exhibiting abnormal development. "Outlier" organoid structures can be identified, for example, using anomaly detection methods.Outlier organoid structures can be identified using a Gaussian curve, for example, if a property value lies outside a 2-sigma or even a 3-sigma range around the mean of a normal distribution, where the mean and sigma are calculated as the standard deviation based on the property values. Such outlier organoid structures could then be ignored when checking whether an event has occurred.

[0065] The preceding section described how, in the case of organoid structures present in a cluster, suitable control of the automated cultivation device can be achieved based on instance-specific monitoring of the organoid structure growth, taking into account the corresponding distribution of values ​​determined in this way for one or more properties. This control optimizes the overall process result in terms of yield. In such a case, instance-specific control of the automated cultivation device is not possible due to the existing cluster of organoid structures; nevertheless, the cultivation result can still be optimized overall by considering the distribution.

[0066] In another scenario, which will be described in detail below, the group can be broken up, meaning singulation can be carried out. This then enables instance-specific control of the automated rearing device in one or more subsequent process steps.

[0067] For example, it would be possible to separate the organoid structures at a specific step in the growth process. This means separating organoid structures from a group and transferring them to individual trays, or even transferring them from one MWP (Mechanical Water Processing Unit) to different MWPs. From this point onward, it is then possible to adjust the process conditions for each individual organoid structure. A suitable manipulator can be used for this separation.

[0068] However, the goal doesn't necessarily have to be isolation. For example, sorting can also be useful to keep organoids of the same category in the same well. For instance, all dying organoids can be removed and / or all healthy ones transferred to a new well. In other examples, organoids that need to undergo a specific processing step can be sorted into a separate well so they can then be processed, e.g., dissection (removing defective parts manually or automatically, e.g., by laser dissection).

[0069] In particular, in the various examples described herein, it is also conceivable that such a process step, which includes the isolation of organoid structures, is carried out in an event-triggered manner when one or more values ​​of one or more properties of the organoid structures correspond to a certain specification.

[0070] For such a separation event or other events, properties of the respective group of organoid structures (and not just the individual organoid structures) can also be considered. This is explained below. For example, with tightly connected organoid structures, such as those found in an aggregate, instance-specific monitoring—that is, instance-specific determination of values—can be performed not only for the organoid structures in the corresponding aggregate, but also—alternatively or additionally—for the aggregate itself. For example, the size of the aggregate could be determined. The number of organoid structures contained in the aggregate could be determined. It would also be conceivable to determine the shape of the aggregate.

[0071] Organoids within a group for which an evaluation and decision are to be made do not need to be physically connected. An organoid group can also include, for example, all organoids within a single well or an entire culture plate.

[0072] In the case of a loosely connected group of organoid structures, the degree of coverage of the organoid structures in a corresponding tub could, for example, be determined. It can therefore be checked whether there is still enough space for the organoid structures in a talking tub. If this is not the case, separation can be triggered.

[0073] In summary, instance-specific monitoring of the breeding process can refer to organoid structures and / or to a group (e.g., an aggregate) of organoid structures. Therefore, values ​​of properties can be determined at the organoid structure level and / or values ​​of properties can be determined at the group level.

[0074] By means of such instance-specific monitoring of the organoid structure growth, the execution of the process protocol can also be instance-specific, taking this monitoring into account. In other words, instance-specific control of the automated growth device for individual organoid structures is possible.

[0075] Figure 1 is a flowchart of an exemplary process. The process shown in Figure 1 is used for the cultivation of organoid structures. Organoid structures are grown in an automated cultivation device through one or more growth phases. For example, several organoid structures can be grown simultaneously. The process can be at least partially computer-implemented. A control device of an automated cultivation device can implement at least parts of the process. For this purpose, a processor can load and execute program code from memory.

[0076] Box 3005 provides a process protocol. This protocol defines a process with several sequential process steps. The process is used for growing organoid structures in the automated growth device. The process covers one or more growth phases of an organoid.

[0077] For example, the process log can be loaded from a database. The process log can be specified by a user. The process log can be loaded from memory.

[0078] According to various examples, the process protocol is defined as event-triggered. This means that for at least one process step, the process protocol specifies a requirement for one or more respective values ​​for one or more properties of organoid structures. If the one or more values ​​meet the requirement, the respective process step is either started or terminated.

[0079] The process protocol is then carried out using the automated rearing device. For this purpose, the automated rearing device is appropriately controlled.

[0080] In Box 3010, the process for growing the organoid structures is monitored using one or more monitoring modalities of the automated growing device. Monitoring results are obtained in Box 3010.

[0081] Based on these monitoring results, the growing device can then be controlled accordingly (Box 3015). This means that the automated growing device is controlled to implement the process based on monitoring the growth of the organoid structures. Box 3015 can therefore send control instructions to one or more modules of the automated growing device. For example, process steps can be triggered or terminated. Process parameter values ​​can also be set.

[0082] Box 3015 can issue one or more control instructions to one or more modules of the automated growing device. For example, a liquid dispenser could be controlled to fill a tray with a specific nutrient solution. A pipetting robot could be controlled to perform a liquid change in a tray. For example, a robotic arm could be controlled to move a plate from one position in a plate hotel to another. One or more growing parameters can be set. For example, the temperature or pH value could be set for individual trays or plates. Optionally, a stirring mechanism could be activated for a specific tray, or, for example, mechanical or optical stimulation of the organoid structure could be initiated.

[0083] In principle, the propagation device in Box 3015 can be controlled instance-specifically, meaning that the propagation progress for individual organoid structures can be adjusted accordingly. However, it would also be possible to control the propagation device in Box 3015 collectively for a group of several organoid structures.

[0084] Further details of the process from FIG. 1 are described below.

[0085] In detail, Box 3010 allows one or more values ​​of one or more properties to be determined for each of several organoid structures. This enables instance-specific monitoring. In this way, it is possible to differentiate whether the process is more advanced for individual organoid structures than for others.

[0086] Optionally, in Box 3010, it would be possible to determine the distribution of the one or more values ​​calculated for the various organoid structures. This is particularly helpful when the organoid structures are present in a group, for example, as an aggregate or even loosely in a common tray. Based on such a distribution, it can be checked whether certain organoid structures within the group are further along in their growth process than other organoid structures within the same group. It can also be checked whether certain "outlier" organoid structures exhibit malformation.

[0087] If individual adjustment of the growing parameters for individual organoid structures is possible (because, for example, they are in separate trays), the automated growing device can also be controlled on an instance-specific basis in Box 3015. For example, it would be conceivable to set growing parameters differently or separately for different process trays of the automated growing device containing individual organoid structures. For example, a pH value could be continuously adjusted within a process step. It would also be conceivable to continuously adjust the concentration during a process step. Alternatively, or in addition to such instance-specific setting of growing parameters, process steps of the process could also be selectively terminated or initiated for specific organoid structures in different process trays.The process can be started depending on whether one or more values ​​of one or more properties of the respective organoid structures correspond to a specification defined in the process protocol. If several organoid structures are grown together in a cluster (in a shared process tank or even in a single unit), instance-specific control of the automated growing device is not possible. In such a case, it would be conceivable to set process parameter values ​​and / or trigger or terminate a process step based on a distribution of the instance-specifically determined one or more values ​​for one or more properties of the organoid structures in the cluster. For example, only a subset of all organoid structures in the cluster could be considered, with this subset being determined based on the distribution. Other substances besides organoids could also be considered, such as...Extracellular matrices (e.g., Matrigel). For example, it would be conceivable to determine the subset based on the half-width of the distribution. Side peaks of the distribution can be ignored because these can, for example, be attributed to organoid structures with a developmental defect.

[0088] For example, (very) good organoid structures can be included (e.g., mean + 3 sigma) and (very) poor ones can be excluded (e.g., mean - 3 sigma). That is, the subset can be defined in such a way that the distribution of values ​​within the subset does not need to be symmetrical. "Cutting off at the bottom" is much more important than "cutting off at the top."

[0089] Other criteria for selecting the subset are conceivable. For example, the distribution can be modeled with a different distributional assumption than a Gaussian distribution, such as a Gaussian mixture model, a chi-square distribution, or a Poisson distribution (for discrete values). The selection can then be derived from this.

[0090] For example, it is conceivable that in an aggregate comprising a large number of organoid structures, a certain proportion of these structures will always be outside the norm. In such a case, progressing to the next process step might be justified for the aggregate as a whole, even if individual organoid structures would require further processing in the current step. However, such further processing could damage the majority of the organoid structures within the aggregate.

[0091] When multiple organoid structures are present, the cultivation process cannot be individually adjusted for each structure. However, once a certain stage of cultivation has progressed, it may be desirable to separate the organoid structures that were previously cultivated in parallel for individual manipulation. A specific process step can therefore include such separation.

[0092] For example, it would be conceivable to determine a distribution of instance-specific values ​​of properties of the organoid structures of a composite and then perform singulation if this distribution exhibits certain properties.

[0093] For example, the width (e.g., half-width) of such a distribution of one or more values ​​could be compared to a threshold value. The width of the distribution describes the deviation of the observed values ​​within the group. If the deviation exceeds a certain threshold, it can be checked whether individual organoid structures within a group exhibit similar or identical values ​​for properties—such as size, shape, color, intensity, structure, or texture, etc.—or whether such property values ​​differ significantly between the organoid structures within the group. Such a significant deviation may necessitate singulation or sorting. This singulation or sorting can be performed using a manipulator in the automated propagation device in Box 3015.It should be understood that sometimes it is not possible to isolate organoid structures. This is particularly true when the organoid structures of a complex are arranged within an aggregate. In such a case, it can be helpful to determine instance-specific values ​​of the aggregate's properties and, based on these values, to determine whether isolating the organoid structures within the aggregate feasible. For example, the size of the aggregate or the number of organoid structures within it could be determined. The inter-organoid distance between the structures within the aggregate could also be determined. Based on such properties, a decision can then be made as to whether is possible or not.

[0094] In principle, it is possible to determine one or more values ​​of properties for the current state of the organoid structures in Box 3010. Alternatively or additionally, it would also be conceivable to make a prediction of one or more property values ​​for a future point in time. That is, determining the values ​​could include predicting one or more property values ​​for a future point in time.

[0095] FIG. 2 illustrates the control of an automated rearing device based on a process protocol that defines several process steps which are triggered by events. The method from FIG. 2 can therefore be implemented, for example, as part of Box 3010 and Box 3015 from FIG. 1.

[0096] It is also conceivable that a process step outside the automated growing device is carried out manually. Such a process step would be, for example, "offboarding" plates (i.e., removing a plate from a system in which the process is taking place) for manual handling and subsequently "onboarding" the plates (i.e., inserting the plate into the system) into the automated process. It would be conceivable that the method from FIG. 2 could be used for each of several

[0097] Organoid structures are executed in parallel. This is made possible when individual

[0098] Organoid structures can be exposed to individual rearing conditions.

[0099] First, in the current iteration 3120 of Box 3105, a current process step is executed. Details of the process step are defined in the process protocol. For example, specific growing conditions for an organoid structure may be defined. A particular nutrient solution might be defined, or a specific pH value might be specified. Then, in step 3110, it is checked whether an event has occurred that terminates the current process step and / or triggers the next process step. If the event has not yet occurred, the preceding process step can continue. Otherwise, the next process step is executed in a further iteration 3120 of Box 3105. Optionally, in Box 3130, a work instruction can be issued to a user via a user interface if the next process step requires user intervention.

[0100] Next, details of the various aspects of the process from FIG. 2 will be explained.

[0101] Various monitoring methods are conceivable in connection with Box 3110 to obtain monitoring information that allows for the determination of whether an event has occurred. One specific possibility involves capturing an image of a container holding one or more organoid structures. The image (e.g., a microscope image) can then be analyzed to determine whether an event has occurred. In one variant of the analysis, the presence of an event can be determined directly from the image. Alternatively, one or more parameters could be derived from the image, and a subsequent analysis could then be performed to determine whether an event has occurred. For example, values ​​for one or more properties of individual organoid structures could be determined.Once the values ​​for one or more properties have been determined, these values ​​can be used to verify whether the event has occurred. For example, if several organoid structures are present in a complex, a corresponding distribution of values ​​can be used to verify whether an event has taken place. In Box 3110, in addition to monitoring whether a specific event occurs, one or more boundary conditions can also be monitored. An example would be a temporal boundary condition. For instance, it is conceivable that no precise time markers are defined within the process step, but rather maximum or minimum time limits are specified. Thus, it would be possible for the process protocol to specify how long a particular process step should be executed at a minimum and / or how long a particular process step should be executed at a maximum.Before this minimum time period is reached, Box 3110 cannot decide whether to execute the next process step. However, it is possible that the process step will continue to execute even after the minimum time period is reached because the event terminating the process step has not yet occurred. Accordingly, the system can wait for the maximum time period before executing the next process step. Such boundary conditions can be defined not only in terms of time but also with respect to other process conditions. For example, it would be possible to obtain information about rearing conditions from one or more sensors. Examples would be temperature or pH value. It would be conceivable, for instance, that the next process step is triggered whenever the pH value exceeds a certain threshold.Other examples would be the release of a specific substance (for example, an organ-specific protein); the consumption of certain nutrients or the accumulation of toxins in the culture medium; or even the onset of mechanical or electrical activity of the organoid structure. In this way, the detection of an event that triggers the next process step can be supported by monitoring other sensor data.

[0102] Sometimes, in Box 3110, in addition to the occurrence of the desired event, it is also possible that the process for a specific organoid structure is proceeding outside the norm. This could be because, for example, the event triggering the next process step does not occur within a certain timeframe, meaning that the organoid structure no longer shows any further development. Another possibility, however, is that monitoring reveals that the organoid structure is exhibiting abnormal development. In such cases, the cultivation of this organoid structure could be terminated (Box 3115). Generally, various termination criteria can be flexibly defined.Examples of termination criteria include the exhaustion of a predetermined time budget (for a specific process step and a set of process steps), and / or the exhaustion of a specific resource budget (for example, the consumption of a certain amount of nutrient solution), and / or the detection of abnormal development of the organoid structure—that is, development occurring outside the norm, for example, due to a lack of (further) growth or a negative change in appearance. Furthermore, if several organoid structures are cultured together in a single tank, a poorly developing organoid can be promptly removed from the culture, as it may be releasing substances that could affect the development of the other structures in the tank. This can significantly increase the success rate of the entire organoid structure.Using techniques like those described above in connection with Box 3115, the reproducibility of the cultivation process can be significantly improved. The variation in the properties of organoid structures produced by such processes is reduced, thereby increasing the reliability of the cultivation process.

[0103] A demolition could be implemented, for example, by removing 'bad' structures from a vat containing multiple structures, emptying an entire vat, discarding an entire slab (if all vats in it are affected), or transferring 'good' structures, or the contents of the unaffected vats, to a new slab.

[0104] In principle, Box 3110 allows for the determination of current values ​​of one or more properties of organoid structures based on a monitoring modality. It would also be conceivable to determine predicted values, that is, values ​​for one or more properties that will be present or expected to be present at a future time, for example, based on prior knowledge such as historical data. Based on such predicted values ​​from monitoring the cultivation process, a timeline for the cultivation of various organoid structures could be predicted. It would be possible to control a user interface to output such a predicted timeline to a user. Prediction can be particularly helpful when, for example, user intervention (cf.Box 3130) must be planned with a certain lead time, meaning that the user is to be provided with temporal contextual information along with the request for user intervention, indicating when the corresponding user intervention should take place. For example, the urgency of the user intervention can be specified. Such a prediction of values ​​for one or more properties of the organoid structures, or more generally, a prediction of a time at which the respective process step is completed and the next process step is to be executed, is not only helpful in connection with the need for manual user intervention. Sometimes, certain modules of an automated growing device may have a limited throughput. If a module is occupied (i.e., currently unavailable), other pending manipulations must be postponed. A waiting list can be maintained.The cultivation of different organoid structures can be prioritized. Resource planning (e.g., waitlist-based) can then be implemented for the various modules of the cultivation device based on a corresponding prediction. For example, the manipulation of individual organoid structures can be prioritized based on a corresponding urgency, which is determined by predicting the values ​​of one or more properties of the organoid structures. Sometimes, certain modules of an automated cultivation device may require a certain amount of lead time to perform the manipulation. This applies, for example, to changing a temperature or preparing a specific nutrient solution. Within the framework of resource planning, corresponding preparatory processes can be triggered on a timed basis.

[0105] A predefined model can be used for decision-making in step 3110. For example, a machine-learned model can be used. Several machine-learned models are known that can be used in conjunction with the techniques described herein. Details are described below.

[0106] For example, machine-learned models can be used to provide localization of organoid structures, such as in an image. A list of localization information can be provided, with each piece of localization indicating the position of a specific organoid structure within the image. Determining such localization information is particularly helpful when an image depicts a large number of organoid structures within a cluster. Specifically, before the organoid structures are isolated, the localization information can be used to determine instance-specific values ​​for one or more properties, for example, for each organoid structure within the cluster, despite the presence of multiple structures within the cluster. Localization information can also be useful for performing foreground / background separation.Such localization information could also be used to control a manipulator in order to grasp and isolate individual organoid structures.

[0107] Based on the localization information (if available), further analysis can be performed for the individual organoid structures. For example, image patches could be extracted, each showing a single organoid structure. Then, based on a subsequent analysis of such image patches, an instance-specific evaluation can be performed. For example, an instance-specific classification or regression could be carried out.

[0108] Localization and further analysis can also be carried out together in one model.

[0109] Another type of machine-learned model that can be used in conjunction with the techniques described herein is a classifier. For example, it would be conceivable to extract an image patch for each localized organoid structure and then perform a classification of that structure. The classification could be binary, for instance, to indicate whether the organoid structure has reached a growth state corresponding to the completion of the superior process step. In other words, it could classify whether an event has occurred that triggers the "Yes" branch from Box 3110. In such a case, it would be unnecessary to determine specific non-latent features for an organoid structure and then perform a subsequent analysis (e.g., using heuristic rules) based on them.This end-to-end classification has the advantage that it eliminates the need to define quantitative criteria for reaching an event. In other words, the event can be defined (directly) purely based on images. This allows for intuitive configuration of the process log, for example, by manually annotating images during a supervised learning step by a domain expert ("event yes" vs. "event no").

[0110] Alternatively or additionally to such a classifier, a machine-learned model could be used that performs a regression analysis (based, for example, on classical machine learning or deep learning) and provides continuous values ​​for one or more geometric properties such as morphology, size, shape, or color, or even the distance to neighboring organoid structures for an organoid structure (this is particularly helpful when an aggregate is present). Such values ​​can then be compared with target values ​​to decide whether the current process step should continue or the next process step should be executed (or optionally, whether to abort, Box 3115).

[0111] Instead of or in addition to such a regression analysis, a segmentation of the organoid structures could be performed. For example, a semantic segmentation could be carried out, which divides the depicted organoid structures or areas within the organoid structures into several classes and provides a segmentation of the instances belonging to each class. Then, corresponding segmentation masks (or comparable output files of the segmentation) can be compared with specific target values. Further characteristic properties can also be derived from the segmentation results. For example, an average size of the organoids could be determined, perhaps using a heuristic algorithm, and then compared with a threshold value. A distribution of properties such as morphology, size, shape, or color, or even distance to neighboring organoid structures, could also be determined.The shape of the distribution or its properties could then be compared with a target specification.

[0112] This technique, in which the machine-learned model does not make a definitive prediction about the occurrence of an event but is instead used to determine specific values, offers the following advantage: quantitative specifications for certain geometric properties can be defined within the process protocol. This makes the process protocol particularly easy to edit and understand. Furthermore, this technique allows for more nuanced decision-making when multiple organoid structures are grown together in a single system. Additionally, such machine-learned models—for example, for segmentation—can be trained with relatively few examples in a training dataset (although this may result in increased annotation effort).

[0113] Another possibility for a machine-learned model would be the detection of anomalies. Previously unseen variants of organoid structures can be identified, and, for example, a corresponding user output with a warning can be provided. The process could also be terminated upon the presence of an anomaly (Box 3115). Anomaly detection can be enabled through autoencoder techniques or, more generally, unsupervised learning. This allows for the inclusion of an additional safety component without significant manual annotation effort by providing the termination criterion (Box 3115).

[0114] Anomaly detection can be performed both spatially resolved (i.e., for pixels or regions) and based on the entire image or a section of the image.

[0115] The autoencoder (AE) can, for example, learn what "familiar" images look like by reconstructing them (under the constraints of limited capacity, such as in bottlenecks) and minimizing the pixel-wise reconstruction error (or a derived parameter) during training. New images will also be reconstructed by the AE, and the difference between the reconstruction and the original image provides an indication of novelty for each pixel or region – the greater the difference, the more "abnormal."

[0116] At the image level, the entire image (or a section thereof) can be represented by a feature vector (for example, image features such as HOG, SIFT, etc., or activations from a learned encoder, e.g., an AE), and the distribution of feature vectors from known images can be learned—for example, using a Gaussian mixture model or Parzen density estimation. For a new image, the feature vector is determined, along with the "probability" that this feature vector is generated by the distribution model—i.e., p(x). The smaller the value for p(x), the more "anomalous." Such machine-learned models, as described above, can therefore process images, such as microscope images with phase contrast. It would also be possible for such machine-learned models to receive additional contextual information as input.For example, such contextual information could be passed to a corresponding encoder branch in the form of a concatenated feature vector. This could then be combined in a feature space with a vector for latent features, which is obtained through the processing of the input image data. Examples of such contextual information include additional sensor values ​​or measurements that describe the growing conditions for the respective organoid structures. For example, an indication of the temperature or pH value of the environment surrounding a specific, imaged organoid structure could be passed to the machine-learned model along with the corresponding image. Similarly, an indication of the time elapsed since the start of the respective process step could also be passed.Another technique involves providing the machine-learned model with additional images of other organoid structures in the same assembly or in a different vat undergoing identical processing as reference images. Predefined reference images can also be used. Such information can serve as a reference to identify deviations from comparable organoid structures. These techniques are based on the understanding that the accuracy in determining whether a specific process step is complete improves when additional contextual information is available.

[0117] In principle, it is possible for the same or different machine-learned models to be used in multiple iterations 3120 of Box 3110. For example, it would be possible for the same machine-learned model—at least within a growth phase—to be used for localizing organoid structures. However, it would also be conceivable to use a model specifically trained for each iteration 3120 or process step. Such machine-learned models can be trained, for example, in a process-step-specific manner, such as through supervised learning, in which domain experts annotate, for each sequence of images obtained for the specific process step, whether the respective process step has been completed or not.For example, a separate segmentation model can be trained for each step, precisely understanding and segmenting the visual characteristics at that stage. The segmentation analysis can then be performed by another evaluation model, individually developed for each step. This process-step-specific selection of machine-learned models offers the advantage of highly reliable control. This approach is based on the understanding that the appearance of organoid structures can change significantly across growth phases; therefore, it is helpful to maintain individual machine-learned models for the various process steps in the different growth phases. To reduce annotation effort, it would be conceivable to derive the various machine-learned models from a common (source) model, such as a foundation model, through adaptation.

[0118] The image processing models can, for example, include a convolutional neural network. The machine-learned image processing models can, for example, incorporate a Vision Transformer (ViT) architecture. This ViT model can be pre-trained via masked auto-encoding or contrastive learning. The ViT model can then be retrained (fine-tuned). This can be done based on the annotated data obtained specifically for each process step (as described above). The image data can be pre-scaled to a reference size. Image data for training can be augmented. Backpropagation can then be used to adjust the ViT model weights. For example, it would be possible to use individual machine-learned models for at least some of the 3120 iterations, that is, for at least some of the different process steps.However, these process-step-specific machine-learned models can be obtained from the same foundation model through adapted training (fine-tuning). This reduces the necessary annotation effort.

[0119] In various examples, the same Foundation model can be used for multiple iterations 3120 to perform corresponding tasks. Such a Foundation model can be not only non-specific to the respective iteration 3120, but optionally also non-specific to the corresponding domain of analyzing microscope images depicting organoid structures. For example, exempling can be used in conjunction with the Foundation model. This would involve passing a reference image to the Foundation model, along with the current microscope image representing the actual state of the respective instance, to determine whether a significant deviation still exists between the actual and target states, or whether the relevant process step has been completed.Image-based foundation models can be used in the various examples described herein. Such image-based foundation models can be configured by a user via text prompts. For example, during a planning phase, the user could specify which properties of depicted organoid structures should be checked by the respective image-based foundation model. This can be done, for example, by entering text such as "large round object, at least ten differentiated cells visible, clearly defined lumen".

[0120] In various examples, a foundation model that has undergone fine-tuning can also be used for multiple iterations (3120). For example, an image encoder branch of a foundation model can be selectively fine-tuned, e.g., based on microscope images.

[0121] If a foundation model has, for example, a text encoder branch and an image encoder branch, at least the image encoder branch could be fine-tuned or adjusted using a downstream projection head. Fine-tuning can be achieved, for instance, using a contrastive loss function. Contrastive learning rewards the model or model part that moves similar data points closer together (for example, in the feature space after the encoder branch) and dissimilar data points further apart. This is done by comparing pairs or groups of data points to learn the similarities and differences. Contrastive learning can use a contrastive loss function that considers image and text feature vectors (resulting from identical or dissimilar text-image pairs) in the latent feature space of the image and text encoder branches.This simplifies the subsequent adaptation to the specific task, as the model is already specifically tailored to microscope images, which can optionally even show organoids, and only needs to learn what exactly is relevant for the task at hand. Instead of fine-tuning a foundation model, a foundation model could also be used that is configured (but not trained) by providing example images and associated annotations. This allows, for example, process-step-specific configuration (i.e., different configurations for the various iterations 3120) to determine which substructures need to be analyzed for the successful completion of the process step.

[0122] FIG. 3 shows an exemplary system 200 configured to carry out the method described above. The system 200 comprises a control device 202 and an automated rearing device 204. The elements of the rearing device 204 can be physically contained within a single housing or be physically separate. The control device 202 can, for example, be an electronic control system, such as a computer system. The rearing device 204 can be at least partially automated and coupled to the control device 202.

[0123] The control device 202 can, for example, be one or more computer systems with main memory, mass storage, a processing unit, a user interface, and input / output interfaces for coupling with the rearing device 204. For example, one computer system can be provided for the automated execution of the process protocol, e.g., for controlling the rearing device 204, and another computer system for monitoring the execution of the process protocol. The control device 202 can be located near the rearing device 204. In other examples, the control device 202 can be located remotely from the rearing device 204 (remote computing) and coupled to the rearing device 204 via a remote data connection. The control device 202 can be configured to perform the procedures described herein.In particular, program code (software) can be provided in a working memory of the control device 202 which, when executed by the processing unit of the control device 202, performs the procedures described herein.

[0124] The growing device 204 can have a sterile interior in which several plates 206 are arranged, for example, on a shelf (so-called "plate hotel"). In addition to sterility, the growing device 204 can also ensure an optimal temperature, humidity, and gas atmosphere for growth (for example, regulated CO2, O2, or N2). In the example shown in FIG. 3, nine plates 206A-206J are depicted. However, this number of plates is exemplary, and any other number of plates 206 can be provided in the growing device 204. The plates 206 can also be arranged differently, for example, on a conveyor belt or in a carousel. Each plate 206 has several process trays 208. In the example shown in FIG. 3, each plate 206 has four trays 208A-208D. This number is only exemplary.In other examples, each plate 206 can have more than four wells, for example, 6, 12, 24, 48, 96, 384, 1536, 3456, 9600, or each plate 206 can have only one well 208. The plates 206 can have different numbers of wells 208. Each well 208 can contain a nutrient solution in which an organoid structure or a composite of several organoid structures can be grown. Alternatively or additionally, each well 208 can contain other substances, for example, gel-like substances, to support the growth of the organoid structures. The plates can have lids (not shown) to ensure closure and / or sterility, which are removed before treatment. The wells can be shaped differently, for example, depending on the organoid structures to be produced or to create different cultivation environments.One or more lighting devices in the growth apparatus 204 can irradiate one or more trays with light, for example, infrared light, visible light, and / or ultraviolet light. The trays can be coupled to mechanical actuators, for example, a shaking or pivoting actuator (e.g., an orbital shaker), to allow slight movement of the organoid structures in their nutrient solution.

[0125] The rearing device 204 may further comprise one or more monitoring modalities 210, one or more actuator or robot devices 212, and one or more treatment devices 214. Modules for implementing a respective monitoring modality may, for example, be provided in each shelf location or at a central location, so that a robot device grasps the respective tray and moves it to the central location to perform the corresponding measurement.

[0126] A transport device (not shown) can move one of the plates 206 such that one of the trays 208 of the corresponding plate 206 is positioned in a working area 216 of the monitoring modalities 210, the robot devices 212, and the treatment devices 214. In other embodiments, several working areas 216 with corresponding monitoring modalities, robot devices, and treatment devices can be provided, so that several trays 208 can be processed simultaneously. In the example shown in FIG. 3, for instance, tray 208B of plate 206E is located in the working area 216 of the monitoring modality 210, the robot device 212, and the treatment device 214.

[0127] Positional information generated during monitoring can be transmitted to the treatment device. This information can be relative to the organoid structure, or to the tub or treatment table. The treatment device may need to determine or at least correct these positions itself (for example, because the organoid structures can move within the tub). For this purpose, the treatment device can either have its own imaging capabilities or utilize the imaging capabilities of the monitoring system.

[0128] Since plates are easier to transport and a pipetting robot or microscope can each take up a lot of space, it's possible that not all actions are performed in the (physically) same work area. For example, the plate might be positioned differently for pipetting than for image acquisition. In some cases (e.g., during picking), however, it can be important that imaging and treatment take place at the same location.

[0129] The monitoring modalities 210 can include, for example, a camera, a temperature sensor, a pressure sensor, an electrical activity sensor, a pH sensor, and the like. The monitoring modalities 210 can be used to record properties of the organoid structure culture and / or properties of the culture medium in the vat 208B, such as the temperature and pH of the culture medium, the ambient pressure in the area of ​​the vat 208B, and an image of the organoid structure culture in the vat 208B.

[0130] The robotic device 212 can, for example, perform mechanical tasks, such as stirring the culture medium in the vat 208B, disassembling the organoid structure culture into different parts, or removing parts of the organoid structure culture from the vat 208B. The robotic device 212 is shown schematically only in FIG. 3. The robotic device 212 can comprise several robotic devices, for example, a robotic device for moving the plate 206 and opening and closing the vats 208, and another robotic device or manipulator for handling the organoid structures and the medium in the vats 208.

[0131] The treatment device 214 can, for example, introduce substances into the vat 208B, such as adding liquid or solid substances, or remove parts or all of the solution from the vat 208B, for example by suction, and replace it with a new solution. The treatment device 214 can, for example, take samples of the solution in the vat 208B for analysis, for example, using appropriate monitoring modalities 210. For example, the composition of the solution in the vat 208B can be determined, such as its nutrient content. Individual or groups of organoid structures can also be removed from their plate by the treatment device and transferred to a new culture vessel. The aim may be to keep the individual objects intact. However, it may also be to deliberately break down the objects into cell aggregates of various sizes, down to individual cells.

[0132] The treatment device 214 can also be used to transfer substances into plates without cells, for example to prepare plates for use with cells / organoid structures by applying a surface coating.

[0133] Another embodiment of the treatment device 214 can, for example, separate parts of the organoid structures, cut them, crush them or otherwise mechanically process them.

[0134] By coupling the growing device 204 with the control device 202, the control device 202 can move any tub 208 of any plate 206 into the working area 216 at a desired time and control and monitor the growing of the organoid structures in the tub 208 according to a process protocol.

[0135] In particular, the method described in FIGS. 1 and FIGS. 2 can be carried out with the System 200 to grow organoid structures. FIG. 4 shows exemplary details of the control and monitoring of the growth of organoid structures according to a process protocol. Reference numeral 302 denotes a time axis t over the course of the growth of organoid structures. The process protocol can, for example, specify process steps E1 to E5, which are to be carried out at specific times. The execution of these process steps can take place earlier or later, as shown by the dashed arrows in relation to process step E1. This is because the process steps are event-triggered. Process step E1 is carried out when a specific event occurs; however, process step E1 always takes place before process step E2.Process step E1, for example, could indicate a 50% change of the medium in which the organoid structure is grown. Process step E2, for example, could indicate a 100% change of the medium. Process step E3, for example, could indicate the addition of a first growth factor to the medium, and process step E4, for example, could indicate the addition of a second growth factor to the medium. Process step E5, for example, could be performed when the organoid is complete and indicate another 100% change of the medium.

[0136] During the execution of the process protocol (see step 3015 in FIG. 1), the state of the organoid structure culture can be continuously or at specific time points using, for example, an imaging device (see step 3010 in FIG. 1). Reference numeral 306 denotes corresponding images over time t. Some or all of the acquired images of the organoid structure culture can be evaluated by an operator, particularly an expert, with regard to the quality of the generated organoids and annotated accordingly (see reference numeral 308). Physical parameters can also be recorded during the execution of the process protocol (see reference numeral 310).

[0137] Monitoring the state of the organoid structure culture using, for example, an imaging device, can include continuous, "simple" and sample-friendly imaging, such as transmitted light imaging. Based on the evaluation of this "simple" and sample-friendly imaging, a second, more "informative" imaging can be performed, such as autofluorescence imaging, which is less sample-friendly. For example, the second, more "informative" imaging is not performed continuously, but only when the structures appear sufficiently clear in the "simple" and sample-friendly imaging to then verify the occurrence of the event in a second, more "informative" imaging.

[0138] Several execution instances of a process protocol can be carried out sequentially or simultaneously in several vats 208 (corresponding to several parallel executions of the method from FIG. 1 and FIG. 2) in order to make the best possible use of the available capacity of the rearing device 204.

[0139] For good scalability, it can be advantageous to automate as many process steps of the process protocol as possible. This can be achieved using an automated growing device, as described in connection with FIG. 3, and continuous image acquisition to monitor the progress of the current process steps. In particular, the processing of the image data can be carried out using machine learning methods, for example, based on the annotated images (reference numbers 306 and 308).

[0140] In summary, the preceding sections have described techniques that enable instance-specific processes for cultivating organoid structures. This is achieved through monitoring, which can also be performed on an instance-specific basis. Furthermore, an automated cultivation device can be controlled in such a way that the cultivation parameters for individual organoid structures can be set individually, or individual process steps of a cultivation process can be triggered event-driven for individual organoid structures. This is based on the understanding that, even with nominally identical process protocols, the complexity of organoid structures leads to individually different results if no control loop adjusts the cultivation based on monitoring. Using the techniques described herein, poorly developing organoid structures can also be sorted out and discarded.

[0141] Naturally, the features of the embodiments and aspects of the invention described above can be combined with one another. In particular, the features can be used not only in the combinations described, but also in other combinations or individually, without leaving the scope of the invention. Thus, the term "organoid structure" is not to be interpreted as limited to the final product, i.e., the finished organoid, but also includes precursors or pre-products of an organoid during its cultivation, especially 3D cell cultures that can arise during the development of a finished organoid, so-called organoid precursors. The term "organoid structure" therefore also includes 3D cell structures, for example, cysts or embryonic bodies.

[0142] Furthermore, techniques for growing organoid structures have been described above. The methods disclosed herein are also generally applicable to other 3D cell cultures, such as spheroids and tumoroids. As with the growth of organoid structures, it has been observed that even with nominally identical growth protocols, instance-specific variations in growth results or growth rate can occur with spheroids and tumoroids. Therefore, in such cases, it can also be helpful to apply the techniques described herein for finding optimized process protocols.

Claims

Patent claims 1. Procedure, comprehensive: - Providing (3005) a process protocol describing a process with several successive process steps for growing organoid structures in an automated growing device, wherein the process protocol specifies for at least one process step of the process a specification for one or more values ​​of one or more properties of the organoid structures that triggers or terminates the respective process step, - Monitoring (3010) the growth of the organoid structures by means of one or more monitoring modalities of the automated growth device, and -Controlling (3015) the automated rearing device based on the process protocol and monitoring.

2. The method of claim 1, wherein monitoring (3010) the growth of the organoid structures comprises: - Instance-specific determination of one or more values ​​of one or more properties for individual organoid structures or a group of organoid structures.

3. Method according to claim 1 or 2, wherein monitoring (3010) the growth of the organoid structures comprises: - Instance-specific determination of one or more values ​​for the quantity or position of extracellular matrices in relation to individual organoid structures or a cluster of organoid structures.

4. Method according to claim 2 or 3, wherein the respective one or more instance-specific values ​​refer to individual organoid structures and / or a composite of physically connected organoid structures and / or a composite of physically unconnected organoid structures and / or all organoid structures in one well of a multiwell plate and / or all organoid structures in a multiwell plate.

5. Method according to one of claims 2-4, wherein the control (3015) of the automated rearing device comprises: - Instance-specific selective triggering or termination of at least one process step if monitoring shows that the respective one or more instance-specific values ​​correspond to the specification.

6. Method according to any one of claims 2-5, wherein monitoring (3010) the growth of the organoid structures comprises: - Based on the instance-specific determined one or more values ​​of the one or more properties for individual organoid structures, determining a distribution of the one or more values ​​for such organoid structures that are in a composite.

7. Method according to claim 6, wherein controlling (3015) the automated rearing device comprises: - selectively triggering or terminating at least one process step if monitoring (3010) shows that for a specific subset of the organoid structures in the network, determined based on the distribution, the respective one or more instance-specific values ​​meet the specification.

8. Method according to any of the preceding claims, comprising controlling the automated rearing device: - Setting one or more process parameter values ​​of the process depending on monitoring.

9. Method according to claims 2 and 8, wherein the setting of the one or more process parameter values ​​of the process is instance-specific depending on the instance-specific values ​​of the properties of individual organoid structures.

10. Method according to any one of the preceding claims, where monitoring the growth of the organoid structures involves determining one or more predicted values ​​of one or more properties.

11. The method of claim 10, wherein the method further comprises: - Prioritizing the cultivation of different organoid structures based on the one or more predicted values ​​of the one or more properties and based on the availability of modules of the automated cultivation device.

12. Method according to one of the preceding claims, wherein the automated propagation device is controlled based on monitoring in an instance-specific manner for individual organoid structures.

13. The method of claim 12, wherein the instance-specific control of the automated rearing device comprises: -Controlling a manipulator to perform manipulation of individual organoid structures according to the process protocol or to perform singulation of individual organoid structures.

14. The method according to claims 2 and 13, wherein the method further comprises: - Determining a distribution of instance-specific determined one or more values ​​for organoid structures in a composite, whereby the manipulator is selectively controlled depending on the distribution in order to perform the singulation of the organoid structures in the composite.

15. Method according to any one of claims 12 to 14, wherein the instance-specific control of the automated rearing device comprises: - Separate setting of rearing parameters for different process tubs of the automated rearing device.

16. A method according to any one of the preceding claims, wherein the method further comprises: - Terminating the cultivation of a specific organoid structure when a termination criterion is met, where the termination criterion is optionally selected from the following group: - a predetermined time budget has been used up, - a predetermined resource budget has been exhausted, and / or - A maldevelopment of the specific organoid structure is detected or predicted based on monitoring.

17. A method according to any of the preceding claims, wherein the method further comprises: - based on monitoring, controlling a user interface to issue a work instruction to a user when user intervention is required or possible to complete a process step.

18. The method of claim 17, wherein the method further comprises: - based on monitoring: Determining temporal context information for a user intervention, where the work instruction is indicative of the temporal context information.

19. A method according to any of the preceding claims, wherein the method further comprises: - based on monitoring the growth of the organoid structures and based on the process protocol: controlling a user interface to output a predicted time sequence of the process.

20. Method according to any of the preceding claims, wherein the monitoring comprises capturing an image of the organoid structures, wherein one or more properties are selected from the following group: appearance of organoid structures or parts thereof in the image; morphology; size; shape; color; inter-organoid structure spacing.

21. A method according to any one of the preceding claims, wherein the process protocol specifies a respective target for one or more values ​​of one or more corresponding properties of the organoid structures for several process steps of the process, wherein, when monitoring the growth of the organoid structures for the several process steps, several different models are used to determine the respective one or more values ​​of the one or more corresponding properties of the organoid structures.

22. A method according to claim 21, wherein the several different models comprise machine-learned weights obtained by fine-tuning a common model.

Images