Image processing and segmentation of z-stack image sets of three-dimensional biological samples

Abstract

Provided herein are methods of projecting a cross-depth stack of limited depth of field images of a sample into a single sample image that can provide focused image information of the three-dimensional content of the image. These methods include applying a filter to the stack of images to identify pixels in each image that are in focus. These in-focus pixels are then combined to provide a single image of the sample. Filtering of the stack of images can also be used to determine a depth map or other geometric information about the components of the sample. This depth information can also influence segmentation of the sample image, e.g., further dividing identified regions corresponding to sample components at multiple different depths.

Description

Background Technology

[0001] Various biological experiments include large-sample analyses, each potentially involving the measurement or evaluation of multiple parameters or other information generated from samples. Such samples can include cells or other biological components, each differing in terms of growth medium (e.g., hormones, cytokines, drugs, or other substances in the growth medium), source (e.g., culture, biopsy, or otherwise transplanted from natural tissue), culture conditions (e.g., temperature, pH, light level or spectrum, ionizing radiation), or other controlled conditions, to observe the response of cells or other biological components to applied conditions. These experiments can be conducted, for example, to assess the response of samples to a putative treatment, to elucidate certain biological processes, or to investigate certain other target questions.

[0002] Evaluation of such samples may include microscopic imaging of the sample using a microscope, fluorescence imaging apparatus, or other methods. In practice, limitations of the optical or other equipment used for sample imaging may hinder large depth-of-field imaging of the sample. For example, the objective elements or other components used for sample imaging may be limited in terms of focal depth, making simultaneous imaging difficult or impossible. At focal point, along the optical axis of the apparatus used to image the sample, the volume covered by the sample is greater than the focal depth of the imaging apparatus. Such “three-dimensional” samples, compared to samples distributed on a slide and / or sliced, cover a volume within the depth of field of the imaging apparatus and may contain various target structures, such as organoids, tumor spheroids, or other three-dimensional multicellular structures. Summary of the Invention

[0003] One aspect of the present invention relates to a method for generating a projected image of a three-dimensional sample, the method comprising: (i) acquiring an image set of the sample, wherein each image in the image set corresponds to a respective focal plane in the sample; (ii) applying a filter to each image in the image set to determine a corresponding depth value for each pixel of an output image of the sample, wherein the given depth value represents an intra-sample depth at which the sample component can be focused and imaged; and (iii) determining an image value for each pixel of the output image based on the depth value of the pixel of the output image. Determining the image value of a specific pixel of the output image comprises: (1) identifying an image in the image set corresponding to the depth value of the specific pixel; and (2) determining the image value of the specific pixel based on the pixel of the identified image, wherein the pixel of the identified image has a position within the identified image corresponding to the specific pixel.

[0004] Another aspect of the present invention relates to a method for generating a projected image of a three-dimensional sample, the method comprising: (i) acquiring an image set of the sample, wherein each image in the image set corresponds to a respective focal plane in the sample; (ii) applying a filter to each image in the image set to determine a corresponding depth value for each pixel of a depth map, wherein the depth value represents an intra-sample depth at which the sample component can be focused and imaged; and (iii) determining an image value for each pixel of an output image based on the depth value of the corresponding pixel in the depth map.

[0005] Another aspect of the present invention relates to a method for segmenting a sample image, the method comprising: (i) acquiring a sample image; (ii) acquiring a depth map of sample components; (iii) generating a first segmentation map of the sample based on the image; and (iv) generating a second segmentation map of the sample based on the depth map by further subdividing at least one region of the first segmentation map.

[0006] Another aspect of the invention relates to a computer-readable medium configured to store at least computer-readable instructions that, when executed by one or more processors of a computing device, cause the computing device to perform computer operations to perform one or more methods described herein. Such a computer-readable medium may be a non-transient computer-readable medium.

[0007] Another aspect of the invention relates to a system comprising: (i) one or more processors; and (ii) a non-transitory computer-readable medium configured to store at least computer-readable instructions that, when executed by one or more processors, cause the system to perform one or more methods described herein.

[0008] These and other aspects, advantages, and alternatives will become apparent to those skilled in the art upon reading the following detailed description and, where appropriate, referring to the accompanying drawings. Furthermore, it should be understood that the summary of this document and the descriptions provided elsewhere are intended to illustrate the claimed subject matter by way of example, and not by way of limitation. Attached Figure Description

[0009] Figure 1 This is a schematic diagram of a cross-section of a sample imaged by an imaging system.

[0010] Figure 2A A set of example images depicting the samples.

[0011] Figure 2B Depicting based on Figure 2A The projected image generated from the example image set.

[0012] Figure 3A Example projection images and example depth maps of the samples are depicted.

[0013] Figure 3B Describes the generation Figure 3A A sample image set of depth map samples.

[0014] Figure 3C It is a set of sample texture values ​​determined by the depth range of the samples.

[0015] Figure 4 A sample image set is depicted for generating sample projection images.

[0016] Figure 5A This is an example image of the sample.

[0017] Figure 5B It is by Figure 5A The image represents an example of the first segmentation map of the sample.

[0018] Figure 5C Shown overlaid on the example depth map Figure 5B The specific segment of the first segmentation graph.

[0019] Figure 5D From Figure 5B An example of a second segmentation map determined by the first segmentation map.

[0020] Figure 6 A perspective view of the example system is shown.

[0021] Figure 7 This is a schematic diagram of the components of the example system.

[0022] Figure 8 This is a flowchart of the example method.

[0023] Figure 9 This is a flowchart of the example method.

[0024] Figure 10 This is a flowchart of the example method. Detailed Implementation

[0025] This document describes embodiments of the methods and systems. It should be understood that the terms "exemplary," "example," and "illustrative" are used herein to mean "as an example, instance, or illustration." Any implementation or feature described herein as "exemplary," "example," or "illustrative" is not necessarily to be construed as superior to or beneficial to other implementations or features. Furthermore, the exemplary implementations described herein are not intended to be limiting. It will be readily understood that certain aspects of the disclosed systems and methods can be arranged and combined in a variety of different configurations.

[0026] I. Sample Imaging Example

[0027] Microscopic imaging of samples is beneficial in a variety of applications. For example, samples may include cultured human cells (such as cancer cells or normal cells), and imaging can help determine the effectiveness of drugs in clearing cells from a sample (such as the effectiveness of chemotherapy drugs in clearing cancer cells), the toxicity of substances (such as the toxicity of chemotherapy drugs to non-cancer cells), or other information about the composition of the sample and / or the effects of substances on that composition. Sample imaging may include using bright-field microscopy, fluorescence microscopy, structured illumination, confocal microscopy, or other imaging techniques to generate image data about the composition of the sample.

[0028] The advantage of brightfield microscopy is that it can be performed without the addition of dyes, fluorophores, or other tags (e.g., by adding tags and / or transfecting sample components to express tags) that can alter the “natural” behavior of sample components (this type of imaging, if unlabeled, can be called “tagless” imaging). Alternatively, fluorescent dyes or other tags can be added to facilitate imaging of specific tissue structures and / or physiological processes. For example, a reagent based on Annexin V green fluorescent dye can be added to image the location, rate, or other information of cell death in the sample. In another embodiment, a NucLight Red indicator can be added to image information about cell proliferation in the sample. In some embodiments, cells in the sample can be genetically modified to express fluorescent proteins or other tags associated with the target process. For example, patient-derived neural induced pluripotent stem cells (iPSCs) can be genetically modified to express Aβ. 1-42 -GFP or certain other fluorescent marker proteins associated with Alzheimer's disease to facilitate the evaluation of Alzheimer's disease treatment options.

[0029] Many objectives or other optical elements of an imaging apparatus used for sample imaging may be limited in depth of field. That is, the optical apparatus may be limited to focusing imaging of light received from a sample of relatively small volume and / or essentially planar volume. Light received outside this volume may be received outside the focal point (via charge-coupled devices or other photosensitive elements of the imaging apparatus). This limitation may be related to the cost or overall quality of the imaging apparatus, aimed at reducing chromaticity, axial, spherical, or other aberrations of the imaging apparatus, aimed at reducing the volume, size, weight, or number of parts of the imaging apparatus, aimed at improving the performance of the device by limiting its use for imaging small volumes and / or essentially planar volumes in a certain way, or due to certain other factors. This limitation of the depth of field of the imaging apparatus can be compensated for in a variety of ways.

[0030] In some embodiments, the sample to be imaged may be small in volume and / or substantially planar, or may be modified to be so. However, many target samples may contain single objects that may encompass a volume that is not suitable for the narrow depth of field of the imaging device, and / or may contain useful information about the distribution, interconnection, or other spatial information of multiple objects within the sample, the distances being perpendicular to the imaging plane of the imaging device and extending beyond the narrow depth of field of the imaging device. Such “three-dimensional” samples may include three-dimensional, multicellular structures of organoids, tumor spheres, or some other type of target. Furthermore, such structures may be cultured or otherwise disposed of in a medium (e.g., an extracellular matrix in a dome or other shaped volume) that encompasses a volume that is not suitable for the narrow depth of field of the imaging device. For example, when tumor spheres (or certain other types of spheres) are arranged in layers and embedded within a volume of base gel, the sample may be part of a multi-spheroid assay.

[0031] Organoids (e.g., pancreatic cell organoids, hepatocyte organoids, intestinal cell organoids) and tumor spheres are specific targets because their three-dimensional structure more closely resembles the “natural” three-dimensional environment of the cultured cells. Therefore, the responses of organoids, tumor spheres, or other such three-dimensional multicellular structures to experimental conditions for drugs or other applications are likely to more closely resemble the responses of corresponding samples in simulated human or other target environments. Organoids can be cultured from a patient’s own cells to predict a specific patient’s response to a range of different possible treatments. For example, iPSCs can be extracted from a patient and used to culture neuronal organoids, breast cancer organoids, or organoids from other healthy or non-healthy (e.g., cancerous) tissues similar to the target, to assess the response of these tissues to potential treatments.

[0032] To image the components of such samples using a limited depth-of-field imaging device, various techniques can be employed. In some embodiments, part or all of the sample may be spread onto a microscope slide or other flat component to facilitate imaging. However, sample spreading methods or other sample pretreatment methods that make the sample compatible with a limited depth-of-field imaging device may not be suitable for the target sample, potentially leading to sample damage, loss or distortion of sample information (e.g., sample distortion due to interaction with a microtome, freezing, fixation, removal from the sample container, spreading, etc.), or other side effects, or may not be applicable to a particular target sample. Furthermore, such sample pretreatment methods exclude the possibility of imaging the same sample (e.g., the same sample of cultured organoids or tumor spheres) at multiple time points, for example, analyzing the long-term effects of drugs or other experimental conditions over time.

[0033] Alternatively or alternatively, an imaging device with shallow depth of field may be replaced with an imaging device with deep depth of field; however, such an improved imaging device may be too expensive, too large to be suitable for an incubator or other target environment, or in some other way impractical.

[0034] Using the methods described herein, the aforementioned limitations of finite depth-of-field imaging devices can be completely or partially alleviated. Applying these methods to the operation of such finite depth-of-field imaging devices enables the generation of image information that can be optionally obtained from an imaging device with a depth of field equal to or greater than that of the sample spanning a volume in a direction parallel to the optical axis of the imaging device. The image processing methods described herein include generating multiple images of a “three-dimensional” sample (which may be referred to as a “stack” of images, each image corresponding to a different depth within the sample). By controlling the position of the imaging device relative to the sample using a motor or other actuator, each image can be captured at a different depth within the focused sample. This can include moving the camera, moving the sample container, or moving both the sample container and the camera.

[0035] Figure 1 An example system 100 for generating this image set is shown. The system 100 includes an imaging device 110 (e.g., a camera, objective lens, and / or other optical or electronic components) and a light source 120 for imaging samples contained in a sample container 130. The sample container 130 may be a well of a multi-well incubation plate or culture plate (e.g., a 96-well culture plate). The sample container 130 contains multiple objects 105 (e.g., organoids, tumor spheres, or other three-dimensional biological objects) within a sample medium 135 (e.g., organoids cultured in a domed extracellular matrix, transferred to the wells of a 96-well multi-well culture plate or some other multi-well culture plate, subsequently applied to various hypothetical treatments or other experimental conditions, or multi-sphere experiments involving tumor spheres arranged in layers and embedded within a substrate gel volume).

[0036] Figure 1The arrangement of the components (e.g., camera 110 and light source 120) is intended as a non-limiting example. The light source for illuminating the sample may be located opposite the sample to the camera or other imaging device, on the same side of the sample as the imaging device, on the sample side opposite the imaging device, or at some other location opposite both the sample and the imaging device. Furthermore, the light source may be incorporated into the imaging device and / or may share one or more optical elements or optical paths with the imaging device (e.g., the objective lens of a confocal imaging system may be shared between a laser or other element for illuminating the sample and a charge-coupled device or other element for imaging the light received from the sample). In some embodiments, multiple light sources may be provided at multiple locations, for example, to facilitate imaging of the sample according to different modes (e.g., fluorescence imaging, bright-field imaging, reflected / scattered visible light imaging, structured light illumination imaging, phase-contrast imaging).

[0037] Camera 110 and / or light source 120 can be configured to enable camera 110 to image light received from a small, substantially planar volume within sample container 130. This configuration can be chosen instead of a large depth-of-field configuration to reduce cost, increase reliability, reduce size or weight, or provide other benefits. For example, such a configuration can be chosen to reduce the size and / or weight of the camera to facilitate mounting camera 110, light source 120, and / or other components of the imaging device on a drive rack within the incubator, thereby allowing long-term automated imaging of multiple samples within the incubator at multiple time points. Region 140a, in cross-section, represents the extent of a first example volume that can be focused and imaged by such a limited depth-of-field imaging device. Region 140b, in cross-section, represents another example of this region, which differs from the first region 140a in depth within sample container 130 but covers substantially the same area in a direction perpendicular to the optical axis of camera 110.

[0038] The position of the camera 110 relative to the sample container 130 can be set at different time points to facilitate imaging of different volumes or depths within the sample container 130. This may include changing the position of the camera 110 relative to the sample container 130 using a drive gantry or other methods (e.g., moving the camera 110 and / or moving the sample container 130) to select a flat area within the sample container 130 for focused imaging. For example, in the first time period (e.g., Figure 1During the time period shown, camera 110 can be operated to focus and image a first volume 140a. Then, camera 110 can be moved downwards relative to sample container 130 and operated to focus and image a second volume 140b. This process can be repeated at multiple different volumes and corresponding depths within sample container 130 to generate an image set or image "stack" representing the components of sample container 130. This process can facilitate the generation of focused imaging information for object 105 (e.g., organoid, tumor sphere, or other three-dimensional biological object) having a feature dimension greater than the depth of field of camera 110 and / or located within sample container 130 at a distance greater than the depth of field of camera 110 arranged along a vertical dimension.

[0039] The process of generating a "stack" of image sets can be performed automatically. For example, a camera and associated actuators can be operated to generate image sets of samples at a specified number of time points (e.g., once per hour, once every 24 hours, etc., to facilitate analysis of the response of sample components to applied experimental conditions over time). Alternatively or additionally, the samples can be located within a porous sample container, and an image set can be generated for each sample in the container by activating the camera in two dimensions to select specific samples (e.g., by operating actuators containing the camera mount) and imaging at different depths within the specific sample in a third dimension.

[0040] Figure 2A An example image set (or "stack") 200 of samples is shown. Each image in image set 200 corresponds to a specific focal plane in the sample. Therefore, each image can represent different components of the focused sample based on the position of the components associated with the focal plane corresponding to the image. In a first example, the top image 210a of image set 200 corresponds to a first focal plane, where no sample components are located. Therefore, all image information in the top image 210a is out of focus. In another example, the second image 210b of image set 200 corresponds to a different focal plane, where some sample components are located. Therefore, some image information in the second image 210a (i.e., the 3D object in the lower left corner of the second image) is in focus, while other portions of the image information in the second image 210b are out of focus.

[0041] Such an image set 200 can contain enough focused image information to generate a projected image of the sample, which provides an improved view of the component distribution in the sample. For example, some or all of the components of the sample can be focused in the projected image. The projected image can approximate an image of the sample as if it were generated using a wider depth of field. Figure 2B It shows from Figure 2AThe illustration shows an example of such a projected image 250 generated from an image set 200. This projected image helps improve imaging and the analysis of sample components. For example, the projected image can be segmented to identify discrete cells, organoids, tumor spheres, or other three-dimensional components within the projected image. This image segmentation can then be used to automatically determine the number, size, identification, morphology, or other information about the cells, organoids, or tumor spheres, or certain other analyses of the sample components. Additionally or optionally, this projected image can facilitate subjective analysis of sample components by pathologists, researchers, epidemiologists, or others.

[0042] Projected images of a sample can be generated from image sets corresponding to different focal planes within the sample in various ways. In some examples, this may include using the image set to determine depth, at which sample components are located and / or focused. The corresponding pixels of the projected image can then be generated using pixels or other image information (or “projection”) from the images or image set that correspond to the determined depth. Additionally or alternatively, the depth information may be used to generate depth maps of the sample components (e.g., to facilitate 3D geometric analysis and / or the alignment of sample components) and / or to improve the segmentation of the projected image (or certain other images of the sample).

[0043] II. Depth Map Generation Example

[0044] A narrow depth-of-field image set (e.g., bright-field images) corresponding to different depths, covering a depth range within the sample, can contain sufficient information to determine the depth of sample components. This "stacked" image set can also contain sufficient image information that can be combined with depth information to generate a sample projection image (e.g., as a form of simulated wide depth-of-field image of the sample) representing sample components focused within a depth range. The method described herein facilitates the generation of such depth information (e.g., depth maps) and projection images from a narrow field-of-field image set (e.g., bright-field images, fluorescence images).

[0045] These methods involve detecting edges, textures, or other high spatial frequency components in each image of the image set. The presence of such high spatial frequency components at specific locations within a particular image indicates that the sample component at that location is focused and imaged. Therefore, it can be assumed that the sample component at that location lies at a depth in the sample corresponding to the depth of that particular image. Thus, a depth map or other depth information can be generated for the sample by identifying “focused” regions in each image of the image set. These identified regions can then be combined across the entire image set to generate a single depth map and / or projected image of the sample.

[0046] The “focused” region in each image of the image set for identifying samples can include applying a filter or performing a transformation on each image. For example, the Canny edge detector or other edge detection filters or algorithms can be applied to generate a separate “edge image” for each image in the image set, which represents the location of the edge in each image. In another example, a texture filter can be applied to each image in the image set to generate a separate “texture image”, which represents the location of the region in each image where added high spatial frequency information (or “texture”) is located.

[0047] Texture can be determined in a variety of ways. For example, the texture value of a particular pixel in an image can be determined by determining the entropy, numerical range, standard deviation, variance, coefficient of variation, or some other measure of variability of the set of pixels in the neighborhood of that particular pixel. This neighborhood can be a square or other shaped region of neighboring pixels, such as a 5x5 square of pixels located at the center of the particular pixel whose texture value is being determined. Pixels in the “neighboring” region can be weighted equally or used in a weighted manner to determine the texture value (e.g., by giving higher weights to pixels farther away than the particular pixel whose texture value is being determined when determining entropy, standard deviation, etc.). The texture value can be determined for each pixel in the image, such as a subset of pixels in the image (e.g., for each other pixel) or some other location in the image.

[0048] As described above, regions of a specific narrow depth-of-field image with higher texture values ​​(or higher values ​​of certain other properties related to the high spatial frequency content of the image) are more likely to be focused and imaged. Therefore, sample components corresponding to a specific high-texture location in a specific image of a sample are likely located at a depth within the sample corresponding to the focal plane depth of said specific image. Texture information at a specific location in the image set (e.g., the location corresponding to a specific pixel index in each image) can be compared across all images to determine a single depth value at that specific location. This determination can be used to generate a full-depth map of the sample and / or to generate pixels of a projected output image of the sample (e.g., by selecting pixels in the image set that correspond to a determined depth used to generate the pixels of the projected image).

[0049] To illustrate this process, Figure 3A Image 301 of the sample is shown (e.g., a projected image generated as described herein). Image 301 is shown in the vicinity of depth map 300 of the sample. Each object in image 301 may include one or more organoids, tumors, or other three-dimensional multicellular objects. Depth map 300 shows that some seemingly exotic objects depicted in image 301 may depict multiple overlapping objects at different depths within the sample.

[0050] Figure 3BA depth map 300 of a sample and narrow depth-of-field image sets 310a, 310b, 310c, and 310d corresponding to samples at different focal planes at different depths within the sample are shown. The depth map 300 and the images in the image sets have the same resolution and pixel location for ease of interpretation. Those skilled in the art will recognize that the method described herein can be adapted to depth maps, projected images, and / or input images of image sets with different resolutions and / or pixel locations.

[0051] The depth value of a specific pixel 305 in the depth map 300 can be determined based on the texture values ​​(or other values ​​representing the local amplitude of high spatial frequency image content) of corresponding pixels 315a, 315b, 315c, and 315d in each image of the image set 310a, 310b, 310c, and 310d. For example... Figure 3B As shown, the images and depth maps in the image set can have the same size and resolution, in which case the pixels may be in a one-to-one correspondence. This is a non-limiting example intended to determine the depth value of a specific pixel in the depth map based on texture information from each input image corresponding to the location of a specific pixel in the depth map in the input image.

[0052] Depth values ​​can be selected based on a set of texture (or other high spatial frequency image content) values ​​in several ways. For example, the highest texture value can be determined (or the lowest texture value can be determined if lower texture values ​​correspond to a larger amount of high spatial frequency image content), and the depth can be determined based on the depth of the input image corresponding to the highest texture value. This can be based on the assumption that for a particular pixel location, the highest texture value may correspond to the most focused image at that particular location, and therefore the components of the sample may lie at the corresponding depth within the sample.

[0053] Other or alternative methods can be applied to determine depth values ​​based on a set of texture values. For example, depth can be determined based on the depth of peaks or other features detected in the set of texture values. Figure 3C A schematic diagram is shown of a set 320 of texture values ​​320 determined for a specific pixel in the output depth map, as a function of the depth of the input image used to generate the texture values. Peaks 325 exist in the texture data; these peaks can be detected, and the depth can be determined thereby. Figure 3C Peak detection can be used to determine depth values ​​based on the assumption that, for a given pixel location, a peak within the texture value as a function of depth may correspond to a sample component focused at that specific location, and therefore the sample component may be located at the corresponding depth within the sample.

[0054] The depth values ​​determined as described above can be used directly as pixels in a depth map to generate pixels in a projected image, improve segmentation of one or more sample images, or facilitate certain other applications. Alternatively, the determined depth values ​​can be spatially preprocessed to some extent before such applications (e.g., the set of depth values ​​determined as described above can be spatially preprocessed to generate a depth map). For example, a two-dimensional low-pass filter or other type of linear filter can be applied to the depth values ​​before they are applied to the application. Additionally or alternatively, nonlinear preprocessing methods can also be applied. For example, an edge-preserving low-pass spatial filter can be applied. As another example, outlier depth values ​​adjacent to them (e.g., those exceeding the mean of their neighbors by a certain amount, such as a multiple of the standard deviation of their neighbors) can be removed, filtered with more stringent filter parameters, or preprocessed in some other way to reduce the impact of such outliers on subsequent processes.

[0055] III. Image Formation Examples

[0056] Depth information of the samples (e.g., depth maps, individual depth values) can be used to project pixels or other image information from an image set of the samples onto a single projected image of the samples (e.g., as in example projected images 250 and 301). When setting the intensity, color, or other image information for each pixel of the projected image, the depth information can be used to determine which image(s) in the image set are extracted from the image set. As described above, when the input image set varies with the focal plane in the samples, the depth information can be used to select the projected image when generating the pixels of the projected image to make the projected image fully focused or improved relative to the input image set.

[0057] Figure 4 A projected image 400 of a sample and narrow depth-of-field image sets 410a, 410b, 410c, and 410d corresponding to different focal planes at different depths within the sample are shown. The projected image 400 and the images in the image sets have the same resolution and pixel positions for ease of interpretation. Those skilled in the art will recognize that the methods described herein can be adapted to depth maps, projected images, and / or input images of image sets with different resolutions and / or pixel positions.

[0058] Image values ​​(e.g., one or more of the color values ​​of the luminance, chroma, red, green, and / or blue channels) of a specific pixel 405 in the projected image 400 can be determined based on a depth value determined for that specific pixel 405. This depth value can be obtained from a depth map of a sample, or it can be determined on a pixel-by-pixel basis. The depth value can be determined as described above (e.g., by identifying the depth of an image set that has the highest texture value at the location corresponding to each pixel of the projected image), or using other methods, such as phase-contrast images, depth sensors, or other depth detection techniques.

[0059] Determining the image value of a specific pixel 405 may include copying the image value of the corresponding pixel in an image that matches the depth value of the specific pixel 405. For example, in Figure 4 In this context, the depth of the second image 410b corresponds to the depth value determined for a specific pixel 405. Therefore, the image value (e.g., brightness, intensity, etc.) of the corresponding pixel 415b of the second image 410b is copied or projected as the image value of the specific pixel 405.

[0060] This one-to-one projection of the image value is intended as a non-limiting example of determining the image value of a specific pixel in a projected image based on a depth value and one or more narrow depth-of-field images corresponding to that depth value. Additional pixels (e.g., combined in a weighted combination) can be used to generate the image value of the specific pixel 405. In some examples, the image value of the specific pixel 405 and / or its neighboring pixels can be determined, in whole or in part, based on the number of pixels adjacent to the corresponding pixel 415b of the second image 410b. Alternatively or additionally, the image value of the specific pixel 405 can be generated using image information from multiple images corresponding to depth values ​​within a specified range of the depth value of the specific pixel 405. For example, the image value of the specific pixel 405 can be determined based on a combination of the corresponding pixel 415b of the second image 410b and the corresponding pixels 415a, 415c of the first image 410a and the third image 410c, where the first image 410a and the third image 410c correspond to depths within a neighborhood (e.g., a specified range) of the depth value of the specific pixel 405.

[0061] IV. Image Segmentation Examples

[0062] In various situations, it can be beneficial to automatically identify the extent, location, size, identification, and / or other information of organoids, tumor spheres, cells, particles, or other three-dimensional, multicellular discrete components in images of biological samples or other target environments. This process can be referred to as "image segmentation." Image segmentation can be used to automatically perform various analyses of image content, such as determining the number, type, volume / size, spatial distribution, shape, growth rate, or other characteristics of cells, organoids, or tumor spheres in a sample.

[0063] Various methods are provided in this art for segmenting microscopic images of biological samples. These methods may include one or more of thresholding, clustering, edge detection, region growing, artificial neural networks, or other machine learning algorithms, or some other techniques or combinations thereof, to identify presumed distinct neighboring regions in an image. Each region identified using these methods may correspond to a corresponding organoid, tumor sphere, cell, or component or portion thereof. These methods can be applied to one or more narrow depth-of-field images of a sample. Segmentation can be improved by segmenting a projection image of the sample determined as described above, because this projection image represents more components of the focused sample compared to any single narrow depth-of-field image used to generate the projection image.

[0064] Figure 5A Example image 500 of the sample is shown. The image may be generated as described elsewhere herein, or may be a projected image obtained in other ways. Figure 5B As shown, image 500 can be segmented to generate a first segmentation map 510 of a sample. The regions include a first example region 515a, a second example region 515b, and a third example region 515c. Each region can represent different objects, multiple objects, and / or portions of objects (e.g., organoids, tumor spheres, cells) in the sample.

[0065] The accuracy of the identified regions may be limited by the image information available in the input image 500. For example, the boundaries between overlapping or adjacent objects within the input image may be blurry or suboptimal to some extent, preventing the segmentation method from identifying the separated objects as distinct regions in the segmentation map. Figure 5C A depth map 520 of a sample represented by an input image 500 is shown. The depth map 520 can be generated using the methods described herein based on a narrow depth-of-field image set (e.g., based on texture information present in the image set), or it can be generated using some other method (e.g., using a depth sensor). The depth map 520 shows the depth difference between different adjacent regions within regions 525a and 525b of the depth map 520. At least the first region 515a and the second region 515b of the first segmentation map 510 may represent multiple distinguishable objects, and the regions 525a and 525b of the depth map 520 correspond to the first region 515a and the second region 515b, respectively.

[0066] Therefore, depth map 520 or depth information from other sources can be applied to improve image segmentation. This can include further segmenting one or more regions of the first segmented image using depth information in the depth map to generate an improved second segmented image. For example, depth information corresponding to a specific region of the first segmentation map 510 can be analyzed to determine whether it represents multiple potential discrete groups of depth values. If so, the specific region can be further subdivided based on the location of the depth values ​​for each discrete group.

[0067] For example, Figure 5C and 5D An analysis of the first region 515a and the second region 515b of the first segmentation map 510 is shown to determine whether these regions should be further segmented when generating the improved second segmentation map 530. For the first region 515a, the analysis can be performed on the first region 515a (in... Figure 5C The location and extent of the first image patch 525a (represented as a depth value) are determined by the corresponding depth values ​​(e.g., pixels in the depth map 520) to identify areas that can be further subdivided into the first region 515a.

[0068] This may include performing clustering, region growing, or other analyses on the depth value clusters within the first image patch 525a to identify two or more regions within the first image patch 525a. The first region 515a can then be further subdivided into regions 535a, 535b, and 535c of the second segmentation map 530. The subdivision may include additional subdivisions based on the positions of depth map pixels corresponding to each identified cluster in the depth map 520. This process may include performing filtering, region growing, edge preservation, or other processes to ensure that the determined segmentation produces contiguous and / or relatively smooth regions 535a, 535b, and 535c after the subdivision of the parent region 515a. Similar processing can be used to subdivide the second region 515b of the first segmentation map 510 into corresponding regions 535d and 535e of the second segmentation map 530. Note that such analysis may also result in no segmentation being performed, as the third region 515c of the first segmentation map 510 is preserved as a single region 535f in the second segmentation map 530.

[0069] V. Application Examples

[0070] The systems and methods described herein can be used to facilitate a variety of biological applications in sample imaging. This can include imaging multiple time points over hours, days, weeks, or other time periods using automated imaging systems, or imaging multiple samples located within an incubator (e.g., 96 samples per well in a 96-well plate) to avoid interference from sample removal from the incubator. The imaged samples can contain 3D cultures of human cells or tumor cells, organoids, tumor spheres, or other cells. The cells can be natural or the result of certain experimental procedures (e.g., exposure to carcinogens to produce cancer cells). Furthermore, the cells can be labeled with fluorescent dyes or genetically modified to express fluorescent proteins, other reporter substances, or to provide other biological insights (e.g., assessing the effects of genetic modifications on cells).

[0071] Samples can be prepared and imaged to facilitate drug development and / or toxicological evaluation of drugs or other substances in fields such as immunology, oncology, neuroscience, cell therapy, or other research. For example, imaging samples, where they include and / or allow for the development of organoids, can facilitate research on organ development, the development of disease models, and / or the quantification and / or advancement of regenerative medicine.

[0072] The systems and methods support a variety of imaging modalities. For example, label-free, bright-field images of multispheres of organoids, embedded extracellular matrix, or other three-dimensional samples (e.g., these samples can be processed in the wells of a 96-well test) can be obtained. Fluorescence imaging can be used to image fluorescent reporter molecules, which can label cells as part of fluorescence analysis and / or serve as reporter molecules for cell function or certain other target properties. For example, imaging based on Annexin V green fluorescent dye reagent can be performed to assess the location, rate, or other information of cell death in a sample; imaging with Nuclight Red indicator can be used to assess information on cell proliferation in a sample; or imaging with certain other fluorescent reporter molecules or reporter molecule systems (e.g., multicolor FUCCI analysis) can be used to assess information about cell division, function, recognition, or differentiation. This can be performed, for example, for cell health analysis of samples containing three-dimensional objects such as organoids, multispheres, or other targets.

[0073] If multiple different samples are imaged in a multi-well plate (e.g., a 96-well plate), the samples may vary considerably in terms of cellular composition (e.g., cell type, tumor cell type), the amount or characteristics of added substances (e.g., the dose of added drug, a specific type of drug variant added as part of a drug development trial), the type of gene modification, or certain other different experimental conditions. The automated imaging and image processing techniques described herein can then be applied to assess the efficacy and / or toxicity of the applied substance / treatment, or to identify other experimental data related to the target.

[0074] VI. System Implementation Examples

[0075] The various implementation schemes described herein can be executed using various systems (e.g., programming). These systems may include desktop computers, laptops, tablets, or other single-user workstations. Alternatively or additionally, the implementation schemes described herein may be executed by servers, cloud computing environments, or other multi-user systems.

[0076] These systems can analyze data received from other systems, such as remote data storage on a server, remote cell counters, or other instruments, or data received from certain other sources. Additionally or alternatively, systems configured to perform the embodiments described herein may include and / or be coupled to automated incubators, sample imaging systems, or other instruments capable of generating experimental data for analysis. For example, such instruments may include incubators containing porous sample containers. Samples in such porous sample containers may differ in terms of sample genome, sample origin, growth medium applied to the sample, drugs or organisms applied to the sample, or certain other conditions applied to the sample.

[0077] Samples within such devices can be experimentally evaluated in a variety of ways. Samples can be imaged (e.g., using visible light, infrared, and / or ultraviolet light). Such imaging can include fluorescence imaging of sample components, such as imaging with fluorescent dyes or reporter molecules added to and / or generated by sample cells (e.g., after insertion of a fluorophore-encoding gene). Automated racks are located within the incubator to facilitate imaging of various samples within the wells of sample containers, or to facilitate measurement and analysis of various samples within the wells of sample containers.

[0078] For example, an automated imaging system can be used to automatically acquire images (e.g., bright-field images, fluorescence images) of multiple biological samples in various wells of a sample container over multiple time periods at different scan cycles. The automated imaging system can take a set of images of each sample in each scan cycle, for example, a set of images with a different focal plane than that within the sample. The images can then be analyzed according to methods described herein, for example, to determine depth maps, projection images, or certain information about the sample.

[0079] Compared to manual imaging, using this automated imaging system can significantly reduce the personnel costs of imaging biological samples and improve the consistency of image generation time, location, and image parameters. Furthermore, this automated imaging system can be configured to operate within the incubator, eliminating the need to remove samples from the incubator for imaging. Therefore, the growth environment of the samples can be maintained more consistently. Additionally, if the automated imaging system is used in situations where the microscope or other imaging device associated with the sample container is moving (rather than, for example, when the moving sample container is imaged by a static imaging device), it can reduce interference related to sample movement. This can improve sample growth and development and reduce movement-related confounding factors.

[0080] This automated imaging system can acquire one or more images during a scan, with intervals between these images exceeding 24 hours, 3 days, 30 days, or longer. Scans can be specified to be performed at a particular rate, such as once a day, multiple times a day, more than twice a day, or more than three times a day. Scans can be specified to be performed at least two, at least three, or more times within a 24-hour period. In some examples, data from one or more scans (e.g., according to the methods described herein) can be analyzed to determine the timing of additional scans (e.g., by increasing the scan rate, duration, image capture rate, or certain other properties to detect the occurrence of discrete events expected to occur in the sample).

[0081] Using such automated imaging systems facilitates imaging of the same biological sample at multiple time points over a longer period. Therefore, the development and / or behavior of individual cells and / or cell networks (e.g., organoids, tumor spheres) can be analyzed over time. For example, cell sets, portions of cells, or other objects can be identified within a single sample, in scans performed at different, long intervals. These identified sets of objects can then be compared between scans to identify the same objects across scans. Thus, the behavior of individual organoids, tumor spheres, cells, or portions of cells can be tracked and analyzed over hours, days, weeks, or months.

[0082] Figure 6The components of the automated imaging system 600 are shown. The automated imaging system 600 includes a frame 610 to which other components of the automated imaging system 600 are attached. The frame 610 can be configured (e.g., sized) for mounting within an incubator. The automated imaging system 600 includes a sample container 620, which is detachably placed within a sample container tray 630 connected to the frame 610. The sample container tray 630 may be removable and / or may include removable inserts to accommodate a variety of different sample containers (e.g., various industry-standard sample containers). The system 600 also includes a drive frame 650 configured relative to a positioning imaging device 640 of the sample container 620, such that the imaging device 640 can be moved to acquire images of the components in the individual holes of the sample container 620 (e.g., example hole 625).

[0083] Imaging apparatus 640 may include a microscope, a fluorescence imager, a two-photon imaging system, a phase-contrast imaging system, one or more light sources, one or more optical filters, and / or other elements configured to facilitate imaging of a sample within sample container 620. In some examples, imaging apparatus 640 includes elements arranged on both sides of sample container 620 (e.g., coherent, polarized, monochromatic, or other designated illumination sources to facilitate, for example, phase-contrast imaging of biological samples). In such examples, the elements on both sides of sample container 620 may be coupled to different racks, may be coupled to different grids on the same rack, and / or the elements on one side of sample container 620 may not be movable relative to sample container 620.

[0084] The actuator frame 650 is coupled to the frame 610 and the imaging device 640 and is configured to control the position of the device 640 in at least two directions relative to the sample container 620 to facilitate imaging of multiple different samples within the sample container 620. The actuator frame 650 may also be configured to control the position of the imaging device 640 toward and away from the sample container 620 in a third-party upward direction to control the focal length of the images obtained using the imaging device 640 and / or control the depth of material within the sample container 620 that can be imaged by the imaging device 640. Alternatively or additionally, the imaging device 640 may include one or more actuators for controlling the focal length of the imaging device 640. The imaging device 640 may include one or more motors, piezoelectric elements, liquid crystal lenses, or other actuators that facilitate control of the focal length setting of the imaging device 640. For example, the imaging device 640 may include a actuator configured to control the focal length between the imaging device 640 and the sample being imaged. This is done to ensure that the image is focused and / or allows the image to be captured such that various different focal planes in the sample are represented in their respective different images.

[0085] The actuator rack 650 may include elements configured to facilitate the detection of the absolute and / or relative position of the imaging device 640 with respect to the sample container 620 (e.g., with respect to a specific opening in the sample container 620). For example, the actuator rack 650 may include encoders, limit switches, and / or other position sensing elements. Additionally or alternatively, the imaging device 640 or other elements of the system may be configured to detect reference marks or other features on the sample container 620 and / or sample container tray 630 to determine the absolute and / or relative position of the imaging device 640 with respect to the sample container 620.

[0086] The computational functions (e.g., the function of operating the drive rack 650 and / or imaging device 640 to image samples within sample container 620 and / or the function of performing certain other methods described herein) can be performed by one or more computing systems. This computing system can be integrated into a laboratory instrumentation system (e.g., 600), can be associated with that system (e.g., via a direct wired or wireless connection, a local network, and / or a secure connection over the Internet), and / or can take some other form (e.g., communicating with an automated imaging system and / or a cloud computing system with access to biological sample image storage).

[0087] Figure 7 An example of such a computing system 700 that can be used to implement the methods described herein is shown. The example computing system 700 includes a communication interface 702, a user interface 704, a processor 706, one or more sensors 707 (e.g., photodetectors, cameras, depth sensors, microscopes, or some other instrumented laboratory equipment), and a data storage 708, all of which are communicatively connected together via a system bus 710.

[0088] The communication interface 702 may have the capability to allow the computing system 700 to communicate with other devices, access networks, and / or transmission networks using analog or digital modulation of electrical, magnetic, electromagnetic, optical, or other signals. Therefore, the communication interface can facilitate circuit-switched and / or packet-switched communications, such as Common Old Telephone Service (POTS) communications and / or Internet Protocol (IP) or other packet communications. For example, the communication interface 702 may include a chipset and antenna for wireless communication with a wireless access network or access point. Additionally, the communication interface 702 may take the form of or include a wireless interface, such as an Ethernet, Universal Serial Bus (USB), or High Definition Multimedia Interface (HDMI) port. The communication interface may also take the form of or include a wireless interface, such as WiFi, Global Positioning System (GPS) or wide-area wireless interface (such as WiMAX or 3GPP Long Term Evolution (LTE)). However, other forms of physical layer interfaces and other types of standard or proprietary communication protocols can be used on communication interface 702. Furthermore, communication interface 702 may include multiple physical communication interfaces (e.g., WiFi interface, etc.). Interface and wide area wireless interface).

[0089] In some implementations, the communication interface 702 may have the capability to allow the computing system 700 to communicate with other devices, remote servers, access networks, and / or transmission networks. For example, the communication interface 702 may be used to transmit and / or receive indications of biological sample images (e.g., a bright-field image set or other type of image set that differs from an image formed on the focal plane within the sample) or certain other information.

[0090] The user interface 704 of the computing system 700 enables the computing system 700 to interact with a user, such as receiving input from the user and / or providing output to the user. Therefore, the user interface 704 may include input components such as a keypad, keyboard, touch-sensitive control panel or presence-sensitive control panel, computer mouse, trackball, joystick, microphone, etc. The user interface 704 may also include one or more output components, such as a display screen that can be combined with a presence-sensitive control panel. The display screen may be based on CRT, LCD, and / or LED technology, or other known or recently developed technologies. The user interface 704 may also be configured to generate audio output via speakers, speaker jacks, audio output ports, audio output devices, headphones, and / or other similar devices.

[0091] In some implementations, the user interface 704 may include a display for presenting a user with video or other images (e.g., a video of images generated during a specific scan of a particular biological sample). Additionally, the user interface 704 may include one or more buttons, switches, knobs, and / or dials to facilitate the configuration and operation of the computing device. Some or all of these buttons, switches, knobs, and / or dials may be implemented as functions on a touch-sensitive control panel or a display-sensitive control panel. The user interface 704 may allow the user to specify the type of sample included in the automated imaging system, specify a schedule for imaging or other evaluation of the sample, specify parameters for image segmentation, event analysis, and / or certain other analyses that the system 700 will perform, or input certain other commands or parameters for the operation of the automated laboratory system and / or the analysis of the data generated therefrom.

[0092] Processor 706 may include one or more general-purpose processors (e.g., microprocessors) and / or one or more special-purpose processors (e.g., digital signal processors (DSPs), graphics processing units (GPUs), floating-point units (FPUs), network processors, tensor processing units (TPUs), or application-specific integrated circuits (ASICs). In some cases, special-purpose processors are capable of applications or functions such as image processing, image alignment, statistical analysis, filtering, or noise reduction. Data memory 708 may include one or more volatile and / or non-volatile memory components, such as magnetic memory, optical memory, flash memory, or organic memory, and may be integrated wholly or partially with processor 706. Data memory 708 may include removable and / or non-removable components.

[0093] Processor 706 is capable of executing program instructions 718 (e.g., compiled or uncompiled program logic and / or machine code) stored in data memory 708 to perform the various functions described herein. Therefore, data memory 708 may include a non-transitory computer-readable medium storing program instructions that, when executed by computing device 700, cause computing device 700 to perform any methods, procedures, or functions disclosed in this specification and / or the accompanying drawings. Processor 706 executing program instructions 718 may cause processor 706 to use data 712.

[0094] For example, program instructions 718 may include an operating system 722 (e.g., an operating system kernel, device drivers, and / or other modules) and one or more application programs 720 (e.g., filtering functions, data processing functions, statistical analysis functions, image processing functions, depth measurement functions, image segmentation functions) installed on computing device 700. Data 712 may include microscopic images or other data, including image sets of individual samples, depth information of the samples, and / or segmentation information of the samples.

[0095] Application 720 can communicate with operating system 722 through one or more application programming interfaces (APIs). These APIs can facilitate, for example, application 720 receiving information via communication interface 702, receiving and / or displaying information from user interface 704, etc.

[0096] Application 720 may take the form of "software (app)," which can be downloaded to computing device 700 through one or more online application stores or app markets (e.g., via communication interface 702). However, the application may also be installed on computing device 700 in other ways, such as through a web browser or through a physical interface of computing device 700 (e.g., a USB port).

[0097] In some examples, depending on the application, portions of the methods described herein may be performed by different devices. For example, different devices in the system may have different amounts of computing resources (e.g., memory, processor cycles) and different information bandwidths for communication between devices. For example, the first device may be an embedded processor that can operate a driver rack, imaging device, or other components to generate information about biological samples at multiple different times and / or within multiple different time periods. A second device can then receive information (e.g., image information, depth information) from the first device (e.g., via the Internet, via a dedicated wired link) and perform the procedures and analytical methods described herein on the received data. Different portions of the methods described herein can be deduced similarly.

[0098] VII. Method Examples

[0099] Figure 8 This is a flowchart of a method 800 for generating a projected image of a 3D sample. Method 800 includes acquiring an image set of the sample, wherein each image in the image set corresponds to a respective focal plane within the sample (810). Method 800 also includes applying a filter to each image in the image set to determine a respective depth value for each pixel of an output image of the sample, wherein a given depth value represents the depth within the sample where the sample component can be focused and imaged (820). Method 800 further includes determining an image value for each pixel of the output image based on the depth values ​​of the pixels in the output image, wherein determining the image value of a particular pixel in the output image includes: (i) identifying an image in the image set corresponding to the depth value of the particular pixel; and (ii) determining the image value of the particular pixel based on the pixels of the identified image, the pixels of the identified image having a position corresponding to the particular pixel in the identified image (830). Method 800 may include additional elements or features.

[0100] Figure 9 This is a flowchart of a method 900 for generating a projected image of a 3D sample. Method 900 includes acquiring an image set of the sample, wherein each image in the image set corresponds to a respective focal plane within the sample (910). Method 900 also includes applying a filter to each image in the image set to determine a respective depth value for each pixel of a depth map, wherein a given depth value represents the depth within the sample where the sample component can be focused and imaged (920). Method 900 further includes determining an image value for each pixel of an output image based on the depth values ​​of the corresponding pixels in the depth map (930). Method 900 may include additional elements or features.

[0101] Figure 10This is a flowchart of a method 1000 for segmenting a sample image. Method 1000 includes acquiring an image of the sample (1010). Method 1000 also includes acquiring a depth map of the sample components (1020). Method 1000 further includes generating a first segmentation map of the sample based on the image (1030). Method 1000 further includes generating a second segmentation map of the sample based on the depth map by further subdividing at least one region of the first segmentation map (1040). Method 1000 may include additional elements or features.

[0102] VIII. Conclusion

[0103] The detailed description above, with reference to the accompanying drawings, illustrates various features and functions of the disclosed systems, devices, and methods. In the drawings, similar symbols generally identify similar components unless the context otherwise indicates. The illustrative embodiments described in the detailed description, drawings, and claims are not intended to be limiting. Other embodiments may be used and other changes may be made without departing from the scope of the subject matter described herein. It should be understood that, as described herein and shown in the drawings, various aspects of this disclosure can be arranged, substituted, combined, separated, and designed in a variety of different configurations, all of which are expressly contemplated herein.

[0104] The accompanying diagrams, scenarios, and flowcharts, and each step, block, and / or communication as discussed herein, may represent information processing and / or information transmission according to the example embodiments. Alternative embodiments are included within the scope of these example embodiments. In these alternative embodiments, for example, functions described as steps, blocks, transmissions, communications, requests, responses, and / or messages may not be performed in the order shown or discussed, including substantially simultaneously or in reverse order, depending on the functions involved. Furthermore, more or fewer steps, blocks, and / or functions may be associated with any of the information flow diagrams, scenarios, and flowcharts discussed herein. Figure 1 They can be used together, and these information flow diagrams, scenarios, and flowcharts can be combined with another information flow diagram, scenario, and flowchart, either partially or entirely.

[0105] A step or block representing information processing may correspond to a circuit that can be configured to perform a specific logical function of the method or technique described herein. Optionally or additionally, a step or block representing information processing may correspond to a module, segment, or portion of program code (including associated data). The program code may include one or more instructions executed by a processor to implement a specific logical function or operation in the method or technique. The program code and / or associated data may be stored on any type of computer-readable medium, such as a storage device, including a disk drive, hard disk drive, or other storage medium.

[0106] Computer-readable media can also include non-transitory computer-readable media, such as computer-readable media that store data for short periods, such as register memory, processor cache, and / or random access memory (RAM). Computer-readable media can also include non-transitory computer-readable media that can store program code and / or data for longer periods, such as secondary or persistent long-term storage, such as read-only memory (ROM), optical discs or magnetic disks, and / or read-only optical discs (CD-ROMs). Computer-readable media can also be any other volatile or non-volatile storage system. For example, a computer-readable medium can be considered a computer-readable storage medium or a tangible storage device.

[0107] Furthermore, a step or block representing one or more information transfers may correspond to information transfers between software and / or hardware modules within the same physical device. However, other information transfers may occur between software and / or hardware modules in different physical devices.

[0108] While several aspects and embodiments are disclosed herein, other aspects and embodiments will be apparent to those skilled in the art. The various aspects and embodiments disclosed herein are intended to be illustrative and not restrictive, and their true scope is as set forth in the claims.

Claims

1. A method for generating a projected image of a three-dimensional sample, the method comprising: Obtain a sample image set, where each image in the image set corresponds to its respective focal plane within the sample; The filter is applied to each image in the image set to determine the corresponding depth value for each pixel in the output image of the sample, where the given depth value represents the depth within the sample that can enable the sample components to be focused and imaged; and The image value of each pixel in the output image is determined based on the depth value of the pixels in the output image, wherein determining the image value of a specific pixel in the output image includes: (i) identifying an image in an image set corresponding to the depth value of the specific pixel; and (ii) determining the image value of the specific pixel based on the pixels of the identified image, wherein the pixels of the identified image have a position corresponding to the specific pixel within the identified image. The method further includes: Generate a first segmentation map of the samples based on the output image; and Based on the determined depth value, a second segmentation map of the sample is generated by further dividing at least one segment of the first segmentation map, wherein further dividing at least one segment of the first segmentation map includes: Select a set of depth values ​​corresponding to a specific region of the first segmentation map from the determined depth values; Identify at least two clusters within the selected depth value set; and The specific region is further divided based on at least two clusters identified.

2. The method of claim 1, wherein the filter is a texture filter, and applying the filter to a specific image in the image set includes, for a specific pixel of the specific image, determining at least one of the standard deviation, entropy, or numerical range of the pixel in the neighborhood of the specific image.

3. The method according to claim 1 further includes generating a depth map based on the depth value of the sample.

4. The method according to claim 1, further comprising: The depth values ​​determined by spatial preprocessing, wherein the image value of each pixel of the output image is determined based on the depth values ​​of the pixels of the output image, including determining the image value of each pixel of the output image based on the spatially preprocessed depth values ​​of the pixels of the output image.

5. The method according to claim 1, wherein the image set is a bright-field image set of the sample.

6. The method according to claim 1, wherein the image set is a fluorescence image set of the sample.

7. The method of claim 1, wherein the sample comprises at least one three-dimensional cultured multicellular structure.

8. The method of claim 7, wherein the at least one three-dimensional cultured multicellular structure comprises at least one organoid embedded in the extracellular matrix or tumor sphere.

9. The method of claim 8, wherein the at least one three-dimensional cultured multicellular structure comprises an organoid embedded in an extracellular matrix, wherein the extracellular matrix is ​​a dome-shaped extracellular matrix.

10. The method of claim 8, wherein the at least one three-dimensional cultured multicellular structure comprises at least one of hepatocyte organoids, pancreatic organoids, or intestinal organoids.

11. The method according to any preceding claim, wherein determining the image value of a specific pixel of the output image further comprises: (iii) Identifying two or more additional images of an image set corresponding to a depth value in the neighborhood of a specific pixel, wherein determining the image value of the specific pixel is based on the pixels of the identified images, the pixels of the identified images having a position corresponding to the specific pixel within the identified images, including performing pixel-level operations on the pixels of the identified images and the two or more additional images, the additional images having a position corresponding to the specific pixel in their respective identified images.

12. A method for generating a three-dimensional sample projection image, the method comprising: Acquire an image set of the sample, wherein each image in the image set corresponds to a respective focal plane within the sample; The filter is applied to each image in the image set to determine the corresponding depth value for each pixel of the depth map, wherein the depth value represents the intra-sample depth that can enable the sample components to be focused and imaged; and wherein applying the filter to each image in the image set includes, for a specific pixel of the image, determining a weighted texture of a set of pixels in the neighborhood of the specific pixel to determine which pixels in the image set are in focus, thereby generating a projected image of the three-dimensional sample; and Based on the depth value of the corresponding pixel in the depth map, the image value of each pixel in the output image is determined.

13. A method for generating a projected image of a three-dimensional sample, the method comprising: Obtain a sample image set, where each image in the image set corresponds to its respective focal plane within the sample; A filter is applied to each image in the image set to determine the corresponding depth value for each pixel of the depth map, where the depth value represents the depth within the sample that allows the sample components to be focused and imaged; and The image value of each pixel in the output image is determined based on the depth value corresponding to the depth map. Select a set of depth values ​​corresponding to a specific region in the first segmentation map from the given depth values; Identify at least two clusters within the selected depth value set; and The specific region is further divided based on at least two clusters identified.

14. The method of claim 12 or 13, wherein the filter is a texture filter, and when the filter is applied to a specific image of the image set, for a specific pixel of the specific image, at least one of the standard deviation, entropy, or numerical range of pixels of the specific image located in the neighborhood of the specific pixel is determined.

15. The method according to claim 12 or 13, further comprising: Generate the first segmentation map of the samples based on the output image; and Based on a determined depth value, a second segmentation map of the sample is generated by further dividing at least one region of the first segmentation map.

16. The method of claim 15, wherein further dividing at least one region of the first segmentation map includes: Select a set of depth values ​​corresponding to a specific region in the first segmentation map from the given depth values; Identify at least two clusters within the selected depth value set; and The specific region is further divided based on at least two identified clusters.

17. The method according to claim 12 or 13, wherein the image set is a bright-field image set of the sample.

18. The method of claim 12 or 13, wherein the image set is a set of fluorescence images of the sample.

19. The method according to claim 12 or 13, wherein the sample contains at least one organoid.

20. A method for segmenting sample images, the method comprising: Obtain the image of the sample; Obtain a depth map of the sample components; The first segmentation map is generated based on the image samples; and Based on the depth map, a second segmentation map of the sample is generated by further dividing at least one region of the first segmentation map; The depth map contains multiple depth values, and the further subdivision of at least one region of the first segmentation map includes: Select a set of depth values ​​corresponding to a specific region in the first segmentation map from the depth values; Identify at least two clusters within the selected depth value set; and The specific region is further divided based on at least two identified clusters.

21. The method of claim 20, wherein the sample contains at least one organoid.

22. A non-transitory computer-readable medium, wherein, The non-transitory computer-readable medium stores at least one processor-executable instruction to perform the following operations: Obtain an image set of the sample, where each image in the image set corresponds to its respective focal plane within the sample; The filter is applied to each image in the image set to determine the corresponding depth value for each pixel of the output image of the sample, where the given depth value represents the depth within the sample that allows the sample components to be focused and imaged; and Determining the image value of each pixel of the output image based on the depth value of the pixels of the output image, wherein determining the image value of a specific pixel of the output image includes: (i) identifying an image from an image set corresponding to the depth value of the specific pixel; and (ii) determining the image value of the specific pixel based on the pixels of the identified image, wherein the pixels of the identified image have a position corresponding to the specific pixel within the identified image; The operation also includes: A first segmentation map of the samples is generated based on the output image; and Based on the determined depth value, a second segmentation map of the sample is generated by further dividing at least one segment of the first segmentation map; Further dividing the first segmented image into at least one segment includes: Select a set of depth values ​​corresponding to a specific region of the first segmentation map from the determined depth values; Identify at least two clusters within the selected depth value set; and Based on at least two identified clusters, specific regions are further divided.

23. The non-transitory computer-readable medium of claim 22, wherein the filter is a texture filter, and applying the filter to a specific image of the image set includes, for a specific pixel of the specific image, determining at least one of the standard deviation, entropy, or numerical range of the pixel in the neighborhood of the specific image.

24. The non-transitory computer-readable medium of claim 22, wherein the operation further comprises generating a depth map of the sample based on the depth value.

25. The non-transitory computer-readable medium of claim 22, wherein the operation further comprises: The spatial preprocessing determines the depth value, wherein the image value of each pixel of the output image is determined based on the depth value of the pixels of the output image, including determining the image value of each pixel of the output image based on the spatial preprocessing depth value of the pixels of the output image.

26. The non-transitory computer-readable medium according to any one of claims 22-25, wherein, Determining the image value of a specific pixel in the output image further includes: (iii) identifying two or more additional images in an image set corresponding to depth values ​​in the neighborhood of the depth value of the specific pixel, and determining the image value of the specific pixel based on the pixels of the identified images, wherein the pixels of the identified images have positions within the identified images corresponding to the specific pixel, including performing pixel-level operations on the pixels of the identified images and the pixels of the two or more additional images, wherein the two or more additional images include positions within their respective identified images corresponding to the specific pixel.

27. A non-transitory computer-readable medium, wherein, The non-transitory computer-readable medium stores at least one processor-executable instruction to perform the following operations: Obtain an image set of the sample, where each image in the image set corresponds to its respective focal plane within the sample; A filter is applied to each image in the image set to determine the corresponding depth value for each pixel in the depth map, where a given depth value represents the depth within the sample that can enable the sample components to be focused and imaged. and Based on the depth value of the corresponding pixel in the depth map, determine the image value of each pixel in the output image; The operation also includes: A first segmentation map of the samples is generated based on the output image; and Based on the determined depth value, a second segmentation map of the sample is generated by further dividing at least one region of the first segmentation map; wherein further dividing at least one region of the first segmentation map includes: Select a set of depth values ​​corresponding to a specific region of the first segmentation map from the determined depth values; Identify at least two clusters within the selected depth value set; and Based on at least two identified clusters, specific regions are further divided.

28. The non-transitory computer-readable medium of claim 27, wherein the filter is a texture filter, and applying the filter to a specific image of the image set includes, for a specific pixel of the specific image, determining at least one of the standard deviation, entropy, or numerical range of pixels of the specific image located in the neighborhood of the specific pixel.

29. A non-transitory computer-readable medium, wherein, The non-transitory computer-readable medium stores at least one processor-executable instruction to perform the following operations: Obtain the image of the sample; Obtain a depth map of the sample components; The first segmentation map is generated based on the image samples; and Based on the depth map, a second segmentation map of the sample is generated by further dividing at least one region of the first segmentation map; wherein the depth map contains multiple depth values, and the further division of at least one region of the first segmentation map includes: Select a set of depth values ​​corresponding to a specific region in the first segmentation map from the depth values; Identify at least two clusters within the selected depth value set; and The specific region is further divided based on at least two identified clusters.

30. A system for generating projected images of three-dimensional samples, comprising: One or more processors; and A non-transitory computer-readable medium having instructions stored thereon that can be executed by at least one of one or more processors to perform the following operations: Acquire an image set of the sample, wherein each image in the image set corresponds to a respective focal plane within the sample; The filter is applied to each image in the image set to determine the corresponding depth value for each pixel in the output image of the sample, where the given depth value represents the depth within the sample that can enable the sample components to be focused and imaged; and Determining the image value of each pixel of the output image based on the depth value of the pixels of the output image, wherein determining the image value of a specific pixel of the output image includes: (i) identifying an image from an image set corresponding to the depth value of the specific pixel; and (ii) determining the image value of the specific pixel based on the pixels of the identified image, wherein the pixels of the identified image have a position within the identified image corresponding to the specific pixel; The operation also includes: A first segmentation map of the samples is generated based on the output image; and Based on the determined depth value, a second segmentation map of the sample is generated by further dividing at least one segment of the first segmentation map; The further division of at least one segment of the first segmentation map includes: Select a set of depth values ​​corresponding to a specific region of the first segmentation map from the determined depth values; Identify at least two clusters within the selected depth value set; and The specific region is further divided based on at least two identified clusters.

31. The system of claim 30, wherein the filter is a texture filter, and applying the filter to a specific image of the image set includes, for a specific pixel of the specific image, determining at least one of the standard deviation, entropy, or numerical range of pixels of the specific image located in the neighborhood of the specific pixel.

32. The system of claim 30, wherein the operation further comprises generating a depth map of the sample based on the depth value.

33. The system of claim 30, wherein the operation further comprises: The spatial preprocessing determines the depth value, wherein the image value of each pixel of the output image is determined based on the depth value of the pixels of the output image, including determining the image value of each pixel of the output image based on the spatial preprocessing depth value of the pixels of the output image.

34. The system according to any one of claims 30-33, wherein determining the image value of a specific pixel of the output image further comprises: (iii) Identifying two or more additional images in an image set corresponding to a depth value in the neighborhood of a specific pixel, wherein determining the image value of the specific pixel based on the pixels of the identified image, the pixels of the identified image having a position corresponding to the specific pixel within the identified image, including performing pixel-level operations on the pixels of the identified image and the pixels of two or more additional images, the two or more additional images including the position corresponding to the specific pixel within their respective identified images.

35. A system for generating projected images of three-dimensional samples, comprising: One or more processors; and A non-transitory computer-readable medium having instructions stored thereon that can be executed by at least one of one or more processors to perform the following operations: Acquire an image set of the sample, wherein each image in the image set corresponds to a respective focal plane within the sample; A filter is applied to each image in the image set to determine the corresponding depth value for each pixel in the depth map, where a given depth value represents the depth within the sample that can enable the sample components to be focused and imaged. and Based on the depth value of the corresponding pixel in the depth map, determine the image value of each pixel in the output image; Select a set of depth values ​​corresponding to a specific region of the first segmentation map from the determined depth values; Identify at least two clusters within the selected depth value set; and The specific region is further divided based on at least two identified clusters.

36. The system of claim 35, wherein the filter is a texture filter, and applying the filter to a specific image of the image set includes, for a specific pixel of the specific image, determining at least one of the standard deviation, entropy, or numerical range of pixels of the specific image located in the neighborhood of the specific pixel.

37. The system according to any one of claims 35-36, wherein the operation further comprises: A first segmentation map of the samples is generated based on the output image; and Based on the determined depth value, a second segmentation map of the sample is generated by further dividing at least one region of the first segmentation map.

38. A system for segmenting sample images, comprising: One or more processors; and A non-transitory computer-readable medium having instructions stored thereon that can be executed by at least one of one or more processors to perform the following operations: Obtain the image of the sample; Obtain a depth map of the sample components; The first segmentation map is generated based on the image samples; and Based on the depth map, a second segmentation map of the sample is generated by further dividing at least one region of the first segmentation map; The depth map contains multiple depth values, and the further subdivision of at least one region of the first segmentation map includes: Select a set of depth values ​​corresponding to a specific region in the first segmentation map from the depth values; Identify at least two clusters within the selected depth value set; and The specific region is further divided based on at least two identified clusters.

Images