Segmentation-based image processing for confluency estimation

Abstract

To provide image evaluation methods for performing accurate coverage evaluation of high-coverage images, such as confluence images of cell culture.SOLUTION: A method of determining a coverage of an image comprises: generating a first segmentation mask by segmenting an image; generating a modified mask by applying a morphological operation to the first segmentation mask; generating a modified masked input based on the image and an inversion of the modified mask; generating a second segmentation mask by segmenting the modified masked input; and determining a coverage of the image based on the first segmentation mask and the second segmentation mask.SELECTED DRAWING: Figure 16

Description

[Technical field]

[0001] CROSS-REFERENCE TO RELATED APPLICATIONS This application is a PCT international application claiming the benefit of and priority to U.S. Provisional Patent Application No. 63 / 211,964, filed June 17, 2021. The entire disclosure of the above application is incorporated herein by reference.

[0002] The present disclosure relates to the field of image analysis, and more particularly to determining image coverage, such as cell confluence in images of cell cultures. [Background technology]

[0003] The background discussion provided herein is intended to generally present the context for the present disclosure. The work of the presently named inventors to the extent described in this Background section and aspects of the present specification that may not be admitted as prior art at the time of filing are not admitted expressly or impliedly as prior art to the present disclosure.

[0004] In the field of imaging, there are many situations that involve the analysis of an image to determine the coverage of the image by a feature. As an example, an image of a cell culture may include a large number of cells that may be of various types, shapes and sizes. It may be desirable for an individual, such as a researcher, to view an image of the cell culture and determine the confluence, i.e., the coverage of the cells in the field of the image. For example, as the cell culture matures, the cells may grow and / or proliferate, thereby increasing the confluence of the image. The individual may wish to determine the confluence at a particular time point, for example, to determine the developmental stage of the cell culture. The individual may use such a determination to study the cell culture, for example, by treating, measuring and / or staining the cell culture upon entering a particular developmental stage, which may be indicated by the degree of confluence.

[0005] However, some characteristics of such estimation techniques may adversely affect or limit the confluence determination by an individual. As a first example, relying on the coverage determination (such as confluence estimation and / or cell counting) by an individual may be time-consuming, for example, when the amount of images to be evaluated is large and / or the number of cells is large. As a result, the coverage determination may be inefficient, which may limit the number of images that can be evaluated in a timely manner for a cell culture population, and / or the presentation of the coverage determination may be delayed, such as when the cells in the cell culture have matured to an advanced developmental stage by the time images of the previous developmental stage are evaluated.

[0006] As a second example, relying on an individual to determine coverage may be inaccurate, for example, if the coverage or confluence estimate is qualitatively measured as "dense," "medium," or "sparse" (where different individuals may have different thresholds between such categories and / or individual thresholds may change over time). As a third example, relying on an individual to determine coverage may be difficult due to the presence of other visible matter in the image, such as debris, e.g., non-cellular tissue debris, contaminants, e.g., dust, and / or image, imaging process, or display artifacts. In such cases, an individual may have difficulty distinguishing between cellular and non-cellular features of the image, which may lead to an overestimation or underestimation of the coverage of the image.

[0007] In some cases, it may be possible to use automated image analysis techniques to assist individuals in assessing image coverage. However, many of these techniques may perform relatively poorly on images with high coverage, such as high confluence slides with densely packed cells. In such cases, many image analysis techniques may not recognize at least some of the cells, resulting in underestimation of coverage, such as underestimation of cell culture confluence. As a result of such underestimation of confluence, further research processes based on cell culture reaching a certain developmental stage indicated by confluence may be delayed until a later time than the actual certain developmental stage, which may lead to unexpected results and / or compromise the integrity of the resulting cell culture or data. Summary of the Invention [Problem to be solved by the invention]

[0008] It may be desirable to provide image evaluation techniques that provide accurate coverage evaluation of images, which may improve the achievable speed and / or amount of image evaluation and / or other processes based thereon, and in particular, it may be desirable to provide image evaluation techniques that provide accurate coverage evaluation of high coverage images, such as high confluence images of cell cultures. [Means for solving the problem]

[0009] In some demonstrative embodiments, the apparatus includes a processing circuit configured to generate a first segmentation mask by segmenting an image, generate a modified mask by applying a morphological operation to the first segmentation mask, generate a modified mask input based on an inverse of the image and the modified mask, generate a second segmentation mask by segmenting the modified mask input, and determine coverage of the image based on the first segmentation mask and the second segmentation mask.

[0010] In some demonstrative embodiments, the system includes an image evaluator configured to generate a first segmentation mask by segmenting the image, generate a modified mask by applying a morphological operation to the first segmentation mask, generate a modified mask input based on an inverse of the image and the modified mask, generate a second segmentation mask by segmenting the modified mask input, and determine a coverage of the image based on the first segmentation mask and the second segmentation mask.

[0011] In some demonstrative embodiments, a non-transitory computer-readable storage medium storing instructions that, when executed by a processing circuit, cause the apparatus to generate a first segmentation mask by segmenting an image, generate a modified mask by applying a morphological operation to the first segmentation mask, generate a modified mask input based on an inverse of the image and the modified mask, generate a second segmentation mask by segmenting the modified mask input, and determine coverage of the image based on the first segmentation mask and the second segmentation mask.

[0012] In some demonstrative embodiments, a method of operating an apparatus includes executing, by a processing circuit, instructions that cause the apparatus to generate a first segmentation mask by segmenting an image, generate a modified mask by applying a morphological operation to the first segmentation mask, generate a modified mask input based on an inverse of the image and the modified mask, generate a second segmentation mask by segmenting the modified mask input, and determine coverage of the image based on the first segmentation mask and the second segmentation mask.

[0013] Some exemplary embodiments include adjusting the image characteristics to generate the adjusted image, and generating the first segmentation mask includes segmenting the adjusted image. In some exemplary embodiments, adjusting the image characteristics includes normalizing an illumination level of the image. In some exemplary embodiments, normalizing the image illumination level of the image includes applying a Gaussian blur. For example, the Gaussian blur may be applied to a copy of the image and then the blurred copy is subtracted from the image.

[0014] In some exemplary embodiments, adjusting the characteristics of the image includes increasing a local contrast level of the image. In some exemplary embodiments, increasing the local contrast level of the image includes applying contrast-limited adaptive histogram equalization to the image.

[0015] In some exemplary embodiments, generating the first segmentation mask includes segmenting the image based on an edge filter. In some exemplary embodiments, generating the first segmentation mask includes applying a Gaussian blur to the image. In some exemplary embodiments, generating the first segmentation mask includes increasing a contrast level of the image.

[0016] In some exemplary embodiments, the morphological operation includes one or more of an open morphological operation, a close morphological operation, a dilation morphological operation, or an erosion morphological operation.

[0017] In some exemplary embodiments, segmenting the modified mask input includes segmenting the modified mask input based on an edge filter. In some exemplary embodiments, segmenting the modified mask input includes applying a Gaussian blur to the modified mask input. In some exemplary embodiments, segmenting the modified mask input includes increasing a contrast level of the modified mask input.

[0018] Some exemplary embodiments may include combining the first segmentation mask and the second segmentation mask to generate a composite mask. In some exemplary embodiments, generating the composite mask includes applying a non-cellular filter to the composite mask to exclude non-cellular areas of the composite mask. In some exemplary embodiments, excluding non-cellular areas of the composite mask is based on one or more of a morphology of the non-cellular areas or a size of the non-cellular areas.

[0019] In some exemplary embodiments, determining the coverage of the image includes presenting the mask input image as a diagram of the coverage of the image. In some exemplary embodiments, determining the coverage includes estimating the coverage of the composite mask.

[0020] The present disclosure will become more fully understood from the detailed description and the accompanying drawings, in which reference numbers may be reused to identify similar and / or identical elements. [Brief description of the drawings]

[0021] [Figure 1] FIG. 1 shows exemplary cell culture images with various confluences. [Diagram 2] FIG. 13 illustrates an exemplary evaluation of a highly confluent cell culture image according to several image evaluation processes that underestimate confluence in some areas of the cell culture image. [Diagram 3] 1 is an example of an imaging technique for determining coverage of an image, according to some example embodiments. [Figure 4] FIG. 2 is a diagram of a first example of image feature adjustment according to some exemplary embodiments. [Diagram 5] FIG. 11 is a diagram of a second example of image feature adjustment according to some exemplary embodiments. [Figure 6]4A-4C are diagrams of a first portion of an exemplary first segmentation of an adjustment image according to some exemplary embodiments. [Figure 7] 4A is a diagram of a second portion of an exemplary first segmentation of an adjustment image, according to some exemplary embodiments. [Figure 8] 11A-11C are diagrams of an example set of morphological operations that may be applied to a first segmentation mask, according to some example embodiments. [Figure 9] FIG. 11 is a diagram of a second modified mask input generated in accordance with some exemplary embodiments. [Figure 10] 11 is a diagram of a first portion of an example second segmentation of a second modified mask input according to some example embodiments. FIG. [Figure 11] 11 is a diagram of a second portion of an example second segmentation of a second modified mask input according to some example embodiments. FIG. [Figure 12] 1 is a diagram of generating a composite mask, according to some example embodiments. [Figure 13] 13A-13C are diagrams of application of an acellular filter to a set of composite masks, according to some exemplary embodiments. [Figure 14] FIG. 1 is a set of images generated by applying image processing techniques to a cell culture, according to some illustrative embodiments. [Figure 15] 1 is a component block diagram of an example apparatus according to some exemplary embodiments. [Figure 16] 16 is a flow diagram of an example method 1600 according to some example embodiments. [Figure 17] FIG. 17 is a diagram of an exemplary non-transitory computer-readable medium 1700 in accordance with some exemplary embodiments. [Figure 18] 1 is a diagram of an example device in which some example embodiments may be implemented. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0022] A. Overview The following overview is intended to provide an overview of some image analysis capabilities, including machine learning techniques, that are relevant to some example embodiments.

[0023] FIG. 1 is a diagram of exemplary cell culture images with different confluences, presented as examples of images with different coverages. In FIG. 1, a first cell culture image 100-1 shows low confluence, i.e. low coverage of the field of the first cell culture image 100-1 by cells. A second cell culture image 100-2 shows medium confluence, i.e. medium coverage of the field of the second cell culture image 100-2 by cells. A third cell culture image 100-3 shows high confluence, i.e. high coverage of the field of the third cell culture image 100-3 by cells. In various implementations, confluence can be quantitatively defined as the percentage of the culture dish covered with adherent cells. When referring to a particular image, confluence can be quantitatively defined as the percentage of the culture dish shown in the image that is covered by adherent cells.

[0024] As can be seen by comparing the cell culture images 100-1, 100-2, 100-3 of FIG. 1, an individual's estimation of coverage of each cell culture image may result in varying degrees of accuracy. As a first such example, in the first cell culture image 100-1, the cells are easily and clearly visible and can be easily counted or estimated due to the wide spacing between the cells. Meanwhile, in the third cell culture image 100-3, the cells may be more difficult to distinguish from one another due to the density of the coverage. Note also the difference in background intensity that may confound the implementation of cell counting algorithms.

[0025] As a second such example, in the first cell culture image 100-1, the cells are spaced apart and can be easily identified as cells having a characteristic shape and not as debris or other material in the first cell culture image 100-1, whereas in the third cell culture image 100-3, the density of cells may make it difficult to distinguish the cells from debris or other material in the first cell culture image 100-3. As a result of such difficulties, the accuracy of the coverage estimates of the cell culture images 100-1, 100-2, 100-3 may vary, and may decrease in accuracy, particularly as the confluence of the cell cultures increases.

[0026] Developing an automated process for assessing image confluence can eliminate human error and increase the reproducibility of confluence assessment: for example, different human experts may provide different estimates for the same image, and even a single expert may provide different estimates for the same image at different times.

[0027] 2 is an illustration of an exemplary evaluation of a highly confluent cell culture image according to several image evaluation processes, which underestimate the confluence in some areas of the cell culture image. In the original cell culture image 200-1, some areas 202-1, 202-2 of the cell culture may contain cells in areas of high visual density where it may be difficult to distinguish individual cells from one another and / or from non-cellular material. In a high contrast or binarized (i.e., mapping pixels to one of two binary values, black or white) version of the cell culture image 200-2, the corresponding areas 202-3, 202-4 may be more difficult to characterize as cells, non-cellular material, or voids, as detail in these high density areas is lost, and may be erroneously characterized as non-cells.

[0028] In the evaluated version of cell culture image 200-3, areas 202-5, 202-6 that appear to be cells are now characterized as voids. As a result of this loss of detail and mischaracterization of dense areas of a highly confluent cell culture, the confluence of cell culture image 200-1 may be underestimated. As a result of this underestimation of confluence, further research processes that are undertaken in response to the cell culture corresponding to cell culture image 200-1 reaching a particular developmental stage, as indicated by confluence, may be delayed until a later time than the particular developmental stage. This may result in unexpected results and / or compromise the integrity of the resulting cell culture or data.

[0029] B. Presented Technology Figure 3 is an example of an image processing technique for determining the coverage of an image, according to some example embodiments. Selected portions of Figure 3 are further illustrated in Figures 4-12.

[0030] In the exemplary image processing technique 302 of FIG. 3, an image 300 for which a coverage determination is desired is provided. As an example, the image 300 may be an image of a cell culture, such as a group of cells taken from an individual's tissue sample. For example, the cell culture may represent tissue of unknown composition, such as immune cells, NK-92 cells, T cells, immune competent cells, CHO cells, HEK cells, HeLa cells, healthy tissue, unhealthy tissue such as a tumor, and / or indeterminate tissue that may or may not contain cancer cells. Other such images 300 may include other organic features, such as microorganisms, and / or inorganic features, such as crystalline structures. The features of the image may be macroscopic, such as visible to the individual's eye without aid, or may be microscopic, such as small features visible under optical and / or digital magnification of the image.

[0031] 3, a coverage determination 330 of the coverage of the image 300 is desired. The coverage determined may include, for example, a coverage mask 332, such as a bitmap indicating which areas of the image 300 are covered by cells (e.g., represented by white pixels) and which areas of the image 300 are not covered by cells (e.g., represented by black pixels) or vice versa, a mask input image 334, such as an outlining of the areas of the image 300 that are covered and / or a density (e.g., number of cells / mm ) that may be further evaluated and / or presented to an individual. 2 ), a count (e.g., 250 cells clearly visible within the area of ​​the image), and / or a coverage estimate such as a confluence or coverage estimate 336 based on a measurement of the area of ​​the image 300 depicting cells within the entire area of ​​the image 300, which may be reported to an individual as a percentage of confluence of the covered area of ​​the image 300 relative to the total area of ​​the image 300 (e.g., 60% confluence).

[0032] The exemplary image processing technique 302 of Figure 3 begins with one or more image feature adjustments 304. As a first example, illumination normalization 306 may be applied to the image 300 to normalize the illumination level of the image 300. For example, the brightness, luminosity and / or contrast levels of the image 300 may be adjusted to raise and / or lower the average illumination of the image 300 towards a target average illumination to which the remaining image processing features are well matched. One form of such illumination normalization 306 may include applying a Gaussian blur to a copy of the image 300 and then subtracting the blurred copy from the image 300.

[0033] In various implementations, local contrast enhancement 308 may be applied to the image 300 to promote a target contrast level in each local area of ​​the image 300 to reduce high contrast shifts that may result from visual artifacts such as inconsistent lighting or shadowing. One such form of local contrast enhancement 308 may include applying a contrast-limited adaptive histogram equalization ("CLAHE") process to the image 300. One or more such image feature adjustments 304 may be applied to the image 300 to generate an adjusted image for further processing by image processing techniques 302.

[0034] 4 is a diagram of a first example of image feature adjustment according to some exemplary embodiments. Images 400-1, 400-2, 400-3 of various confluences are processed by an illumination normalization process and contrast enhancement (such as CLAHE). The corresponding adjusted images 402-1, 402-2, 402-3 present higher contrast images with sharper boundaries that can be more accurately evaluated for segmentation.

[0035] 5 is a diagram of a contrast adjustment process according to some exemplary embodiments. In FIG. 5, a local contrast adjustment process such as CLAHE is applied to an image 500 to generate an adjusted image 502 with sharper boundaries that can be more accurately evaluated for segmentation. In various implementations, the CLAHE can be adjusted by a "clipLimit" parameter that is automatically determined based on the average contrast of the image 500.

[0036] The local contrast adjustment shown in Figure 5 may be applied alternatively or together with illumination normalization and / or Gaussian blurring. In various implementations, the local contrast adjustment shown in Figure 5 may be applied before, simultaneously with, and / or after illumination normalization and / or Gaussian blurring.

[0037] Returning to Figure 3, the example image processing technique 302 of Figure 3 may include a first segmentation 310, in which the image 300 (or the generated adjusted image provided by image feature adjustment 304 in some example embodiments) is evaluated to determine edges or boundaries to generate a first segmentation mask 312. The first segmentation 310 may generate a set of descriptors of edges or boundaries of the image 300 or the adjusted image, such as, for example, the location, size and / or shape of high contrast boundaries that may represent visual features such as cells.

[0038] In some exemplary embodiments, the first segmentation 310 may include applying one or more edge detection filters to the image 300. As a first such example, a Sobel X-axis filter may be applied to the image 300 or the adjusted image. The Sobel X-axis filter is configured to detect high contrast differences between horizontally adjacent pixels that may indicate vertical edges. An exemplary 3×3 Sobel X-axis filter is represented by the following filter matrix:

number

[0039] As a second such example, a Sobel Y-axis filter may be applied to image 300 or the adjusted image. The Sobel Y-axis filter is configured to detect high contrast differences between vertically adjacent pixels, which may indicate horizontal edges. An exemplary 3×3 Sobel Y-axis filter is represented by the following filter matrix:

number

[0040] In various implementations, a Sobel X-axis filter and a Sobel Y-axis filter can be applied separately to the image, and then the resulting filtered images can be combined to generate a combined Sobel XY filter. In various implementations, other edge detection mechanisms, such as a Canny edge detector, can be used. See, for example, U.S. Pat. No. 10,229,342, issued Mar. 12, 2019 to Song et al., the entire disclosure of which is incorporated herein by reference. Once edges (ideally of one or more cells) are identified, the edges can be used to determine the geometric properties of the clearly visible cells, such as major axis, minor axis, size, vertex, etc.

[0041] Many such segmentation image processing techniques may be applied to segment the image 300 or the adjusted image based on the image feature adjustment 304 to generate the first segmentation mask 312. Furthermore, in some exemplary embodiments, further image processing may be applied to the image 300 (including the adjusted image) before, during and / or after the first segmentation 310. As a first such example, generating the first segmentation mask 312 may include applying a Gaussian blur to the segmented image 300, for example, to increase smoothness and / or reduce roughness or unevenness of identified edges or boundaries.

[0042] As a second such example, generating the first segmentation mask 312 may include increasing the contrast level of the segmented image 300, such as by applying an intensity threshold to convert the grayscale segmented image into a binarized 1-bit bitmap indicating which areas of the segmented image 300 are within the segmented area (e.g., cells defined by cell boundaries) and which areas of the image 300 are not covered by cells (e.g., void space or non-cellular material) or vice versa. The first segmentation 310 may output the first segmentation mask 312 for further processing.

[0043] FIG. 6 is a diagram of a first portion of an exemplary first segmentation of an adjusted image, according to some exemplary embodiments.

[0044] In the diagram of Figure 6, an adjusted image 600 is received (e.g., after applying image feature adjustment 304 as shown in Figures 4 and 5) and is filtered through a Sobel X-axis filter to produce a first segmented image 602-1 with enhanced vertical edges or boundaries. Alternatively or additionally, the adjusted image 600 is filtered through a Sobel Y-axis filter to produce a second segmented image 602-2 with enhanced horizontal edges or boundaries.

[0045] The first segmented image 602-1 and the second segmented image 602-2 may be combined to generate an image 604 having essentially a combined Sobel XY filter. The image 604 may then be subjected to a Gaussian blur, as described with respect to FIG.

[0046] 7 is a diagram of a second portion of an exemplary first segmentation of an adjusted image, according to some exemplary embodiments. A Gaussian blur is applied to an image, such as image 604 of FIG. 6, to generate a third filtered image 700, characterized by smoother edges or boundaries.

[0047] Then, binarization may be applied (i.e., converting the grayscale third filtered image 700 to a 1-bit bitmap) to generate a high-contrast segmented image, the first segmentation mask 312. In the present disclosure, binarization may be performed using an adaptive threshold that depends on the data of the image. For example, the adaptive threshold may be determined using Otsu's method. Otsu's method may start with a threshold of 0 to minimize information loss within the binarization stage. For example, only the OpenCV implementation of Otsu's method may be used.

[0048] Returning to FIG. 3, the example image processing technique 302 of FIG. 3 may include applying one or more morphological operations 314 to the first segmentation mask 312, each of which may change the shape of one or more visual features of the first segmentation mask 312, to generate a first modified mask 316. In various embodiments, the one or more morphological operations 314 may be applied sequentially in a predetermined order. The idea behind the morphological operators is to "clean" the estimated binary mask from small "white noise" (i.e. pixels incorrectly classified as cells) and "black noise" (i.e. pixels incorrectly classified as background).

[0049] The one or more morphological operations 314 may include a dilation (or dilation) operation, an erosion (or erosion) operation, an opening operation, and a closing operation. Further description of morphological operations can be found in Wikipedia's Mathematical Morphology, available at en.wikipedia.org / wiki / Mathematical_morphology, the entire contents of which are incorporated by reference (as of June 16, 2021).

[0050] In various embodiments, the dilation operation adds one or more white pixels to an edge or boundary of an object. In various embodiments, the erosion operation subtracts one or more white pixels from the boundary of an object. In various embodiments, the open operator is a composite operator that includes an erosion operation followed by a dilation operation, which has the effect of eliminating small areas of white pixels surrounded by black pixels. In various embodiments, the close operator is a composite operator that includes a dilation operation followed by an erosion operation, which has the effect of eliminating small areas of black pixels surrounded by white pixels.

[0051] In various implementations, morphological operations are generally used to "prepare" the image for the second segmentation stage. For example, morphological operations are selected to "clean" the first segmented image, such as by using morphological open and close operations to erase white and black noise, respectively. After such operations, two morphological erosion operations, which have the effect of erasing some pixels (in this case, one per operation) from the edges of the cells, are applied to the resulting image. The result is a modified first segmentation mask with significant erosion of the edges of the cells. The modified mask is then sign-negated and applied to the input image (which may first be subjected to illumination correction and contrast enhancement).

[0052] The effect of modifying and then sign-negating the first segmentation mask to create a new mask for the second segmentation stage is to provide a mask that is more likely to include cell edges in previously unsegmented areas, thereby providing the edge detection (such as Sobel or Canny) with the real edges of the cells instead of the artificial edges provided by the application of the mask (e.g., black areas as shown in 902 in FIG. 9). In various implementations, after the morphological open and close operations, the first segmentation mask is modified with two morphological erosion operations. Since this modified mask is sign-negated, the effect of such erosion is to artificially enlarge the "cell area" to include the real edges of the cells in order to perform the second segmentation in a fairer context (with fewer "artificial edges").

[0053] In various implementations, the set of morphological operations can be integrated into the imaging process method as follows. 0) Get the input image (with illumination normalization and local contrast enhancement) 1) Generate the first segmentation mask 2) Perform a morphological "open" on 1, which in various implementations removes small white noise 3) Perform a morphological "closing" on 2, which in various implementations removes small black noise ("fills" small black "holes" surrounded by white pixels) 4) Perform a morphological "erosion" on 3, which in various implementations shrinks one (white) pixel from the object in 3 5) Perform a morphological "erosion" on 4, which in various implementations shrinks one (white) pixel from the 4 object. 6) Execute pixel-wise NOT (sign negation) of 5 7) Filter the processed input image 0 with a mask 6 by performing a pixel-wise multiplication of 0 and 6. The effect may be to obtain a masked input image in which the actual edges of cells (which were not segmented in the first segmentation stage) are present. 8) Process 7 to get the second segmentation mask

[0054] The choice of morphological operation may be independent of image resolution, image size and cell type. The kernel size may be context dependent, but in various implementations the smallest possible size is used to clean the image from "single pixel noise" while minimizing the introduction of artifactual information.

[0055] A priori knowledge may be used. For example, in various implementations, a circularity filter filters out small circular objects from the final segmentation that are assumed not to be cells. This a priori knowledge may be based on the cells under study. For example, mesenchymal stem cells are elongated in shape and therefore should not appear as small circles. Using this a priori knowledge, objects that do not correspond to the expected average size and elongated shape of cells may be filtered out by an area and circularity filter (see FIG. 13), which recognizes small, approximately round objects and filters them out from the final binary mask. The size and roundness parameters may be experimentally determined for a given type of cell. Further examples of cell types are human muscle fibers and human fibroblasts, both of which are elongated in shape. Visual inspection of the automated results confirmed that the method described in this application works as expected for a variety of cell types, including cells characterized by elongated shapes.

[0056] FIG. 8 is a diagram of an exemplary set of morphological operations that may be applied to a first segmentation mask, according to some exemplary embodiments. In the diagram of FIG. 8, the first segmentation mask 312 of FIG. 7 may be subjected to one or more of several morphological operations. As a first such example, an open morphological operation 800-1 may be applied to the first segmentation mask 312, in which an edge or boundary of the first segmentation mask 312 is expanded and then contracted by one or more pixels. As a second such example, a closed morphological operation 800-2 may be applied to the first segmentation mask 312, in which an edge or boundary of the first segmentation mask 312 is contracted and then expanded by one or more pixels. As a third such example, an erosion morphological operation 800-3 may be applied to the first segmentation mask 312, in which concave and / or convex irregularities in the edges or boundaries of the first segmentation mask 312 are reduced by one or more pixels.

[0057] In some demonstrative embodiments, one or more of the morphological operations 800-1, 800-2, 800-3 may be applied before, simultaneously with, and / or after another one or more of the morphological operations 800-1, 800-2, 800-3 to generate the first modified mask 316 according to some demonstrative embodiments. For example, the morphological operations may be applied successively in a cascade to the first segmentation mask 312.

[0058] Returning to Figure 3, the example image processing technique 302 of Figure 3 may include generating a second modified mask input 320 based on the image 300 (or, optionally, an adjusted image generated by the image feature adjustment 304) and an inverse 318 of the first modified mask 316. For example, if the first modified mask 316 is a 1-bit bitmap in which white pixels represent covered areas and black pixels represent uncovered areas, then the inverse 318 of the first modified mask 316 may include a 1-bit bitmap in which black pixels represent covered areas and white pixels represent uncovered areas, or vice versa.

[0059] Further, in some exemplary embodiments, the second modified mask input 320 may be generated by combining the image 300 (or, optionally, the adjusted image generated by the image feature adjustment 304) and the inverse 318 of the first modified mask 316. In some exemplary embodiments, the second modified mask input 320 may be generated by performing a pixel-wise multiplication (e.g., a pixel-wise AND) of the adjusted image generated by the image feature adjustment 304 and the inverse 318 of the first modified mask 316. In some exemplary embodiments, the second modified mask input 320 may be generated by otherwise combining (such as by adding or averaging) the image 300 (or, optionally, the adjusted image generated by the image feature adjustment 304) and the inverse 318 of the first modified mask 316.

[0060] FIG. 9 is a diagram of a second modified mask input generated in accordance with some example embodiments.

[0061] 9, the first modification mask 316 is inverted to generate an inverted first modification mask 900, for example, by inverting a 1-bit value of a 1-bit bitmap of the first modification mask 316. Furthermore, the inverted first modification mask 900 may be combined with the image 300 (or, optionally, the adjusted image generated by the image feature adjustment 304) to generate a second modification mask input 902, according to some example embodiments.

[0062] Returning to Figure 3, the example image processing technique 302 of Figure 3 may include a second segmentation 322 of the second modified mask input 320, which is evaluated to determine edges or boundaries to generate a second segmentation mask 324. The second segmentation 322 may generate a set of descriptors of the edges or boundaries of the second modified mask input 320, such as, for example, the location, dimensions and / or shape of high contrast boundaries that may represent visual features such as cells.

[0063] In some exemplary embodiments, the second segmentation 322 may include applying one or more edge detection filters to the second modified mask input 320. As a first such example, a Sobel X-axis filter may be applied to the second modified mask input 320, the Sobel X-axis filter configured to detect high contrast differences between horizontally adjacent pixels that may indicate vertical edges. As a second such example, a Sobel Y-axis filter may be applied to the second modified mask input 320, the Sobel Y-axis filter configured to detect high contrast differences between vertically adjacent pixels that may indicate horizontal edges. Many such segmentation image processing techniques may be applied to segment the second modified mask input 320 to generate the second segmentation mask 324.

[0064] Further, in some exemplary embodiments, additional image processing may be applied to the second segmentation mask 324 before, during and / or after the second segmentation 322. As a first such example, generating the second segmentation mask 324 may include, for example, applying a Gaussian blur to the segmented second modified mask input 320 to increase the smoothness and / or reduce roughness or unevenness of identified edges or boundaries. As a second such example, generating the second segmentation mask 324 may include increasing the contrast level of the segmented second modified mask input 320, such as applying an intensity threshold to convert the grayscale segmented image into a binarized 1-bit bitmap indicating which areas of the segmented second modified mask input 320 are within the segmented area (e.g., cells defined by cell boundaries) and which areas of the segmented second modified mask input 320 are not covered by cells (e.g., void space or non-cellular material) or vice versa. The second segmentation 322 may output a second segmentation mask 324 for further processing.

[0065] In some exemplary embodiments, the first segmentation 310 and the second segmentation 322 may be the same or similar. For example, both the first segmentation 310 and the second segmentation 322 may use Sobel X-axis and / or Sobel Y-axis filters having the same or similar size and / or filter value, such that the segmentation of the image 300 by the first segmentation 310 to generate the first segmentation mask 312 is similar to the segmentation of the second modified mask input 320 by the second segmentation 322 to generate the second segmentation mask 324.

[0066] In some other exemplary embodiments, the first segmentation 310 and the second segmentation 322 may be substantially different. In some exemplary embodiments, the first segmentation 310 may be a subset of the second segmentation 322 or vice versa. For example, the first segmentation 310 may include a Gaussian blur and the second segmentation 322 may not include a Gaussian blur or vice versa. In some exemplary embodiments, the first segmentation 310 and the second segmentation 322 may include similar operators with different parameters, such as a Sobel filter with different filter values. In some exemplary embodiments, the operation of the first segmentation 310 and the operation of the second segmentation 322 may be totally or substantially independent.

[0067] 10 is a diagram of a first portion of an exemplary second segmentation of a second modified mask input, according to some exemplary embodiments. In the diagram of FIG. 10, a second modified mask input 902 (e.g., based on the inverted first modified mask 316 and the image 300 or the adjusted image 600) is received and filtered with a Sobel X-axis filter, thereby generating a first segmented image 1000-1 with enhanced vertical edges or boundaries. Alternatively or additionally, the second modified mask input 902 is filtered with a Sobel Y-axis filter, thereby generating a second segmented image 1000-2 with enhanced horizontal edges or boundaries. The images 1000-1 and 1000-2 may be combined to generate a Sobel XY image 1002. The combination may be based on pixel-wise AND, pixel-wise OR, pixel-wise XOR, or another operation that may or may not operate on a pixel-wise basis.

[0068] 11 is a diagram of a second portion of an exemplary second segmentation of a second modified mask input, according to some exemplary embodiments. In the diagram of FIG. 11, a Gaussian blur process is used on the Sobel XY image 1002, etc., to generate a third filtered image 1100 that is characterized by smoother edges or boundaries. Specifically, a Gaussian blur may be applied to a copy of the Sobel XY image 1002 to generate the third filtered image 1100.

[0069] Binarization (i.e., converting the grayscale third filtered image 1100 to a 1-bit bitmap) may then be applied to generate a high-contrast segmented image 1102 that serves as the second segmentation mask 324.

[0070] Returning to FIG. 3, in some example embodiments, the first segmentation mask 312 and the second segmentation mask 324 are combined to form a composite mask 326 .

[0071] Fig. 12 is a diagram of the generation of a composite mask, according to some example embodiments. In Fig. 12, an adjusted image 600 is processed to generate a first segmentation mask 312 (as shown in Fig. 6) and a second segmentation mask (as shown in Fig. 11). A composite mask 1200 may be generated based on the first segmentation mask 312 and the second segmentation mask 324, for example, by pixel-wise addition (e.g., pixel-wise OR) of the first segmentation mask 312 and the second segmentation mask 324.

[0072] Returning to FIG. 3, in some exemplary embodiments, the exemplary image processing technique 302 may include applying a non-cellular filter 328 to the composite mask 326 to filter out non-cellular areas of the composite mask 326.

[0073] FIG. 13 is a diagram of the application of a non-cellular filter to a set of composite masks, according to some exemplary embodiments. In FIG. 13, a set of composite masks 1200 generated from images of various confluences are subjected to a non-cellular filter that excludes non-cellular areas of the composite masks 1200. As a first such example, a first composite mask 1200-1 generated from a low-confluence image is subjected to a non-cellular filter to generate a first filtered composite mask 1300-1. As a second such example, a second composite mask 1200-2 generated from a medium-confluence image is subjected to a non-cellular filter to generate a second filtered composite mask 1300-2. As a third such example, a third composite mask 1200-3 generated from a high-confluence image is subjected to a non-cellular filter to generate a third filtered composite mask 1300-3. As can be seen from inspection of the filtered composite masks, application of the non-cellular filter removes non-cellular areas from each of the composite masks 1200-1, 1200-2, 1200-3 generated from the low, medium and high confluence images.

[0074] Returning to FIG. 3, in some exemplary embodiments, the exemplary image processing technique 302 may include a coverage determination 330 of the coverage of the image 300 based on the first segmentation mask 312 and the second segmentation mask 324 .

[0075] As further shown in the exemplary image processing technique 302 of FIG. 3, many types of coverage determinations 330 may be generated based on a composite mask 326 generated based on the first segmentation mask 312 and the second segmentation mask 324. As a first example, the coverage determination 330 may include generating a coverage mask 332, such as a 1-bit bitmap, indicating areas of the image 300 that are determined to be covered. The coverage mask 332 may be presented to an individual, such as a user of the exemplary image processing technique 302, and / or may be used in further image processing. As a second such example, the coverage determination 330 may include bitwise multiplication of the coverage mask 332, the image 300 (or an adjustment image, optionally based on one or more image feature adjustments 304), to generate a mask input image 334 as an output image, such that covered areas of the image 300 are indicated and uncovered areas of the image 300 are not indicated.

[0076] Other such mask input images 334 may be generated and presented. For example, the composite mask 326 may indicate edges or boundaries between covered and uncovered areas of the image 300, and a mask input image 334 may be generated in which the edges or boundaries are drawn, highlighted, colored, etc. to define the covered and uncovered areas of the image 300. As a third such example, the coverage determination 330 may include generating a coverage estimate 336, such as a calculation of the confluence of cells in the image 300 of the tissue sample. The coverage estimate 336 may include, for example, one or more of measurements of areas of the image 300 depicting cells within the total area of ​​the image 300, which may include density (e.g., cells / mm 2 ), a count (e.g., 250 cells clearly visible within the area of ​​the image) and / or a percentage of confluence of the covered area of ​​the image 300 relative to the total area of ​​the image 300 (e.g., 60% confluence).

[0077] The elements of the image processing technique 302 of Figure 3 may vary without departing significantly from the presently disclosed technique. In some exemplary embodiments, one or more of the elements of the image processing technique 302 of Figure 3 may be combined (e.g., the morphological operation 314 and the inversion 318 may be performed simultaneously as a single step). In some exemplary embodiments, one or more of the elements of the image processing technique 302 of Figure 3 may be split into two or more separate elements (e.g., performing a first morphological operation 314, such as an open operation, followed by a second morphological operation 314, such as a close operation, in sequence).

[0078] In some exemplary embodiments, one or more of the elements of the image processing techniques 302 of FIG. 3 may be performed conditionally (e.g., apply the open morphological operation 314 only on the condition that the edge of the visual feature of the first segmentation mask 312 exceeds a roughness threshold). In some exemplary embodiments, one or more of the elements of the image processing techniques 302 of FIG. 3 may overlap (e.g., apply the same, similar, and / or different morphological operation 314 not only to the first segmentation mask 312 after the first segmentation 310 but also to the second segmentation mask 324 after the second segmentation 322) or may be presented in a different arrangement (e.g., perform one or more of the one or more morphological operations 314 simultaneously with the first segmentation 310 and / or after the second segmentation 322). Many such variations of the elements of the image processing techniques 302 of FIG. 3 that may include features of the techniques disclosed herein will be understood by those of skill in the art.

[0079] C. Experimental Results FIG. 14 is an illustration of a set of images generated by applying image processing techniques to a cell culture, according to some illustrative embodiments.

[0080] As shown in Figure 14, a first "ground truth" image 1400-1 of a low-confluence cell culture is provided, which is a low-confluence image labeled by a trained human labeler, resulting in a confluence estimate of 5.93%. A first mask input image is generated by the techniques presented herein, resulting in a confluence estimate of 8.33%. A first comparison 1402-1 of the first mask input image and the first "ground truth" image 1400-1 is shown with true positive cells shown in yellow, false positive cells shown in green, false negative cells shown in red, and true negative areas of the image shown in black.

[0081] As further shown in FIG. 14, a medium confluence image labeled by a skilled human labeler is provided as a second “ground truth” image 1400-2 of the medium confluence cell culture, resulting in a confluence estimate of 48.94%. A second mask input image is generated by the techniques presented herein, resulting in a confluence estimate of 60.47%. A second comparison 1402-2 of the first mask input image and the first “ground truth” image 1400-2 is shown with true positive cells shown in yellow, false positive cells shown in green, false negative cells shown in red, and true negative areas of the image shown in black.

[0082] As further shown in FIG. 14, a high confluence image labeled by a skilled human labeler is provided as a third “ground truth” image 1400-3 of the high confluence cell culture, resulting in a confluence estimate of 99.22%. A third mask input image is generated by the techniques presented herein, resulting in a confluence estimate of 95.08%. A third comparison 1402-3 of the third mask input image and the third “ground truth” image 1400-3 is shown with true positive cells shown in yellow, false positive cells shown in green, false negative cells shown in red, and true negative areas of the image shown in black.

[0083] As seen in comparisons 1400-1, 1400-2, 1400-3 of Figure 14, the high percentage of yellow (true positive) and black (true negative) areas indicates a strong correspondence between the ground truth images and the mask input images generated by the techniques presented herein. In particular, the high correspondence in the third comparison 1402-3 and the agreement of the confluence estimates and mask input images generated by a highly confluent expert human labeler demonstrates a coverage accuracy estimate with high accuracy even for images of very high confluence.

[0084] D. Exemplary Embodiments FIG. 15 is a component block diagram of an example apparatus according to some example embodiments.

[0085] As shown in Figure 15, some example devices 1500 may include a processing circuit 1502 and a memory 1504. The memory 1504 may store instructions 1506 that, when executed by the processing circuit 1502, cause the example device 1500 to determine coverage of an image 300. In some example embodiments, execution of the instructions 1506 may cause the example device 1500 to instantiate and / or use a set of components of a system 1508. Although Figure 15 illustrates one such system 1508, some example embodiments may embody any of the methods disclosed herein.

[0086] The image 300 may be provided by an individual 1520, such as a user of the exemplary device 1500. The individual 1520 may acquire the image 300 from another system or may instruct an image acquisition system to acquire the image 300 and provide the image 300 to the device 1500. In various embodiments, the device 1500 may include some or all of an image acquisition system. The image acquisition system may include a set of lights, a set of lenses, and a set of image sensors that provide manual and / or automatic control. For example, the illumination may be determined automatically using a light meter, and the image focus may be set automatically using an autofocus system. The image acquisition may also be automated, such as performed at scheduled intervals. The interval may be dynamic based on the observed coverage of the subject, e.g., the interval may be longer if the coverage is slowly increasing. Additionally or alternatively, the interval may be shorter as the measured coverage approaches the desired coverage.

[0087] The example system 1508 of Figure 15 includes a first segmenter 1510 that generates a first segmentation mask 312 by segmenting the image 300. Examples of such first segmentations are shown, for example, in Figures 6 and 7.

[0088] The example system 1508 of Figure 15 includes a morphological operator 1512 that applies a morphological operation 314 to the first segmentation mask 312 to generate a first modified mask 316. Examples of such morphological operations 314 are shown, for example, in Figure 8.

[0089] The example system 1508 of Figure 15 includes an inverter 1514 that generates a second modified mask input 320 based on the image 300 and the inverse of the first modified mask 316. An example of such a second modified mask input generation is shown, for example, in Figure 9.

[0090] The example system 1508 of Figure 15 includes a second segmenter 1516 that generates a second segmentation mask 324 by segmenting the second modified mask input 320. Examples of such second segmentations are shown, for example, in Figures 10 and 11.

[0091] The example system 1508 of FIG. 15 includes a coverage determiner 1518 that determines a coverage of the image 300 based on the first segmentation mask 312 and the second segmentation mask 324 .

[0092] As shown in Figure 15, some example devices 1500 may include processing circuitry 1502 capable of executing instructions. The processing circuitry 1502 may include hardware including logic circuits; a hardware / software combination such as a processor executing software; or a combination thereof, etc. For example, a processor may include, but is not limited to, a central processing unit (CPU), a graphic processing unit (GPU), an arithmetic logic unit (ALU), a digital signal processor, a microcomputer, a field programmable gate array (FPGA), a system on a chip (SoC), a programmable logic unit, a microprocessor, an application specific integrated circuit (ASIC), etc.

[0093] As further shown in FIG. 15, some example devices 1500 may include memory 1504 having instructions 1506 stored therein. Memory 1504 may include, for example, random access memory (RAM), read only memory (ROM), erasable programmable read only memory (EPROM), electrically erasable programmable read only memory (EEPROM), etc. Memory 1504 may be volatile, such as system memory, and / or non-volatile, such as a hard disk drive, a solid state storage device, flash memory, or magnetic tape. The instructions 1506 stored in memory 1504 may be specified as assembly instructions and / or machine language (e.g., binary) instructions according to a native instruction set architecture of the processor, such as a variant of the IA-32 instruction set architecture or a variant of the ARM instruction set architecture; instructions of a high-level imperative language and / or a declarative language that are compilable and / or interpretable to be executed on the processor; and / or instructions that are compilable and / or interpretable to be executed by a virtual processor of a virtual machine, such as a web browser. A non-limiting set of examples of such high level languages ​​may include, for example, C, C++, C#, Objective-C, Swift, Haskell, Go, SQL, R, Lisp, Java, Fortran, Perl, Pascal, Curl, OCaml, JavaScript, HTML5 (Hypertext Markup Language 5th revision), Ada, ASP (Active Server Pages), PHP (PHP: Hypertext Preprocessor), Scala, Swift, Eiffel, Smalltalk, Erlang, Ruby, Flash, Visual Basic, Lua, MATLAB, SIMULINK, and Python. Such instructions 1506 may also include instructions for libraries, resources, platforms, application programming interfaces (APIs), etc., utilized in determining clinical value (e.g., prognosis) for an individual based on tumors depicted in images.

[0094] As shown in FIG. 15, some exemplary systems 1508 may be organized in a particular manner, for example, to assign some functionality to each component of the system. Some exemplary embodiments may implement such components in various ways, such as software, hardware (e.g., processing circuitry), or a combination thereof. In some exemplary embodiments, the organization of the system may be different as compared to some other exemplary embodiments, including the exemplary system 1508 shown in FIG. 15. For example, some exemplary embodiments may include systems featuring different organization of components, such as renaming, rearranging, adding, splitting, overlapping, merging, and / or removing components, sets of components, and relationships therebetween, without departing from the scope of the present disclosure. All such variations that are reasonably technically and logically possible and not inconsistent with other descriptions are intended to be included in the present disclosure, the scope of which is to be understood as limited only by the claims.

[0095] FIG. 16 is a flow diagram of an example method 1600 according to some example embodiments.

[0096] The exemplary method 1600 may be implemented, for example, as a set of instructions that, when executed by a processing circuitry of an apparatus, cause the apparatus to perform each element of the exemplary method 1600. The exemplary method 1600 may also be implemented, for example, as a set of instructions that, when executed by a processing circuitry of an apparatus, cause the apparatus to provide a system of components such as a first segmenter, a morphological operator, an inverter, a second segmenter, and a coverage determiner.

[0097] The exemplary method 1600 includes executing (1602), by a processing circuit of the device, instructions that cause the device to perform a set of elements. For example, executing the instructions may cause the device to generate (1604) a first segmentation mask 312 by segmenting the image 300. Executing the instructions may cause the device to generate (1606) a first modified mask 316 by applying a morphological operation 314 to the first segmentation mask 312. Executing the instructions may cause the device to generate (1608) a second modified mask input 320 based on the image 300 and an inverse 318 of the first modified mask 316. Executing the instructions may cause the device to generate (1610) a second segmentation mask 324 by segmenting the second modified mask input 320. Execution of the instructions may cause the apparatus to determine 1612 a coverage determination 330 for the image 300 based on the first segmentation mask 312 and the second segmentation mask 324. In this manner, execution of the instructions by the processing circuitry may cause the apparatus to perform elements of the example method 1600, thereby causing the example method 1600 to end.

[0098] FIG. 17 is a diagram of an example computer-readable medium 1700 according to some example embodiments.

[0099] As shown in FIG. 17, a non-transitory computer-readable medium 1700 may store binary data 1702 that encodes a set of instructions 1704 that, when executed by a processing circuit 1502 of the exemplary apparatus 1500, causes the exemplary apparatus 1500 to perform operations 1706 including determining a coverage determination 330 of the image 300. As a first such example, the instructions 1704 may encode components of the exemplary apparatus 1500 of FIG. 15. As a second such example, the instructions 1704 may encode components of the exemplary system 1508 of FIG. 15. As a third such example, the instructions 1704 may encode elements of an exemplary method, such as the exemplary method 1600 of FIG. 16.

[0100] In some exemplary embodiments, the system may include an image evaluation means for determining a coverage of the image 300. The system may include a first segmentation means for generating a first segmentation mask 312 by segmenting the image 300, a first modified mask generation means for generating a first modified mask 316 by applying a morphological operation to the first segmentation mask 312, a second modified mask input generation means for generating a second modified mask input 320 based on the image 300 and an inverse 318 of the first modified mask 316, a second segmentation means for generating a second segmentation mask 324 by segmenting the second modified mask input 320, and a determination means for determining a coverage determination 330 of the image 300 based on the first segmentation mask 312 and the second segmentation mask 324.

[0101] E. Technical Effects Some exemplary embodiments of the present disclosure may exhibit one or more technical advantages.

[0102] As a first such example, some exemplary embodiments may enable processing of images 300, such as images of cell cultures, in a faster and / or more efficient manner than other techniques, including inspection by skilled human labelers. For example, components of one or more exemplary embodiments may be implemented in a parallel, distributed, streaming, and / or pipelined architecture such that images 300 may be rapidly processed. As an example, such rapid evaluation of images 300 may enable cell cultures in a research environment to be timely evaluated for cell culture populations and / or provide rapid coverage determinations once it has been determined that cells in the cell culture have reached a selected stage of maturity as indicated by the study, such that those cells may be rapidly subjected to further study.

[0103] As a second such example, some exemplary embodiments may enable images 300, such as images of cell cultures, to be processed in a more consistent manner than other techniques involving inspection by skilled human labelers. For example, by avoiding decisions based on the judgment of a team of skilled human labelers, who may have different criteria for coverage or confluence and / or different accuracy rates in assessing images, some exemplary embodiments of the presented techniques may apply more consistent criteria to a series of images, resulting in more consistent coverage assessments. Furthermore, some exemplary embodiments of the presented techniques may exhibit less variance over time, as may be exhibited by the judgment of skilled human labelers over time, resulting in less variance in coverage decisions for similar images 300 over time.

[0104] As a third such example, some exemplary embodiments may enable an image 300, such as an image of a cell culture, to be processed in a more accurate manner than other techniques, including inspection by a skilled human labeler. For example, particularly in high confluence images of cell cultures, distinguishing between cells or between cells and non-cellular material may be difficult in dense areas of the image 300. Many techniques, including some skilled human labelers, may underestimate coverage determinations of the image 300 due to such difficulties, as shown in FIG. 2. In contrast, as shown in the techniques presented herein (e.g., FIG. 14), some exemplary embodiments may exhibit a high degree of correspondence between coverage determinations and matching labeled ground truth images 300. Many such technical effects may be exhibited by some exemplary embodiments.

[0105] F. Exemplary Computing Environment FIG. 18 is a diagram of an example device in which some example embodiments may be implemented.

[0106] Figure 18 and the following discussion provide a brief general description of a suitable computing environment for implementing one or more embodiments of the present disclosure described herein. The operating environment of Figure 18 is only one example of a suitable operating environment and is not intended to suggest any limitations on the scope of use or functionality of the operating environment. Exemplary computing devices include, but are not limited to, personal computers, server computers, handheld or laptop devices, mobile devices (e.g., mobile phones, personal digital assistants (PDAs), media players), multiprocessor systems, media devices such as televisions, home appliances, embedded devices, minicomputers, mainframe computers, distributed computing environments including any of the above systems or devices, wearable computing devices (e.g., glasses, earphones, watches, rings, pendants, handheld and / or body mounted cameras, clothing integrated devices and embeddable devices), autonomous vehicles, extended reality (XR) devices such as augmented reality (AR) devices and / or virtual reality (VR) devices, Internet of Things (IoT) devices, and the like.

[0107] Some exemplary embodiments may include combinations of components of the same and / or different types, such as multiple processors and / or processing cores in a uni- or multi-processor computer; two or more processors operating in tandem, such as a CPU and a GPU; a CPU utilizing an ASIC; and / or software executed by the processing circuitry. Some exemplary embodiments may include components of a single device, such as a computer including one or more CPUs that store, access, and manage a cache. Some exemplary embodiments may include components of multiple devices, such as two or more devices with CPUs that communicate to access and / or manage a cache. Some exemplary embodiments may include one or more components included in a server computing device, a server computer, a series of server computers, a server farm, a cloud computer, a content platform, a mobile computing device, a smartphone, a tablet, or a set-top box. Some exemplary embodiments may include components that communicate directly (e.g., two or more cores of a multi-core processor) and / or indirectly (e.g., via a bus, via a wired or wireless channel or network, and / or via an intermediate component such as a microcontroller or arbiter). Some exemplary embodiments may include multiple instances or instances of a system executed by devices or components, respectively, and such instances of a system may be executed simultaneously, sequentially, and / or in an interleaved manner. Some exemplary embodiments may feature a distribution of instances or systems across two or more devices or components.

[0108] Although not required, some exemplary embodiments are described in the general context of "computer-readable instructions" being executed by one or more computing devices. The computer-readable instructions may be distributed via a computer-readable medium (described below). The computer-readable instructions may be implemented as program modules, such as functions, objects, application programming interfaces (APIs), data structures, etc. that perform particular tasks or implement particular abstract data types. Typically, the functionality of the computer-readable instructions may be combined or distributed as desired in various environments.

[0109] 18 illustrates one example of an exemplary device 1800 configured as or to include one or more exemplary embodiments, such as the exemplary embodiments provided herein. In one device configuration 1802, the exemplary device 1800 may include a processing circuit 1502 and a memory 1804. Depending on the exact configuration and type of computing device, the memory 1804 may be volatile (e.g., RAM, etc.), non-volatile (e.g., ROM, flash memory, etc.), or some combination of the two.

[0110] In some exemplary embodiments, exemplary device 1800 may include additional features and / or functionality. For example, exemplary device 1800 may also include additional storage (e.g., removable and / or non-removable) including, but not limited to, magnetic storage, optical storage, etc. Such additional storage is illustrated in FIG. 18 by storage 1806. In some exemplary embodiments, computer-readable instructions for implementing one or more embodiments provided herein may be stored in memory 1804 and / or storage 1806.

[0111] In some exemplary embodiments, storage 1806 may be configured to store other computer readable instructions for implementing an operating system, application programs, and the like. The computer readable instructions may be loaded into memory 1804 for execution by processing circuit 1502, for example. Storage may include volatile and non-volatile, removable and non-removable media implemented in any method or technology for storing information such as computer readable instructions or other data. Storage may include, but is not limited to, RAM, ROM, EEPROM, flash memory or other memory technology, CD-ROM, digital versatile disk (DVD) or other optical storage, magnetic cassettes, magnetic tape, magnetic disk storage or other magnetic storage devices, or any other medium that can be used to store the desired information and that can be accessed by exemplary device 1800. Such computer storage media may be part of exemplary device 1800.

[0112] In some exemplary embodiments, the exemplary device 1800 may include one or more input devices 1810, such as a keyboard, a mouse, a pen, a voice input device, a touch input device, an infrared camera, a video input device, and / or any other input device. One or more output devices 1808, such as one or more displays, speakers, a printer, and / or any other output device, may also be included in the exemplary device 1800. The one or more input devices 1810 and the one or more output devices 1808 may be connected to the exemplary device 1800 by a wired connection, a wireless connection, or any combination thereof. In some exemplary embodiments, an input device or an output device from another computing device may be used as the one or more input devices 1810 or the one or more output devices 1808 for the exemplary device 1800.

[0113] In some exemplary embodiments, the exemplary device 1800 may be connected by various interconnects, such as a bus. Such interconnects may include Peripheral Component Interconnects (PCI), such as PCI Express, Universal Serial Bus (USB), Firewire (IEEE 1394), optical bus structures, and the like. In other exemplary embodiments, the components of the exemplary device 1800 may be interconnected by a network. For example, the memory 1804 may include multiple physical memory units located in different physical locations interconnected by a network.

[0114] In some exemplary embodiments, the exemplary apparatus 1800 may include one or more communication devices 1812 through which the exemplary apparatus 1800 can communicate with other devices. The one or more communication devices 1812 may include, for example, a modem, a network interface card (NIC), an integrated network interface, a radio frequency transceiver, an infrared port, a USB connection, or other interfaces for connecting the exemplary apparatus 1800 to other computing devices, including remote devices 1816. The one or more communication devices 1812 may include wired or wireless connections. The one or more communication devices 1812 may be configured to transmit and / or receive communication media.

[0115] Those skilled in the art will appreciate that storage devices used to store computer-readable instructions may be distributed across a network. For example, the exemplary apparatus 1800 may communicate with a remote device 1816 via a network 1814 to store and / or retrieve computer-readable instructions for implementing one or more exemplary embodiments provided herein. For example, the exemplary apparatus 1800 may be configured to access the remote device 1816 and download some or all of the computer-readable instructions for execution. Alternatively, the exemplary apparatus 1800 may be configured to download some of the computer-readable instructions as needed, and some instructions may be executed in or by the exemplary apparatus 1800, and other some instructions may be executed in or by the remote device 1816.

[0116] In this application, including the definitions below, the term “module” or “controller” may be interchangeable with the term “circuitry.” The term “module” may refer to, be part of, or include processing circuitry 1502 (shared, dedicated, or group) that executes code and memory circuitry / hardware (shared, dedicated, or group) that stores code executed by the processing circuitry 1502.

[0117] The module may include one or more interface circuits. In some examples, the one or more interface circuits may implement a wired or wireless interface to connect to a local area network (LAN) or a wireless personal area network (WPAN). Examples of LANs include the Institute of Electrical and Electronics Engineers (IEEE) standard 802.11-2016 (also known as the WIFI wireless networking standard) and the IEEE standard 802.3-2015 (also known as the ETHERNET wired networking standard). Examples of WPANs include the IEEE standard 802.15.4 (including the ZIGBEE standard by the ZigBee Alliance) and the BLUETOOTH wireless networking standard by the Bluetooth Special Interest Group (SIG) (including Core Specification versions 3.0, 4.0, 4.1, 4.2, 5.0 and 5.1 by the Bluetooth SIG).

[0118] Modules may communicate with other modules using one or more interface circuits. Although in this disclosure modules may be depicted as logically communicating directly with other modules, in various implementations modules may actually communicate through a communication system. A communication system includes physical and / or virtual network equipment such as hubs, switches, routers and gateways. In some implementations, a communication system connects to or traverses a wide area network (WAN) such as the Internet. For example, a communication system may include multiple LANs connected to each other via the Internet or point-to-point leased lines using technologies including Multiprotocol Label Switching (MPLS) and Virtual Private Networks (VPNs).

[0119] In various implementations, the functionality of a module may be distributed among multiple modules connected via a communication system. For example, multiple modules may implement the same functionality distributed by a load balancing system. In a further example, the functionality of a module may be split between a server (also known as remote or cloud) module and a client (or user) module.

[0120] The term code as used above may include software, firmware, and / or microcode and may refer to programs, routines, functions, classes, data structures, and / or objects. The shared processing circuit 1502 may include a single microprocessor that executes some or all code from multiple modules. The group processing circuit 1502 may include a microprocessor that executes some or all code from one or more modules in combination with additional microprocessors. References to multiple microprocessors include multiple microprocessors on separate dies, multiple microprocessors on a single die, multiple cores of a single microprocessor, multiple threads of a single microprocessor, or combinations of the above.

[0121] Shared memory hardware encompasses a single memory device that stores some or all code from multiple modules. Group memory hardware encompasses memory devices that, in combination with other memory devices, store some or all code from one or more modules.

[0122] The term memory hardware is a subset of the term computer-readable medium. As used herein, the term computer-readable medium does not encompass transitory electrical or electromagnetic signals propagating through a medium (such as on a carrier wave), and thus the term computer-readable medium is considered to be tangible and non-transitory. Non-limiting examples of non-transitory computer-readable media are non-volatile memory devices (such as flash memory devices, erasable programmable read-only memory devices or masked read-only memory devices), volatile memory devices (such as static random access memory devices or dynamic random access memory devices), magnetic storage media (such as analog or digital magnetic tape or hard disk drives), and optical storage media (such as CDs, DVDs, or Blu-ray disks).

[0123] Exemplary embodiments of the apparatus and methods described herein may be implemented partially or completely by a special purpose computer created by configuring a general purpose computer to perform one or more specific functions embodied in a computer program. The functional blocks and flow chart elements described herein may function as software specifications that can be converted into a computer program by the routine work of a skilled technician or programmer.

[0124] A computer program includes processor-executable instructions stored on at least one non-transitory computer-readable medium. A computer program may also include or depend on stored data. A computer program may include a basic input / output system (BIOS) that interacts with the hardware of a special-purpose computer, device drivers that interact with specific devices of a special-purpose computer, one or more operating systems, user applications, background services, background applications, and the like.

[0125] A computer program may include (i) parsed text, such as HTML (HyperText Markup Language), XML (Extensible Markup Language), or JSON (JavaScript Object Notation); (ii) assembly code; (iii) object code generated from source code by a compiler; (iv) source code for execution by an interpreter; or (v) source code for compilation and execution by a just-in-time compiler. By way of example only, the source code may be written using syntax from languages ​​including C, C++, C#, Objective-C, Swift, Haskell, Go, SQL, R, Lisp, Java®, Fortran, Perl, Pascal, Curl, OCaml, JavaScript®, HTML5 (Hypertext Markup Language 5th revision), Ada, ASP (Active Server Pages), PHP (PHP: Hypertext Preprocessor), Scala, Eiffel, Smalltalk, Erlang, Ruby, Flash®, Visual Basic®, Lua, MATLAB®, SIMULINK®, and Python®.

[0126] G. Use of Terms The foregoing description is merely exemplary in nature and is in no way intended to limit the disclosure, its application, or uses. The broad teachings of the disclosure can be embodied in various forms. Thus, while the disclosure includes certain examples, the true scope of the disclosure is not so limited, as other variations will become apparent upon review of the drawings, the specification, and the following claims. It is to be understood that one or more steps in a method may be performed in a different order (or simultaneously) without altering the principles of the disclosure. Furthermore, although each embodiment is described above as having certain features, any one or more of those features described with respect to any embodiment of the disclosure may be implemented in and / or combined with the features of any other exemplary embodiment (even if the combination is not explicitly described). That is, the described embodiments are not mutually exclusive, and substitution of one or more embodiments for one another remains within the scope of the disclosure.

[0127] Spatial and functional relationships between elements (e.g., between modules) are described using various terms, including "connected," "engaged," "interfaced," and "coupled." Unless expressly described as "direct," when a relationship between a first element and a second element is described in the disclosure above, the relationship encompasses a direct relationship where there is no other intervening element between the first element and the second element, and also encompasses an indirect relationship where there is one or more intervening elements (spatial or functional) between the first element and the second element. In this specification, the phrase at least one of A, B, and C is to be interpreted as meaning a logical formula (A, or B, or C) with a non-exclusive OR, and not as meaning "at least one of A, at least one of B, and at least one of C."

[0128] In a diagram, the direction of the arrow, indicated by the arrowhead, generally indicates the flow of information (such as data or instructions) of interest to the diagram. For example, if element A and element B exchange various information, but information sent from element A to element B is relevant to the diagram, the arrow may point from element A to element B. This unidirectional arrow does not imply that no other information is sent from element B to element A. Furthermore, in response to information sent from element A to element B, element B may send a request for information or an acknowledgment of receipt to element A. The term subset does not necessarily require a proper subset; that is, a first subset of a first set may be comparable (equal) to the first set.

[0129] Although the present subject matter has been described in language specific to structural features and / or methodological acts, it is to be understood that the subject matter defined in the appended claims is not necessarily limited to the specific features or acts described above. Rather, the specific features and acts described above are disclosed as example forms of implementing the claims.

[0130] As used herein, terms such as "component," "module," "system," "interface," and the like are intended to generally refer to a computer-related entity, whether hardware, a combination of hardware and software, software, a software suite, or software in execution. For example, a component may be, but is not limited to, a process running on a processing circuit 1502, the processing circuit 1502, an object, an executable, a thread of execution, a program, and / or a computer. By way of example, both an application running on a controller and the controller may be a component. One or more components may reside within a process and / or thread of execution, and a component may be localized on one computer and / or distributed between two or more computers.

[0131] Additionally, some exemplary embodiments may include methods, apparatus, or articles of manufacture using standard programming and / or engineering techniques to generate software, firmware, hardware, or any combination thereof to control a computer to implement the disclosed subject matter. As used herein, the term "article of manufacture" is intended to encompass a computer program accessible from any computer-readable device, carrier, or media. Of course, those skilled in the art will recognize that many modifications to this configuration can be made without departing from the scope or spirit of the claimed subject matter.

[0132] Various operations of the embodiments are provided herein. In some exemplary embodiments, one or more of the described operations may constitute computer-readable instructions stored on one or more computer-readable media that, when executed by a computing device, cause the computing device to perform the described operations. The order in which some or all of the operations are described should not be construed to suggest that these operations are necessarily order dependent. Alternative orderings will be understood by those of ordinary skill in the art having the benefit of this specification. Additionally, it will be understood that not all operations are necessarily present in each exemplary embodiment provided herein.

[0133] As used herein, the term "or" is intended to mean an inclusive "or" rather than an exclusive "or." That is, unless otherwise specified or clear from the context, "X employs A or B" is intended to mean any of the natural inclusive permutations. That is, if X employs A, if X employs B, or if X employs both A and B, then "X employs A or B" is satisfied in any of the foregoing examples. The articles "a" and "an" as used in this specification and the appended claims may be construed generally to mean "one or more," unless otherwise specified or clear from the context to be directed to the singular form.

[0134] Although the present disclosure has been shown and described with respect to several exemplary embodiments, equivalent modifications and alterations will occur to those skilled in the art based on reading and understanding the present specification and the accompanying drawings. The present disclosure includes all such modifications and alterations and is limited only by the scope of the following claims. In particular, with respect to the various functions performed by the above components (e.g., elements, resources, etc.), the terms used to describe such components are intended to correspond to any component that performs the specified function of the described component (e.g., is functionally equivalent), even if it is not structurally equivalent to the disclosed structures that perform the function in several exemplary embodiments shown herein of the present disclosure, unless otherwise specified. In addition, although a particular feature of the present disclosure may be disclosed with respect to only one of several implementations, such feature may be combined with one or more other features of other implementations as may be desirable and advantageous for any given application or specific application. Furthermore, to the extent that the terms "including," "having," "having," "comprising," or variations thereof are used in either the detailed description or the claims, such terms are intended to be as inclusive as the term "including."

Claims

1. 1. A method of operating an apparatus including a processing circuit used to determine confluence of cells in an image, comprising: The processing circuitry causes the device to generating a first segmentation mask by segmenting the image; generating a modified mask by applying a morphological operation to the first segmentation mask; generating a modified mask input based on the image and an inverse of the modified mask; generating a second segmentation mask by segmenting the modified mask input; Executing the instructions further comprises causing the device to: combining the first segmentation mask and the second segmentation mask to generate a composite mask; The method, wherein determining the confluence in the image includes presenting the image masked with the composite mask as a view of non-cellular areas of the image.

2. Executing the instructions further comprises causing the device to: adjusting a characteristic of the image to generate an adjusted image; The method of claim 1 , wherein generating the first segmentation mask comprises segmenting the adjustment image.

3. The method of claim 2 , wherein adjusting the characteristics of the image comprises normalizing an illumination level of the image.

4. 4. The method of claim 3, wherein normalizing the illumination level of the image comprises applying a Gaussian blur to the image and subtracting the blurred image from the image to generate a blurred image.

5. The method of claim 2 , wherein adjusting the characteristics of the image comprises increasing a local contrast level of the image.

6. The method of claim 5 , wherein increasing the local contrast level of the image comprises applying contrast-limited adaptive histogram equalization to the image.

7. The method of claim 1 , wherein generating the first segmentation mask comprises segmenting the image based on an edge filter.

8. The method of claim 1 , wherein generating the first segmentation mask comprises applying a Gaussian blur to the image.

9. The method of claim 1 , wherein generating the first segmentation mask comprises increasing a contrast level of the image.

10. The morphological manipulation may include: Open morphological operations, Close morphological operations, Dilation morphological operations, and Erosion morphological manipulation The method of claim 1 , comprising at least one of:

11. The method of claim 1 , wherein segmenting the modified mask input comprises segmenting the modified mask input based on an edge filter.

12. The method of claim 1 , wherein segmenting the modified mask input comprises applying a Gaussian blur to the modified mask input.

13. The method of claim 1 , wherein segmenting the modified mask input comprises increasing a contrast level of the modified mask input.

14. The method of claim 1 , wherein generating the composite mask comprises applying a non-cellular filter to the composite mask to exclude non-cellular areas of the composite mask.

15. excluding the non-cellular areas of the composite mask includes: the morphology of the non-cellular areas, and The size of the non-cellular area The method of claim 14 based on at least one of the following:

16. The method of claim 1 , wherein determining the confluence comprises estimating a non-cellular area of ​​the composite mask.

Images